pyspotlight 0.7.2

Installation

The newest stable release can be found on the Python Package Index (PyPI).

Therefore installation is as easy as:

pip install pyspotlight

Older releases can be installed by specifying a version:

pip install pyspotlight~=0.6.1

Requirements for installation from source/github

This module has been tested with Python 2.7 and Python 3.5.

As long as you use the setup.py for the installation (python setup.py install), you’ll be fine because Python takes care of the dependencies for you.

If you decide not to use the setup.py you will need the requests library.

All of these packages can be found on the Python PackageIndex and easily installed via either easy_install or, the recommended, pip.

Using pip it is especially easy because you can just do this:

pip install -r requirements.txt

and it will install all package dependencies listed in that file.

Usage

Usage is simple and easy, just as the API is:

>>> import spotlight
>>> annotations = spotlight.annotate('http://localhost/rest/annotate',
...                                  'Your test text',
...                                  confidence=0.4, support=20)

This should return a list of all resources found within the given text. Assuming we did this for the following text:

President Obama on Monday will call for a new minimum tax rate for individuals making more than $1 million a year to ensure that they pay at least the same percentage of their earnings as other taxpayers, according to administration officials.

We might get this back:

>>> spotlight.annotate('http://localhost/rest/annotate', sample_txt)
[
  {
    'URI': 'http://dbpedia.org/resource/Presidency_of_Barack_Obama',
    'offset': 0,
    'percentageOfSecondRank': -1.0,
    'similarityScore': 0.10031112283468246,
    'support': 134,
    'surfaceForm': 'President Obama',
    'types': 'DBpedia:OfficeHolder,DBpedia:Person,Schema:Person,Freebase:/book/book_subject,Freebase:/book,Freebase:/book/periodical_subject,Freebase:/media_common/quotation_subject,Freebase:/media_common'
  },
  …(truncated remaining elements)…
]

Any additional filter parameters that are supported by the Spotlight API can be passed to the filters argument in a dictionary.

For example:

>>> only_person_filter = {
...     'policy': "whitelist",
...     'types': "DBpedia:Person",
...     'coreferenceResolution': False
... }

>>> spotlight.annotate(
...     "http://localhost/rest/annotate",
...     "Any collaboration between Shakira and Metallica seems highly unlikely.",
...     filters=only_person_filter
... )

[{
    'URI': 'http://dbpedia.org/resource/Shakira',
    'offset': 26,
    'percentageOfSecondRank': 1.511934771738109e-09,
    'similarityScore': 0.9999999984880361,
    'support': 2587,
    'surfaceForm': 'Shakira',
    'types': 'Schema:MusicGroup,DBpedia:Agent,Schema:Person,DBpedia:Person,DBpedia:Artist,DBpedia:MusicalArtist'
}]

The same parameters apply to the spotlight.candidates function, which returns a list of all matching candidate entities rather than only the top candidate.

Note that the Spotlight API may support other interfaces that have not been implemented in pyspotlight. Feel free to contribute :-)!

Tips

We highly recommend playing around with the confidence and support values. Furthermore it might be preferable to filter out more annotations by looking at their similiarityScore (read: contextual score).

If you want to change the default values, feel free to use itertools.partial to create a little wrapper with simplified signature:

>>> from spotlight import annotate
>>> from functools import partial
>>> api = partial(annotate, 'http://localhost/rest/annotate',
...               confidence=0.4, support=20,
...               spotter='SpotXmlParser')
>>> api('This is your test text. This function uses a non-default
...      confidence, support, and spotter. Furthermore all calls go
...      directly to localhost/rest/annotate.')

As you can see this reduces the function’s complexity greatly. Pyspotlight provides an interface based on functions rather than classes, to avoid an unnecessary layer of indirection.

Tests

If you want to run the tests, you will have to install nose2 (~0.6) from PyPI. Then you can simply run nose2 from the command line in this or the spotlight/ directory.

All development and regular dependencies can be installed with a single command:

pip install -r requirements-dev.txt

Bugs

In case you spot a bug, please open an issue and attach the raw response you sent. Have a look at ubergrape/pyspotlight#3 for an example on how to file a good bug report.

Changelog