A set of utilities for generating quality scores for MediaWiki revisions

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

Revision Scoring

A generic, machine learning-based revision scoring system designed to be used to automatically differentiate damage from productive contributory behavior on Wikipedia.

Examples

Scoring models:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.languages import english
>>> from revscoring.scorers import MLScorerModel
>>>
>>> api_session = Session("https://en.wikipedia.org/w/api.php")
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>> extractor = APIExtractor(api_session, english)
>>>
>>> filename = "models/reverts.halfak_mix.trained.model"
>>> model = MLScorerModel.load(open(filename, 'rb'))
>>>
>>> rev_ids = [105, 642215410, 638307884]
>>> feature_values = [extractor.extract(id, model.features) for id in rev_ids]

>>> scores = model.score(feature_values, probabilities=True)
>>> for rev_id, score in zip(rev_ids, scores):
...     print("{0}: {1}".format(rev_id, score))
...
105: {'probabilities': array([ 0.96441465,  0.03558535]), 'prediction': False}
642215410: {'probabilities': array([ 0.75884553,  0.24115447]), 'prediction': True}
638307884: {'probabilities': array([ 0.98441738,  0.01558262]), 'prediction': False}

Feature extraction:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.features import diff, parent_revision, revision, user
>>>
>>> api_extractor = APIExtractor(Session("https://en.wikipedia.org/w/api.php"))
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>>
>>> features = [revision.day_of_week,
...             revision.hour_of_day,
...             revision.has_custom_comment,
...             parent_revision.bytes_changed,
...             diff.chars_added,
...             user.age,
...             user.is_anon,
...             user.is_bot]
>>>
>>> values = api_extractor.extract(
...     624577024,
...     features
... )
>>> for feature, value in zip(features, values):
...     print("{0}: {1}".format(feature, value))
...
<revision.day_of_week>: 6
<revision.hour_of_day>: 19
<revision.has_custom_comment>: True
<(revision.bytes - parent_revision.bytes_changed)>: 3
<diff.chars_added>: 8
<user.age>: 71821407
<user.is_anon>: False
<user.is_bot>: False

Installation

In order to use this, you need to install a few packages first:

pip install revscoring

You’ll need to download NLTK data in order to make use of language features.

>>> python
>>> import nltk
>>> nltk.download()
>>> Downloader> d
>>> Identifier> wordnet
>>> Downloader> d
>>> Identifier> omw
>>> Downloader> d
>>> Identifier> stopwords
>>> Downloader> q
>>> exit()

You might need to install some other dependencies depending on your operating system. These are for scipy and numpy.

Linux Mint 17.1:

  1. sudo apt-get install g++ gfortran liblapack-dev python3-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za myspell-fr aspell-id myspell-es hunspell-vi

Ubuntu 14.04:

  1. sudo apt-get install g++ gfortran liblapack-dev libopenblas-dev python3-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za myspell-fr aspell-id myspell-es hunspell-vi

Authors

Aaron Halfaker:

  • http://halfaker.info

Helder:

  • https://github.com/he7d3r

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

2.11.13

2.11.12

2.11.10

2.11.9

2.11.8

2.11.6

2.11.5

2.11.4

2.11.3

2.11.2

2.11.1

2.11.0

2.10.0

2.9.3

2.9.2

2.9.1

2.9.0

2.8.2

2.8.1

2.8.0

2.7.2

2.7.1

2.7.0

2.6.9

2.6.8

2.6.7

2.6.6

2.6.5

2.6.4

2.6.3

2.6.2

2.6.1

2.6.0

2.5.2

2.5.1

2.5.0

2.4.0

2.3.4

2.3.3

2.3.2

2.3.1

2.3.0

2.2.6

2.2.5

2.2.4

2.2.3

2.2.2

2.2.1

2.2.0

2.1.0

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

0.7.11

0.7.10

0.7.8

0.7.7

0.7.6

0.7.5

0.7.3

0.7.2

0.7.1

0.7.0

0.6.7

0.6.6

0.6.5

0.6.4

0.6.3

0.6.2

0.6.1

0.6.0

0.5.0

0.4.10

0.4.9

0.4.8

0.4.7

0.4.6

This version

0.4.5

0.4.4

0.4.3

0.4.2

0.4.1

0.4.0

0.3.0

0.2.1

0.2.0

0.1.0

0.0.3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

revscoring-0.4.5.zip (85.4 kB view details)

Uploaded Source

revscoring-0.4.5.tar.gz (52.7 kB view details)

Uploaded Source

File details

Details for the file revscoring-0.4.5.zip.

File metadata

  • Download URL: revscoring-0.4.5.zip
  • Upload date:
  • Size: 85.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.4.5.zip
Algorithm Hash digest
SHA256 3fe1a7d4789eb258e5f76c4a3c67d3db816dad7e6eea41595210f3c1d7bbeb11
MD5 619b8249a2b6a8390acacfa8b594f67c
BLAKE2b-256 1a08a2f536f4ddae8e1d3dfd4bb2a0770354372f2133d340c5b73fda11906640

See more details on using hashes here.

File details

Details for the file revscoring-0.4.5.tar.gz.

File metadata

  • Download URL: revscoring-0.4.5.tar.gz
  • Upload date:
  • Size: 52.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.4.5.tar.gz
Algorithm Hash digest
SHA256 21d132b95b4ee4166bec83bdb5836627ae181d6bcf388ddc873969a682439c8e
MD5 4fc062ee73dbde4da96f07416a7067eb
BLAKE2b-256 9d8456566159c4a0d1ba8aa6a7a155785c8a769b6870f2df70a872dc4cf0f121

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page