A set of utilities for generating quality scores for MediaWiki revisions

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

Revision Scoring

A generic, machine learning-based revision scoring system designed to be used to automatically differentiate damage from productive contributory behavior on Wikipedia.

Examples

Scoring models:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.languages import english
>>> from revscoring.scorers import MLScorerModel
>>>
>>> api_session = Session("https://en.wikipedia.org/w/api.php")
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>> extractor = APIExtractor(api_session, english)
>>>
>>> filename = "models/reverts.halfak_mix.trained.model"
>>> model = MLScorerModel.load(open(filename, 'rb'))
>>>
>>> rev_ids = [105, 642215410, 638307884]
>>> feature_values = [extractor.extract(id, model.features) for id in rev_ids]

>>> scores = model.score(feature_values, probabilities=True)
>>> for rev_id, score in zip(rev_ids, scores):
...     print("{0}: {1}".format(rev_id, score))
...
105: {'probabilities': array([ 0.96441465,  0.03558535]), 'prediction': False}
642215410: {'probabilities': array([ 0.75884553,  0.24115447]), 'prediction': True}
638307884: {'probabilities': array([ 0.98441738,  0.01558262]), 'prediction': False}

Feature extraction:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.features import diff, parent_revision, revision, user
>>>
>>> api_extractor = APIExtractor(Session("https://en.wikipedia.org/w/api.php"))
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>>
>>> features = [revision.day_of_week,
...             revision.hour_of_day,
...             revision.has_custom_comment,
...             parent_revision.bytes_changed,
...             diff.chars_added,
...             user.age,
...             user.is_anon,
...             user.is_bot]
>>>
>>> values = api_extractor.extract(
...     624577024,
...     features
... )
>>> for feature, value in zip(features, values):
...     print("{0}: {1}".format(feature, value))
...
<revision.day_of_week>: 6
<revision.hour_of_day>: 19
<revision.has_custom_comment>: True
<(revision.bytes - parent_revision.bytes_changed)>: 3
<diff.chars_added>: 8
<user.age>: 71821407
<user.is_anon>: False
<user.is_bot>: False

Installation

In order to use this, you need to install a few packages first:

pip install revscoring

You’ll need to download NLTK data in order to make use of language features.

>>> python
>>> import nltk
>>> nltk.download()
>>> Downloader> d
>>> Identifier> wordnet
>>> Downloader> d
>>> Identifier> omw
>>> Downloader> d
>>> Identifier> stopwords
>>> Downloader> q
>>> exit()

You might need to install some other dependencies depending on your operating system. These are for scipy and numpy.

Linux Mint 17.1:

  1. sudo apt-get install g++ gfortran liblapack-dev python3-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za

Ubuntu 14.04:

  1. sudo apt-get install g++ gfortran liblapack-dev libopenblas-dev python3-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za

Authors

Aaron Halfaker:

  • http://halfaker.info

Helder:

  • https://github.com/he7d3r

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

2.11.13

2.11.12

2.11.10

2.11.9

2.11.8

2.11.6

2.11.5

2.11.4

2.11.3

2.11.2

2.11.1

2.11.0

2.10.0

2.9.3

2.9.2

2.9.1

2.9.0

2.8.2

2.8.1

2.8.0

2.7.2

2.7.1

2.7.0

2.6.9

2.6.8

2.6.7

2.6.6

2.6.5

2.6.4

2.6.3

2.6.2

2.6.1

2.6.0

2.5.2

2.5.1

2.5.0

2.4.0

2.3.4

2.3.3

2.3.2

2.3.1

2.3.0

2.2.6

2.2.5

2.2.4

2.2.3

2.2.2

2.2.1

2.2.0

2.1.0

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

0.7.11

0.7.10

0.7.8

0.7.7

0.7.6

0.7.5

0.7.3

0.7.2

0.7.1

0.7.0

0.6.7

0.6.6

0.6.5

0.6.4

0.6.3

0.6.2

0.6.1

0.6.0

0.5.0

0.4.10

0.4.9

0.4.8

0.4.7

0.4.6

0.4.5

0.4.4

0.4.3

0.4.2

0.4.1

0.4.0

0.3.0

This version

0.2.1

0.2.0

0.1.0

0.0.3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

revscoring-0.2.1.zip (67.5 kB view details)

Uploaded Source

revscoring-0.2.1.tar.gz (41.1 kB view details)

Uploaded Source

File details

Details for the file revscoring-0.2.1.zip.

File metadata

  • Download URL: revscoring-0.2.1.zip
  • Upload date:
  • Size: 67.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.2.1.zip
Algorithm Hash digest
SHA256 8027dfd34366034da08235fc49ae9320aaf3afa1170961a9849d977be874f1fc
MD5 03ada302b1530c772befc407254fb555
BLAKE2b-256 53fa73e5525f190fb61f5c95d5f05cae9bd5dac8266d6d457f9418d30d0f5b2c

See more details on using hashes here.

File details

Details for the file revscoring-0.2.1.tar.gz.

File metadata

  • Download URL: revscoring-0.2.1.tar.gz
  • Upload date:
  • Size: 41.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.2.1.tar.gz
Algorithm Hash digest
SHA256 06380647b77c02652723a5d432eecc992dcc1610dc5bc51c4da42224e18d54e9
MD5 990a39b6ded3e6d3c0e8b201cbec3964
BLAKE2b-256 0deefb98c0643805e00a53e4990d01287b3a5f4c56566e4d65de3ad3b0312819

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page