A set of utilities for generating quality scores for MediaWiki revisions

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

Revision Scoring

A generic, machine learning-based revision scoring system designed to be used to automatically differentiate damage from productive contributory behavior on Wikipedia.

Examples

Scoring models:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.languages import english
>>> from revscoring.scorers import MLScorerModel
>>>
>>> api_session = Session("https://en.wikipedia.org/w/api.php")
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>> extractor = APIExtractor(api_session, english)
>>>
>>> filename = "models/reverts.halfak_mix.trained.model"
>>> model = MLScorerModel.load(open(filename, 'rb'))
>>>
>>> rev_ids = [105, 642215410, 638307884]
>>> feature_values = [extractor.extract(id, model.features) for id in rev_ids]

>>> scores = model.score(feature_values, probabilities=True)
>>> for rev_id, score in zip(rev_ids, scores):
...     print("{0}: {1}".format(rev_id, score))
...
105: {'probabilities': array([ 0.96441465,  0.03558535]), 'prediction': False}
642215410: {'probabilities': array([ 0.75884553,  0.24115447]), 'prediction': True}
638307884: {'probabilities': array([ 0.98441738,  0.01558262]), 'prediction': False}

Feature extraction:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.features import diff, parent_revision, revision, user
>>>
>>> api_extractor = APIExtractor(Session("https://en.wikipedia.org/w/api.php"))
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>>
>>> features = [revision.day_of_week,
...             revision.hour_of_day,
...             revision.has_custom_comment,
...             parent_revision.bytes_changed,
...             diff.chars_added,
...             user.age,
...             user.is_anon,
...             user.is_bot]
>>>
>>> values = api_extractor.extract(
...     624577024,
...     features
... )
>>> for feature, value in zip(features, values):
...     print("{0}: {1}".format(feature, value))
...
<revision.day_of_week>: 6
<revision.hour_of_day>: 19
<revision.has_custom_comment>: True
<(revision.bytes - parent_revision.bytes_changed)>: 3
<diff.chars_added>: 8
<user.age>: 71821407
<user.is_anon>: False
<user.is_bot>: False

Installation

In order to use this, you need to install a few packages first:

pip install revscoring

You’ll need to download NLTK data in order to make use of language features.

>>> python
>>> import nltk
>>> nltk.download()
>>> Downloader> d
>>> Identifier> wordnet
>>> Downloader> d
>>> Identifier> omw
>>> Downloader> d
>>> Identifier> stopwords
>>> Downloader> q
>>> exit()

You might need to install some other dependencies depending on your operating system. These are for scipy and numpy.

Linux Mint 17.1:

  1. sudo apt-get install g++ gfortran liblapack-dev python3-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za myspell-fr aspell-id myspell-es

Ubuntu 14.04:

  1. sudo apt-get install g++ gfortran liblapack-dev libopenblas-dev python3-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za myspell-fr aspell-id myspell-es

Authors

Aaron Halfaker:

  • http://halfaker.info

Helder:

  • https://github.com/he7d3r

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

2.11.13

2.11.12

2.11.10

2.11.9

2.11.8

2.11.6

2.11.5

2.11.4

2.11.3

2.11.2

2.11.1

2.11.0

2.10.0

2.9.3

2.9.2

2.9.1

2.9.0

2.8.2

2.8.1

2.8.0

2.7.2

2.7.1

2.7.0

2.6.9

2.6.8

2.6.7

2.6.6

2.6.5

2.6.4

2.6.3

2.6.2

2.6.1

2.6.0

2.5.2

2.5.1

2.5.0

2.4.0

2.3.4

2.3.3

2.3.2

2.3.1

2.3.0

2.2.6

2.2.5

2.2.4

2.2.3

2.2.2

2.2.1

2.2.0

2.1.0

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

0.7.11

0.7.10

0.7.8

0.7.7

0.7.6

0.7.5

0.7.3

0.7.2

0.7.1

0.7.0

0.6.7

0.6.6

0.6.5

0.6.4

0.6.3

0.6.2

0.6.1

0.6.0

0.5.0

0.4.10

0.4.9

0.4.8

0.4.7

0.4.6

0.4.5

This version

0.4.4

0.4.3

0.4.2

0.4.1

0.4.0

0.3.0

0.2.1

0.2.0

0.1.0

0.0.3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

revscoring-0.4.4.zip (83.4 kB view details)

Uploaded Source

revscoring-0.4.4.tar.gz (51.6 kB view details)

Uploaded Source

File details

Details for the file revscoring-0.4.4.zip.

File metadata

  • Download URL: revscoring-0.4.4.zip
  • Upload date:
  • Size: 83.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.4.4.zip
Algorithm Hash digest
SHA256 56934e4b2f55214cb2a1288425450021000b002b3d68cfaac3b4cb807afb08ad
MD5 856e2f987d9e660afaffe367754ca72e
BLAKE2b-256 0ad0d9899930961648832bc4b951c23bd1ec9506dd13bea88bfdf158ae7f7a49

See more details on using hashes here.

File details

Details for the file revscoring-0.4.4.tar.gz.

File metadata

  • Download URL: revscoring-0.4.4.tar.gz
  • Upload date:
  • Size: 51.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.4.4.tar.gz
Algorithm Hash digest
SHA256 04829d5e1b5e1a344dd4bfd307f13c88dc6f858f1208fbbb9ca3689c9bf3765e
MD5 f740322044b3de52d3d6f18031b374ad
BLAKE2b-256 4171ffc0425d925fc286b761b17a068d9638295766300318f9e589fe3d3fd517

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page