A set of utilities for generating quality scores for MediaWiki revisions

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

Revision Scoring

A generic, machine learning-based revision scoring system designed to be used to automatically differentiate damage from productive contributory behavior on Wikipedia.

Examples

Scoring models:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.languages import english
>>> from revscoring.scorers import MLScorerModel
>>>
>>> api_session = Session("https://en.wikipedia.org/w/api.php")
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>> extractor = APIExtractor(api_session, english)
>>>
>>> filename = "models/reverts.halfak_mix.trained.model"
>>> model = MLScorerModel.load(open(filename, 'rb'))
>>>
>>> rev_ids = [105, 642215410, 638307884]
>>> feature_values = [extractor.extract(id, model.features) for id in rev_ids]

>>> scores = model.score(feature_values, probabilities=True)
>>> for rev_id, score in zip(rev_ids, scores):
...     print("{0}: {1}".format(rev_id, score))
...
105: {'probabilities': array([ 0.96441465,  0.03558535]), 'prediction': False}
642215410: {'probabilities': array([ 0.75884553,  0.24115447]), 'prediction': True}
638307884: {'probabilities': array([ 0.98441738,  0.01558262]), 'prediction': False}

Feature extraction:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.features import diff, parent_revision, revision, user
>>>
>>> api_extractor = APIExtractor(Session("https://en.wikipedia.org/w/api.php"))
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>>
>>> features = [revision.day_of_week,
...             revision.hour_of_day,
...             revision.has_custom_comment,
...             parent_revision.bytes_changed,
...             diff.chars_added,
...             user.age,
...             user.is_anon,
...             user.is_bot]
>>>
>>> values = api_extractor.extract(
...     624577024,
...     features
... )
>>> for feature, value in zip(features, values):
...     print("{0}: {1}".format(feature, value))
...
<revision.day_of_week>: 6
<revision.hour_of_day>: 19
<revision.has_custom_comment>: True
<(revision.bytes - parent_revision.bytes_changed)>: 3
<diff.chars_added>: 8
<user.age>: 71821407
<user.is_anon>: False
<user.is_bot>: False

Installation

In order to use this, you need to install a few packages first:

pip install revscoring

You’ll need to download NLTK data in order to make use of language features.

>>> python
>>> import nltk
>>> nltk.download()
>>> Downloader> d
>>> Identifier> wordnet
>>> Downloader> d
>>> Identifier> omw
>>> Downloader> d
>>> Identifier> stopwords
>>> Downloader> q
>>> exit()

You might need to install some other dependencies depending on your operating system. These are for scipy and numpy.

Linux Mint 17.1:

  1. sudo apt-get install g++ gfortran liblapack-dev python3-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za myspell-fr

Ubuntu 14.04:

  1. sudo apt-get install g++ gfortran liblapack-dev libopenblas-dev python3-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za myspell-fr

Authors

Aaron Halfaker:

  • http://halfaker.info

Helder:

  • https://github.com/he7d3r

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

2.11.13

2.11.12

2.11.10

2.11.9

2.11.8

2.11.6

2.11.5

2.11.4

2.11.3

2.11.2

2.11.1

2.11.0

2.10.0

2.9.3

2.9.2

2.9.1

2.9.0

2.8.2

2.8.1

2.8.0

2.7.2

2.7.1

2.7.0

2.6.9

2.6.8

2.6.7

2.6.6

2.6.5

2.6.4

2.6.3

2.6.2

2.6.1

2.6.0

2.5.2

2.5.1

2.5.0

2.4.0

2.3.4

2.3.3

2.3.2

2.3.1

2.3.0

2.2.6

2.2.5

2.2.4

2.2.3

2.2.2

2.2.1

2.2.0

2.1.0

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

0.7.11

0.7.10

0.7.8

0.7.7

0.7.6

0.7.5

0.7.3

0.7.2

0.7.1

0.7.0

0.6.7

0.6.6

0.6.5

0.6.4

0.6.3

0.6.2

0.6.1

0.6.0

0.5.0

0.4.10

0.4.9

0.4.8

0.4.7

0.4.6

0.4.5

0.4.4

0.4.3

0.4.2

0.4.1

This version

0.4.0

0.3.0

0.2.1

0.2.0

0.1.0

0.0.3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

revscoring-0.4.0.zip (78.4 kB view details)

Uploaded Source

revscoring-0.4.0.tar.gz (47.5 kB view details)

Uploaded Source

File details

Details for the file revscoring-0.4.0.zip.

File metadata

  • Download URL: revscoring-0.4.0.zip
  • Upload date:
  • Size: 78.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.4.0.zip
Algorithm Hash digest
SHA256 a1093749dce79ecc2dec30a6def1b6adea49b89ec668ae7dbe1f81b2900478b8
MD5 c171d070678014c7a3e9a2134d336b57
BLAKE2b-256 3bd63b69673a18bd33e1ce26543d6721ec324bffdc210c246b336d38d7dc5ba3

See more details on using hashes here.

File details

Details for the file revscoring-0.4.0.tar.gz.

File metadata

  • Download URL: revscoring-0.4.0.tar.gz
  • Upload date:
  • Size: 47.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.4.0.tar.gz
Algorithm Hash digest
SHA256 9da8c86c7fc9d3e33734b482c99e2a184dc9677ee976f4eb9759f016ce61ed6e
MD5 78ebc67d84be6d57f598a3a59883ccac
BLAKE2b-256 3e2eb7b0ca73fa8ccc9752675718d6a884b573e45cc25bb2552cbb4ef3dd4158

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page