A set of utilities for generating quality scores for MediaWiki revisions

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

Revision Scoring

A generic, machine learning-based revision scoring system designed to be used to automatically differentiate damage from productive contributory behavior on Wikipedia.

Examples

Scoring models:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.languages import english
>>> from revscoring.scorers import MLScorerModel
>>>
>>> api_session = Session("https://en.wikipedia.org/w/api.php")
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>> extractor = APIExtractor(api_session, english)
>>>
>>> filename = "models/reverts.halfak_mix.trained.model"
>>> model = MLScorerModel.load(open(filename, 'rb'))
>>>
>>> rev_ids = [105, 642215410, 638307884]
>>> feature_values = [extractor.extract(id, model.features) for id in rev_ids]

>>> scores = model.score(feature_values, probabilities=True)
>>> for rev_id, score in zip(rev_ids, scores):
...     print("{0}: {1}".format(rev_id, score))
...
105: {'probabilities': array([ 0.96441465,  0.03558535]), 'prediction': False}
642215410: {'probabilities': array([ 0.75884553,  0.24115447]), 'prediction': True}
638307884: {'probabilities': array([ 0.98441738,  0.01558262]), 'prediction': False}

Feature extraction:

>>> from mw.api import Session
>>>
>>> from revscoring.extractors import APIExtractor
>>> from revscoring.features import diff, parent_revision, revision, user
>>>
>>> api_extractor = APIExtractor(Session("https://en.wikipedia.org/w/api.php"))
Sending requests with default User-Agent.  Set 'user_agent' on api.Session to quiet this message.
>>>
>>> features = [revision.day_of_week,
...             revision.hour_of_day,
...             revision.has_custom_comment,
...             parent_revision.bytes_changed,
...             diff.chars_added,
...             user.age,
...             user.is_anon,
...             user.is_bot]
>>>
>>> values = api_extractor.extract(
...     624577024,
...     features
... )
>>> for feature, value in zip(features, values):
...     print("{0}: {1}".format(feature, value))
...
<revision.day_of_week>: 6
<revision.hour_of_day>: 19
<revision.has_custom_comment>: True
<(revision.bytes - parent_revision.bytes_changed)>: 3
<diff.chars_added>: 8
<user.age>: 71821407
<user.is_anon>: False
<user.is_bot>: False

Installation

Packages

In order to use this, you need to install a few packages first:

You might need to install some other dependencies depending on your operating system. Try using the packages,

sudo apt-get install python3-dev python3-numpy python3-scipy g++ gfortran liblapack-dev libopenblas-dev myspell-pt myspell-fa myspell-en-au myspell-en-gb myspell-en-us myspell-en-za myspell-fr myspell-es hunspell-vi myspell-he

If you’re on Ubuntu, you might also be able to install an Indonesian dictionary:

sudo apt-get install aspell-id

Virtualenv users, please note that you’ll have to use the –system-site-packages option if you install scipy and numpy via apt-get. You can also use pip3 within your virtualenv.

Python modules

If you need the Python package installer,

sudo easy_install3 pip

Then, install this module,

pip3 install --user revscoring

You’ll need to download NLTK data in order to make use of language features.

python3 -m nltk.downloader stopwords

Authors

Aaron Halfaker:

  • http://halfaker.info

Helder:

  • https://github.com/he7d3r

Adam Roses Wight:

  • https://mediawiki.org/wiki/User:Adamw

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for aaron.halfaker from gravatar.com aaron.halfaker Avatar for adamw from gravatar.com adamw Avatar for Amir_Sarabadani from gravatar.com Amir_Sarabadani Avatar for scoring-internal from gravatar.com scoring-internal

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

2.11.13

2.11.12

2.11.10

2.11.9

2.11.8

2.11.6

2.11.5

2.11.4

2.11.3

2.11.2

2.11.1

2.11.0

2.10.0

2.9.3

2.9.2

2.9.1

2.9.0

2.8.2

2.8.1

2.8.0

2.7.2

2.7.1

2.7.0

2.6.9

2.6.8

2.6.7

2.6.6

2.6.5

2.6.4

2.6.3

2.6.2

2.6.1

2.6.0

2.5.2

2.5.1

2.5.0

2.4.0

2.3.4

2.3.3

2.3.2

2.3.1

2.3.0

2.2.6

2.2.5

2.2.4

2.2.3

2.2.2

2.2.1

2.2.0

2.1.0

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

0.7.11

0.7.10

0.7.8

0.7.7

0.7.6

0.7.5

0.7.3

0.7.2

0.7.1

0.7.0

0.6.7

0.6.6

0.6.5

0.6.4

This version

0.6.3

0.6.2

0.6.1

0.6.0

0.5.0

0.4.10

0.4.9

0.4.8

0.4.7

0.4.6

0.4.5

0.4.4

0.4.3

0.4.2

0.4.1

0.4.0

0.3.0

0.2.1

0.2.0

0.1.0

0.0.3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

revscoring-0.6.3.zip (108.0 kB view details)

Uploaded Source

revscoring-0.6.3.tar.gz (68.5 kB view details)

Uploaded Source

File details

Details for the file revscoring-0.6.3.zip.

File metadata

  • Download URL: revscoring-0.6.3.zip
  • Upload date:
  • Size: 108.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.6.3.zip
Algorithm Hash digest
SHA256 eb41773134641e67fe81f00fadfadd1b94e9304fa6d3b4feefe00db3f944e7da
MD5 d775d9b888b55c259322cc2c72dc4676
BLAKE2b-256 0640e76a6062b339175ea2870118788a0cc6a1f3df0a1b1d31eb835c80a1e44f

See more details on using hashes here.

File details

Details for the file revscoring-0.6.3.tar.gz.

File metadata

  • Download URL: revscoring-0.6.3.tar.gz
  • Upload date:
  • Size: 68.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for revscoring-0.6.3.tar.gz
Algorithm Hash digest
SHA256 5409f55bef7162d0565b1def6382adfcfedeabc13521719d9b6997067f27ae20
MD5 3c1b160117256e6dd719c71376134624
BLAKE2b-256 02eca1e862814402f06f0b594580a4a2cb5334e95736924d6e84f41746642d89

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page