adblockparser 0.4

Installation

pip install adblockparser

If you plan to use this library with a large number of filters installing pyre2 library is highly recommended: the speedup for a list of default EasyList filters can be greater than 1000x.

pip install ‘re2 >= 0.2.21’

Note that pyre2 library requires C++ re2 library installed. On OS X you can get it using homebrew (brew install re2).

Usage

To learn about Adblock Plus filter syntax check these links:

• https://adblockplus.org/en/filter-cheatsheet

• https://adblockplus.org/en/filters

1. Get filter rules somewhere: write them manually, read lines from a file downloaded from EasyList, etc.:

   >>> raw_rules = [
   ...     "||ads.example.com^",
   ...     "@@||ads.example.com/notbanner^$~script",
   ... ]

2. Create AdblockRules instance from rule strings:

   >>> from adblockparser import AdblockRules
   >>> rules = AdblockRules(raw_rules)

3. Use this instance to check if an URL should be blocked or not:

   >>> rules.should_block("http://ads.example.com")
   True

   Rules with options are ignored unless you pass a dict with options values:

   >>> rules.should_block("http://ads.example.com/notbanner")
   True
   >>> rules.should_block("http://ads.example.com/notbanner", {'script': False})
   False
   >>> rules.should_block("http://ads.example.com/notbanner", {'script': True})
   True

Consult with Adblock Plus docs for options description. These options allow to write filters that depend on some external information not available in URL itself.

Performance

>>> rules = AdblockRules(raw_rules, use_re2=True, max_mem=512*1024*1024)  # doctest: +SKIP

>>> rules = AdblockRules(
...    raw_rules,
...    supported_options=['script', 'domain'],
...    skip_unsupported_rules=False
... )
>>> options = {'script': False, 'domain': 'www.mystartpage.com'}
>>> rules.should_block("http://ads.example.com/notbanner", options)
False

Limitations

There are some known limitations of the current implementation:

• element hiding rules are ignored;

• matching URLs against a large number of filters can be slow-ish, especially if pyre2 is not installed and many filter options are enabled;

• match-case filter option is not properly supported (it is ignored);

• document filter option is not properly supported;

• rules are not validated before parsing, so invalid rules may raise inconsistent exceptions or silently work incorrectly;

• regular expressions in rules are not supported.

It is possible to remove all these limitations. Pull requests are welcome if you want to make it happen sooner!

Contributing

• source code: https://github.com/scrapinghub/adblockparser

• issue tracker: https://github.com/scrapinghub/adblockparser/issues

In order to run tests, install tox and type

tox

from the source checkout.

The license is MIT.

0.4 (2015-03-29)

• AdblockRule now caches the compiled regexes (thanks https://github.com/mozbugbox);

• Fixed an issue with “domain” option handling (thanks https://github.com/nbraem for the bug report and a test case);

• cleanups and test improvements.

0.3 (2014-07-11)

• Switch to setuptools;

• better __repr__ for AdblockRule;

• Python 3.4 support is confirmed;

• testing improvements.

0.2 (2014-03-20)

This release provides much faster AdblockRules.should_block() method for rules without options and rules with ‘domain’ option.

• better combined regex for option-less rules that makes re2 library always use DFA without falling back to NFA;

• an index for rules with domains;

• params method arguments are renamed to options for consistency.

0.1.1 (2014-03-11)

By default AdblockRules autodetects re2 library and uses it if a compatible version is detected.

0.1 (2014-03-03)

Initial release.