adblockparser 0.1

Installation

pip install adblockparser

For faster filter matching (2x-10x for a list of default EasyList filters) install pyre2 library. Version from github is required:

pip install git+https://github.com/axiak/pyre2.git#egg=re2

Usage

To learn about Adblock Plus filter syntax check these links:

• https://adblockplus.org/en/filter-cheatsheet

• https://adblockplus.org/en/filters

1. Get filter rules somewhere: write them manually, read lines from a file downloaded from EasyList, etc.:

   >>> raw_rules = [
   ...     "||ads.example.com^",
   ...     "@@||ads.example.com/notbanner^$~script",
   ... ]

2. Create AdblockRules instance from rule strings:

   >>> from adblockparser import AdblockRules
   >>> rules = AdblockRules(raw_rules)

3. Use this instance to check if an URL should be blocked or not:

   >>> rules.should_block("http://ads.example.com")
   True

   Rules with options are ignored unless you pass a dict with options values:

   >>> rules.should_block("http://ads.example.com/notbanner")
   True
   >>> rules.should_block("http://ads.example.com/notbanner", {'script': False})
   False
   >>> rules.should_block("http://ads.example.com/notbanner", {'script': True})
   True

Consult with Adblock Plus docs for options description. These options allow to write filters that depend on some external information not available in URL itself.

Performance

>>> rules = AdblockRules(raw_rules, use_re2=True)  # doctest: +SKIP

>>> rules = AdblockRules(raw_rules, use_re2=True, max_mem=512*1024*1024)  # doctest: +SKIP

>>> rules = AdblockRules(
...    raw_rules,
...    supported_options=['script', 'domain'],
...    skip_unsupported_rules=False
... )
>>> params = {'script': False, 'domain': 'www.mystartpage.com'}
>>> rules.should_block("http://ads.example.com/notbanner", params)
False

Limitations

There are some known limitations of the current implementation:

• element hiding rules are ignored;

• matching URLs against a large number of filters can be slow-ish, especially if pyre2 is not installed and many filter options are enabled;

• match-case filter option is not properly supported (it is ignored);

• document filter option is not properly supported;

• rules are not validated before parsing, so invalid rules may raise inconsistent exceptions or silently work incorrectly;

• regular expressions in rules are not supported.

It is possible to remove all these limitations. Pull requests are welcome if you want to make it happen sooner!

Contributing

• source code: https://github.com/scrapinghub/adblockparser

• issue tracker: https://github.com/scrapinghub/adblockparser/issues

In order to run tests, install tox and type

tox

from the source checkout.

The license is MIT.