Generate a blocklists.xml file from the Kinto collections.
Project description
amo2kinto
kinto2xml: Export the blocklist in the legacy XML format
blockpages-generator: Generate the blocklist documentation pages.
Installation
To install the release:
pip install amo2kinto
To install the development environment:
make install
System dependencies
In order to build amo2kinto dependencies you may need the following libraries:
Building lxml will require: libxml2-dev libxslt-dev python-dev
Otherwise you will run into errors like:
libxml/xmlversion.h: File not found
Generate a blocklist.xml file from Kinto collections
If you want to export blocklists stored in Kinto in the addons server XML export format, you can use the kinto2xml script:
kinto2xml -s http://localhost:8888/v1
kinto2xml gives you the ability to configure what you want to export and in which bucket and collection are the data stored.
usage: kinto2xml [-h] [-s SERVER] [-a AUTH] [-v] [-q] [-D]
[--certificates-bucket CERTIFICATES_BUCKET]
[--certificates-collection CERTIFICATES_COLLECTION]
[--gfx-bucket GFX_BUCKET] [--gfx-collection GFX_COLLECTION]
[--addons-bucket ADDONS_BUCKET]
[--addons-collection ADDONS_COLLECTION]
[--plugins-bucket PLUGINS_BUCKET]
[--plugins-collection PLUGINS_COLLECTION]
[--api-version API_VERSION] [--app APP]
[--app-version APP_VERSION] [-o OUT]
Build a blocklists.xml file from Kinto blocklists.
optional arguments:
-h, --help show this help message and exit
-s SERVER, --server SERVER
The location of the remote server (with prefix)
-a AUTH, --auth AUTH BasicAuth token:my-secret
-v, --verbose Show all messages.
-q, --quiet Show only critical errors.
-D, --debug Show all messages, including debug messages.
--certificates-bucket CERTIFICATES_BUCKET
Bucket name for certificates
--certificates-collection CERTIFICATES_COLLECTION
Collection name for certificates
--gfx-bucket GFX_BUCKET
Bucket name for gfx
--gfx-collection GFX_COLLECTION
Collection name for gfx
--addons-bucket ADDONS_BUCKET
Bucket name for addons
--addons-collection ADDONS_COLLECTION
Collection name for addon
--plugins-bucket PLUGINS_BUCKET
Bucket name for plugins
--plugins-collection PLUGINS_COLLECTION
Collection name for plugin
--api-version API_VERSION
Targeted blocklists.xml APP id
--app APP Targeted blocklists.xml APP id
--app-version APP_VERSION
The targetted app version
-o OUT, --out OUT Output XML file.
Generate blocked addons and plugins description pages
You might want to export the human readable list and description of add-ons and plugins that were blocked.
You can do that using the blockpages-generator cli tool:
usage: blockpages-generator [-h] [-s SERVER] [-a AUTH] [-b BUCKET] [-v] [-q]
[-D] [--addons-collection ADDONS_COLLECTION]
[--plugins-collection PLUGINS_COLLECTION]
[-d TARGET_DIR]
Generate blocked item description files.
optional arguments:
-h, --help show this help message and exit
-s SERVER, --server SERVER
The location of the remote server (with prefix)
-a AUTH, --auth AUTH BasicAuth token:my-secret
-b BUCKET, --bucket BUCKET
Bucket name.
-v, --verbose Show all messages.
-q, --quiet Show only critical errors.
-D, --debug Show all messages, including debug messages.
--addons-collection ADDONS_COLLECTION
Collection name for addon
--plugins-collection PLUGINS_COLLECTION
Collection name for plugin
-d TARGET_DIR, --target-dir TARGET_DIR
Destination directory to write files in.
It will generate an index.html file with the list of records present in the addons and plugins collections in the target-dir directory.
It will also generate a file per add-on and plugin using the blockID or the id. e.g i487.html or 08db5018-2c80-4c4d-aa98-dafe6aacc28c.html
CHANGELOG
This document describes changes between each past release.
4.0.2 (2020-05-18)
Bug fix
Fix crash when prefs field is missing from addons entry
4.0.1 (2018-07-04)
Bug fix
Fix the XML item ID when squashing entries by addon ID
4.0.0 (2018-07-04)
Bug fix
Fix the affected users section (#87), thanks @rctgamer3!
Breaking changes
Removed code in charge of updating the collection schema (#85)
3.2.1 (2018-02-28)
Fix bug with Python 3 and writing files (#80).
3.2.0 (2018-02-06)
Filter add-ons and plugins in v3 based on the requesting application and version. (#74)
Stop exporting cert items to Firefox 58 and above, where they aren’t used. (#75)
3.1.0 (2017-10-03)
Add support for cert items subject and pubKeyHash attributes. (#70)
3.0.0 (2017-09-14)
Remove json2kinto importer
Remove xml verifier
2.0.1 (2017-06-02)
Bug fix
Fix synchronize kinto-http parameters. (#67)
2.0.0 (2017-06-02)
Breaking changes
Upgrade to kinto-http.py 8.0 and drop support for Python < 3.5
1.7.2 (2017-02-20)
Fix XML exporter on missing blockID. (#63)
1.7.1 (2016-11-24)
Bug fix
Allow the importer to work even without the permission to create collections. (#56)
Use PUT instead of PATCH to fully overwrite the destination with the source (#58)
1.7.0 (2016-11-21)
New features
Retry downloading the XML in case it fails the first time. (#50)
Bug fix
Remove redundant footer when generating pages. (#51)
Allow the importer to work even without the permission to create buckets. (#53)
Internal changes
Improve the way the Makefile works (#52)
1.6.0 (2016-10-04)
New features
Add a configurable signoff workflow to-review → to-sign (#48)
1.5.1 (2016-09-08)
Bug fix
session_kwargs is not accessible anymore with last kinto-http client release. (#45)
1.5.0 (2016-08-25)
Add the blockpages-generator CLI tool (#43)
1.4.1 (2016-08-02)
Fix blockID ordering to make it the same as the addons-server.
1.4.0 (2016-07-18)
New features
Add version ranges for GFX items (#39) â Bug 1283601
1.3.1 (2016-07-06)
Bug fix
Fix patch_records properties.
1.3.0 (2016-07-06)
Update records that were changed (#37)
1.2.1 (2016-05-26)
Bug fix
kinto_client.delete_records API changed. (#35)
1.2.0 (2016-05-26)
New features
Let people specify --app-version and --api-version while using kinto2xml (#33)
Add functional tests for both version of the XML file format. (#33)
Configure the kinto-admin as well as the JSONSchema (#32)
Bug fix
Group addons by the biggest guid (#33)
1.1.0 (2016-05-18)
Bug fix
In case there is a diff using xml-verifier, the command now returns an error code (#28)
Internal changes
json2kinto does not set destination collections as publicly readable (#27)
1.0.0 (2016-05-12)
Breaking changes
kinto2xml was renamed amo2kinto (#21)
New JSON Schema file format with a “collection” prefix (#22)
New features
Use the schema to validate AMO records in the importer script. (#5)
Warn if the server does not have the schema capability (#24)
Internal changes
Document amo2kinto commands: json2kinto, kinto2xml and verifier. (#23)
0.1.0 (2016-04-27)
Initial version
Create collection with the definition of the JSON schema.
Fetch AMO blocklists information from the /blocked/blocklists.json AMO endpoint.
Handle import configuration on the CLI. - Bucket / Collection names - Remote AMO endpoint configuration - Schema file path configuration - Schema or not schema - Verbosity level - Server selection - Auth credentials - Importation type selection
Support for kinto-signer triggering
Full SSL support for Python 2.7
Full Python 2.7 and Python 3.4/3.5 support
Handle the enabled flag to activate records
Makefile rule to update the schema definition
Export kinto blocklists in XML blocklist file version 2
Export kinto blocklists in XML blocklist file version 3
XML verifier that create a diff of two XML files
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file amo2kinto-4.0.2.tar.gz
.
File metadata
- Download URL: amo2kinto-4.0.2.tar.gz
- Upload date:
- Size: 137.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.4.0 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.8.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 15e054a21316108305318c8144ce7e0009bea0fdc11e0c905b3d1483dd6df30a |
|
MD5 | e8e9a53a0e2a6d2d57b7974feeb1b1bc |
|
BLAKE2b-256 | 493579edcb03fbc9a2e193691ece7358d8052478cbffab4f06da5022bbb6ad4b |
File details
Details for the file amo2kinto-4.0.2-py3-none-any.whl
.
File metadata
- Download URL: amo2kinto-4.0.2-py3-none-any.whl
- Upload date:
- Size: 137.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.4.0 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.8.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | cbff58a61de1b2e6582aff5d912532e1bdbcbf31e4a0239856c7019c2214df34 |
|
MD5 | 0a2e9ed6d4d6d5ac39762a2286657730 |
|
BLAKE2b-256 | 8852570f9b316133104440a3f1a2dd27f8fb3b6419bcf23b38b9bfd452d02a74 |