Skip to main content

A fast, extensible Markdown parser in pure Python.

Project description

mistletoe-ebp

CI Status Coverage Documentation Status Code style: black PyPI Conda

This is a version of mistletoe maintained by the Excutable Book Project (EBP). It tracks the myst branch of ExecutableBookProject/mistletoe which eventually, it is hoped, will be merged into mistletoe itself.

mistletoe is a Markdown parser in pure Python, designed to be fast, spec-compliant and fully customizable.

Apart from being the fastest CommonMark-compliant Markdown parser implementation in pure Python, mistletoe also supports easy definitions of custom tokens. Parsing Markdown into an abstract syntax tree also allows us to swap out renderers for different output formats, without touching any of the core components.

Unfortunately, mistletoe is not currently being actively maintained (as of June 8th 2019), and so this fork has been created to allow for a deployed release that can be utilised by EBP. Here is a working list of 'up-streamable' changes that would be desired of mistletoe that this version has begun to implement:

  • Move testing from unittest to pytest: pytest is now the de facto testing architecture and vastly improves the usability/flexibility of testing.
  • Introduce pre-commit code linting and formatting: This standardizes the code style across the package, and ensures that new commits and Pull Requests also conform to it.
  • Introduce ReadTheDocs documentation
  • Add a conda-forge distribution of the package
  • Improve the AST API and documentation: I view panflute's implementation of the pandoc API in python, as the gold standard for how a pythonic AST API should be written and documented. Some tweaks to the current token class objects, and creating auto-generated RTD documentation, could achieve this.
  • Storage of source line/column ranges: LSP and good rendering reporting of warnings/errors, requires access to the source line and column ranges of each parsed token.
  • Asynchronous parsing: LSP requires documents to be parsed asynchronously. Currently, mistletoe contains a number of global state objects, which make parsing inherently not thread-safe. The simple solution to this is to store these items as threading.local objects. A related but slightly more complete solution is to introduce the idea of a 'scoped session', similar to that used by sqlalchemy for database access: Contextual/Thread-local Sessions
  • Improve extensibility of block tokens: A Markdown parser is inherently a Finite State-Machine + Memory (a.k.a Push-down Automata (PDA)), with parsing tokens as states (for a good example of a python state-machine see pytransitions). The problem with extensibility, is that inherently states are interdependent; when introducing a new state/token you must provide logic to all the other tokens, w.r.t to when to transition to this new token. Currently, MyST Parser sub-classes nearly all the Mistletoe block tokens to implement the extensions it requires, but it would be ideal if there was a more systematic approach for this.
  • Improve extensibility for span tokens: Mistletoe does allow for span token extensions to be added, at least in a simple way. However, as with block tokens above, there is often an interconnectivity to them, especially when considering nested span tokens. As of 7cc2c92, MyST-Parser now overrides some of Mistetoe's core logic to achieve correct parsing of Math tokens, but if possible this should be made more general.
  • Improve rendering logic: Currently, there is no concept of recursive walk-throughs or 'visitor' patterns in the Misteltoe BaseRenderer, which is a better method for rendering tree like structures (as used by docutils/panflute). Also, the current token instantiating (within context managers) needs improvement (see miyuchina/mistletoe#56).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mistletoe-ebp-0.9.2.tar.gz (41.5 kB view details)

Uploaded Source

Built Distribution

mistletoe_ebp-0.9.2-py3-none-any.whl (49.5 kB view details)

Uploaded Python 3

File details

Details for the file mistletoe-ebp-0.9.2.tar.gz.

File metadata

  • Download URL: mistletoe-ebp-0.9.2.tar.gz
  • Upload date:
  • Size: 41.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.1

File hashes

Hashes for mistletoe-ebp-0.9.2.tar.gz
Algorithm Hash digest
SHA256 b8278398271a5655516b90161b4dcf049c8ed5b699467336289564f80e5f8e86
MD5 86e820d9f311215018e7643f1f1c8318
BLAKE2b-256 8f3df7d562962bc0a84344213502b3448178a35494c41ddd14d7a7589730814e

See more details on using hashes here.

Provenance

File details

Details for the file mistletoe_ebp-0.9.2-py3-none-any.whl.

File metadata

  • Download URL: mistletoe_ebp-0.9.2-py3-none-any.whl
  • Upload date:
  • Size: 49.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.1

File hashes

Hashes for mistletoe_ebp-0.9.2-py3-none-any.whl
Algorithm Hash digest
SHA256 1e791f5c52792c01449e6b9bba9cd8d89023f82fe20973ee50d5e9bed5dae229
MD5 577d06efdaba77312818c4d7092f8969
BLAKE2b-256 e58bde1b8bf74fc23f092e211de5bdb069ffd238f13983253dce2edccd337f95

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page