Skip to main content

Grako++ is a packrat parser runtime for Grako, written in C++.

Project description

Work in progress!

Build Status

Grako++

Grako++ is a packrat parser runtime for Grako, written in C++. The aim is to provide a Grako-compatible parser generator, but with improved performance over Python.

Differences to Grako

  • The output is always AST/JSON.

  • The semantics class can be implemented in C++ or Cython (pure Python through Cython is planned).

  • The regular expression syntax is ECMAScript, not Python.

  • Some features of Grako are missing, see below in the TODO section.

  • State types must implement operator< for std::map (for future flexibility, consider implementing the hash trait, too).

Build

The library is header-files only currently, so you can just copy the files in include/ to a convenient location, or build from there. More formally, you can use cmake.

Build with clang by invoking: cmake -DCMAKE_CXX_COMPILER=/path/to/clang++

Building with g++ < 4.9 requires linking with the boost_regex library, because the std::regex implementation is broken. You can rely on the exported cmake library file to configure the right settings.

Other useful option: -DCMAKE_INSTALL_PREFIX:PATH=/path/to/install

Usage

The grakopp program is used like grako to compile PEG files to source code. There are four different source code output formats that can be specified with the -f/–format option:

–format

–output

Purpose

hpp

name.hpp

C++ declaration

cpp

name.cpp

C++ implementation

pxd

name.pxd

Cython declaration

pyx

name.pyx

Cython implementation

For pure C++ parsers, generating the hpp and cpp files is sufficient. For Python integration, the pxd and pyx files are also needed. For “name” you should subsitute the actual name of the parser (either the base name of the PEG file or the argument to the –name option). The filenames are, for now, hard-coded into the source files (the underscore protects the C++ implementation from Cython-generated source files).

Here is an example how to build a parser:

$ ./grakopp -f hpp -o _basic.hpp tests/basic/basic.peg
$ ./grakopp --whitespace="" --no-nameguard -f cpp -o _basic.cpp tests/basic/basic.peg
$ g++ -DGRAKOPP_MAIN -std=c++11 -Iinclude -O4 -o basic _basic.cpp -lboost_regex

You can invoke the parser like a Python parser generated with grako (currently, no options are supported, though, except for an internal option –test that compares the AST with a text fixture file):

$ echo -n e1e2 | ./basic /dev/stdin sequence
[
    "e1",
    "e2"
]

C++ Interface

The following header files exist:

Header

Purpose

grakopp/exception.hpp

Parser exceptions

grakopp/buffer.hpp

Buffer for I/O

grakopp/ast.hpp

AST implementation

grakopp/parser.hpp

Parser base class

grakopp/grakopp/hpp

Include all above

grakopp/ast-io.hpp

Optional AST stream I/O

Python Integration

Until the Python package is prepared properly, build the Cython extensions manually, for example like this:

$ cd python/grakopp
$ cython --cplus buffer.pyx
$ cython --cplus ast.pyx
$ g++ -std=c++11 -I../../include -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I/usr/include/python2.7 -o buffer.so buffer.cpp
$ g++ -std=c++11 -I../../include -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I/usr/include/python2.7 -o ast.so ast.cpp
$ cd ../..

To continue the above example:

$ ./grakopp -f pxd -o basic.pxd tests/basic/basic.peg
$ ./grakopp -f pyx -o basic.pyx tests/basic/basic.peg
$ cython -Ipython --cplus basic.pyx
$ g++ -std=c++11 -Iinclude -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I/usr/include/python2.7 -o basic.so basic.cpp _basic.cpp -l boost_regex

You can then use it from Python:

$ PYTHONPATH=python python
>>> from grakopp import buffer
>>> b = buffer.PyBuffer()
>>> b.from_string("e1e2")
>>> import basic
>>> p = basic.basicPyParser()
>>> p.set_buffer(b)
>>> a = p._sequence_()
>>> a.to_python()
['e1', 'e2']
>>> b.pos
4
>>> a = p._sequence_()
>>> a.to_python()
FailedToken('e1')

TODO

  • dynamic Ast objects (so you can pass through Python or XML objects)

  • python/distutils integration

  • automatic compilation a la pyximport

  • add namespace

  • unicode support?

  • more support and tests for stateful parsing

  • regex syntax tests (make sure generated C strings are always proper)

  • profile and optimize

Grako features missing:

  • ignorecase (buffer match, matchre)

  • comments skipping

  • buffer line parsing and trace output (also in exceptions)

  • ParseInfo

  • rules with arguments

  • left recursion

  • semantic action “_default”

Authors

Copyright (C) 2014 semantics Kommunikationsmanagement GmbH
Written by Marcus Brinkmann <m.brinkmann@semantics.de>
See LICENSE.txt for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

grakopp-0.1.0.tar.gz (62.3 kB view details)

Uploaded Source

Built Distribution

grakopp-0.1.0.linux-x86_64.tar.gz (285.3 kB view details)

Uploaded Source

File details

Details for the file grakopp-0.1.0.tar.gz.

File metadata

  • Download URL: grakopp-0.1.0.tar.gz
  • Upload date:
  • Size: 62.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for grakopp-0.1.0.tar.gz
Algorithm Hash digest
SHA256 4d3b2892e5e8494dc773cc7d26a3396b92bb4e8d7c42f99f981b6140e7fbc2f6
MD5 ee931d383d83a66bf88785360a519eee
BLAKE2b-256 2ceda47e685b29ada7e1435b0d13eb4044f731bbdcba3f0fbe52ddba079dd1c7

See more details on using hashes here.

File details

Details for the file grakopp-0.1.0.linux-x86_64.tar.gz.

File metadata

File hashes

Hashes for grakopp-0.1.0.linux-x86_64.tar.gz
Algorithm Hash digest
SHA256 8f1f8a7fe69f13f739cfa358cb7674f55df6db8ba5059525ba16486da488df2a
MD5 b647fff80f0f1ab584f6e832802f014b
BLAKE2b-256 569e902734185b8c8e8c4c6f34f0e1f2152f77498c9b396beccd62670366bc7b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page