tree-sitter-languages

Binary Python wheels for all tree sitter languages.

These details have not been verified by PyPI

Project links

Development Status
- 5 - Production/Stable
Intended Audience
- Developers
License
- OSI Approved :: Apache Software License
Natural Language
- English
Programming Language

Project description

Binary Python wheels for all tree sitter languages.

py-tree-sitter is a fantastic library that provides Python bindings for the even more fantastic tree-sitter parsing library.

py-tree-sitter-languages provides binary Python wheels for all tree sitter languages. The binary wheels remove the need to download and compile support for individual languages.

Install

pip install tree_sitter_languages

Source installs are not supported. To see how the binary wheels are built, look at:

setup.py — Python package setup.
build.sh — Shell script to download support for all languages.
build.py — Python script to build support for all languages.
.github/workflows/release.yml — GitHub action to invoke cibuildwheel and release to PyPI.

Usage

from tree_sitter_languages import get_language, get_parser

language = get_language('python')
parser = get_parser('python')

That’s the whole API!

Refer to py-tree-sitter for the language and parser API. Notice the Language.build_library(...) step can be skipped! The binary wheel includes the language binary.

Demo

Want to know something crazy? Python lacks multi-line comments. Whhaaa!?!

It’s really not such a big deal. Instead of writing:

"""
My awesome
multi-line
comment.
"""

Simply write:

# My awesome
# multi-line
# comment.

So multi-line comments are made by putting multiple single-line comments in sequence. Amazing!

Now, how to find all the strings being used as comments?

Start with some example Python code:

example = """
#!shebang
# License blah blah (Apache 2.0)
"This is a module docstring."

a = 1

'''This
is
not
a
multiline
comment.'''

b = 2

class Test:
    "This is a class docstring."

    'This is bogus.'

    def test(self):
        "This is a function docstring."

        "Please, no."

        return 1

c = 3
"""

Notice a couple things:

Python has module, class, and function docstrings that bare a striking resemblance to the phony string comments.
Python supports single-quoted, double-quoted, triple-single-quoted, and triple-double-quoted strings (not to mention prefixes for raw strings, unicode strings, and more).

Creating a regular expression to capture the phony string comments would be exceedingly difficult!

Enter tree-sitter:

from tree_sitter_languages import get_language, get_parser

language = get_language('python')
parser = get_parser('python')

Tree-sitter creates an abstract syntax tree (actually, a concrete syntax tree) and supports queries:

tree = parser.parse(example.encode())
node = tree.root_node
print(node.sexp())

Look for statements that are a single string expression:

stmt_str_pattern = '(expression_statement (string)) @stmt_str'
stmt_str_query = language.query(stmt_str_pattern)
stmt_strs = stmt_str_query.captures(node)
stmt_str_points = set(
    (node.start_point, node.end_point) for node, _ in stmt_strs
)
print(stmt_str_points)

Now, find those statement string expressions that are actually module, class, or function docstrings:

doc_str_pattern = """
    (module . (comment)* . (expression_statement (string)) @module_doc_str)

    (class_definition
        body: (block . (expression_statement (string)) @class_doc_str))

    (function_definition
        body: (block . (expression_statement (string)) @function_doc_str))
"""
doc_str_query = language.query(doc_str_pattern)
doc_strs = doc_str_query.captures(node)
doc_str_points = set(
    (node.start_point, node.end_point) for node, _ in doc_strs
)

With the set of string expression statements and the set of docstring statements, the locations of all phony string comments is:

comment_strs = stmt_str_points - doc_str_points
print(sorted(comment_strs))

License

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

The project also includes the following other projects distributed in binary form:

https://github.com/tree-sitter/tree-sitter — licensed under the MIT License.
https://github.com/AbstractMachinesLab/tree-sitter-erlang — licensed under the Apache License, Version 2.0.
https://github.com/Azganoth/tree-sitter-lua — licensed under the MIT License.
https://github.com/Wilfred/tree-sitter-elisp — licensed under the MIT License.
https://github.com/alemuller/tree-sitter-make — licensed under the MIT License.
https://github.com/camdencheek/tree-sitter-dockerfile — licensed under the MIT License.
https://github.com/camdencheek/tree-sitter-go-mod — licensed under the MIT License.
https://github.com/elixir-lang/tree-sitter-elixir — licensed under the Apache License, Version 2.0.
https://github.com/elm-tooling/tree-sitter-elm — licensed under the MIT License.
https://github.com/fwcd/tree-sitter-kotlin — licensed under the MIT License.
https://github.com/ganezdragon/tree-sitter-perl — licensed under the MIT License.
https://github.com/ikatyang/tree-sitter-markdown — licensed under the MIT License.
https://github.com/ikatyang/tree-sitter-toml — licensed under the MIT License.
https://github.com/ikatyang/tree-sitter-yaml — licensed under the MIT License.
https://github.com/jiyee/tree-sitter-objc — licensed under the MIT License.
https://github.com/m-novikov/tree-sitter-sql — licensed under the MIT License.
https://github.com/r-lib/tree-sitter-r — licensed under the MIT License.
https://github.com/rydesun/tree-sitter-dot — licensed under the MIT License.
https://github.com/slackhq/tree-sitter-hack — licensed under the MIT License.
https://github.com/theHamsta/tree-sitter-commonlisp — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-bash — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-c — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-c-sharp — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-cpp — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-css — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-embedded-template — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-go — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-haskell — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-html — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-java — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-javascript — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-jsdoc — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-json — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-julia — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-ocaml — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-php — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-python — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-ql — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-regex — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-ruby — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-rust — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-scala — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-swift — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-toml — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-tsq — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-typescript — licensed under the MIT License.
https://github.com/tree-sitter/tree-sitter-verilog — licensed under the MIT License.

Project details

These details have not been verified by PyPI

Project links

Development Status
- 5 - Production/Stable
Intended Audience
- Developers
License
- OSI Approved :: Apache Software License
Natural Language
- English
Programming Language

Release history Release notifications | RSS feed

1.10.2

Feb 4, 2024

1.10.0

Jan 30, 2024

1.9.1

Jan 3, 2024

1.8.0

Oct 14, 2023

1.7.0

Jul 18, 2023

1.6.1

Jun 26, 2023

1.5.0

Sep 14, 2022

1.4.0

Aug 20, 2022

1.3.0

Aug 14, 2022

This version

1.2.2

May 23, 2022

1.2.0

May 21, 2022

1.1.0

May 16, 2022

0.1.0

May 15, 2022

0.0.0

May 15, 2022

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

tree_sitter_languages-1.2.2-cp310-cp310-musllinux_1_1_x86_64.whl (5.2 MB view details)

Uploaded May 23, 2022 CPython 3.10 musllinux: musl 1.1+ x86-64

tree_sitter_languages-1.2.2-cp310-cp310-musllinux_1_1_i686.whl (5.3 MB view details)

Uploaded May 23, 2022 CPython 3.10 musllinux: musl 1.1+ i686

tree_sitter_languages-1.2.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB view details)

Uploaded May 23, 2022 CPython 3.10 manylinux: glibc 2.17+ x86-64

tree_sitter_languages-1.2.2-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.10 manylinux: glibc 2.17+ i686

tree_sitter_languages-1.2.2-cp310-cp310-macosx_10_9_x86_64.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.10 macOS 10.9+ x86-64

tree_sitter_languages-1.2.2-cp39-cp39-musllinux_1_1_x86_64.whl (5.2 MB view details)

Uploaded May 23, 2022 CPython 3.9 musllinux: musl 1.1+ x86-64

tree_sitter_languages-1.2.2-cp39-cp39-musllinux_1_1_i686.whl (5.3 MB view details)

Uploaded May 23, 2022 CPython 3.9 musllinux: musl 1.1+ i686

tree_sitter_languages-1.2.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB view details)

Uploaded May 23, 2022 CPython 3.9 manylinux: glibc 2.17+ x86-64

tree_sitter_languages-1.2.2-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.9 manylinux: glibc 2.17+ i686

tree_sitter_languages-1.2.2-cp39-cp39-macosx_10_9_x86_64.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.9 macOS 10.9+ x86-64

tree_sitter_languages-1.2.2-cp38-cp38-musllinux_1_1_x86_64.whl (5.2 MB view details)

Uploaded May 23, 2022 CPython 3.8 musllinux: musl 1.1+ x86-64

tree_sitter_languages-1.2.2-cp38-cp38-musllinux_1_1_i686.whl (5.3 MB view details)

Uploaded May 23, 2022 CPython 3.8 musllinux: musl 1.1+ i686

tree_sitter_languages-1.2.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB view details)

Uploaded May 23, 2022 CPython 3.8 manylinux: glibc 2.17+ x86-64

tree_sitter_languages-1.2.2-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.8 manylinux: glibc 2.17+ i686

tree_sitter_languages-1.2.2-cp38-cp38-macosx_10_9_x86_64.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.8 macOS 10.9+ x86-64

tree_sitter_languages-1.2.2-cp37-cp37m-musllinux_1_1_x86_64.whl (5.2 MB view details)

Uploaded May 23, 2022 CPython 3.7m musllinux: musl 1.1+ x86-64

tree_sitter_languages-1.2.2-cp37-cp37m-musllinux_1_1_i686.whl (5.3 MB view details)

Uploaded May 23, 2022 CPython 3.7m musllinux: musl 1.1+ i686

tree_sitter_languages-1.2.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB view details)

Uploaded May 23, 2022 CPython 3.7m manylinux: glibc 2.17+ x86-64

tree_sitter_languages-1.2.2-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.7m manylinux: glibc 2.17+ i686

tree_sitter_languages-1.2.2-cp37-cp37m-macosx_10_9_x86_64.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.7m macOS 10.9+ x86-64

tree_sitter_languages-1.2.2-cp36-cp36m-musllinux_1_1_x86_64.whl (5.2 MB view details)

Uploaded May 23, 2022 CPython 3.6m musllinux: musl 1.1+ x86-64

tree_sitter_languages-1.2.2-cp36-cp36m-musllinux_1_1_i686.whl (5.3 MB view details)

Uploaded May 23, 2022 CPython 3.6m musllinux: musl 1.1+ i686

tree_sitter_languages-1.2.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB view details)

Uploaded May 23, 2022 CPython 3.6m manylinux: glibc 2.17+ x86-64

tree_sitter_languages-1.2.2-cp36-cp36m-manylinux_2_17_i686.manylinux2014_i686.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.6m manylinux: glibc 2.17+ i686

tree_sitter_languages-1.2.2-cp36-cp36m-macosx_10_9_x86_64.whl (4.7 MB view details)

Uploaded May 23, 2022 CPython 3.6m macOS 10.9+ x86-64

File details

Details for the file tree_sitter_languages-1.2.2-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

Download URL: tree_sitter_languages-1.2.2-cp310-cp310-musllinux_1_1_x86_64.whl
Upload date: May 23, 2022
Size: 5.2 MB
Tags: CPython 3.10, musllinux: musl 1.1+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: twine/4.0.0 CPython/3.9.13

File hashes

Hashes for tree_sitter_languages-1.2.2-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm	Hash digest
SHA256	`b5f8f379b926ddfa91476ccc8abb4cbfad3297052c30493378090aeee4694487`
MD5	`3e13797bd10ec2a82663cb0f79afb4cb`
BLAKE2b-256	`200a2953b2e8694b3260a97094e7fb62f9e9628628101150ec7721fcc1a66207`

Algorithm	Hash digest
SHA256	`6dfed13c38a4e20384daed135122e5be711e58f10e02b6e91820d9da7ae2fbe4`
MD5	`0aec51ccfd8900f76f694140b9ee086a`
BLAKE2b-256	`e3b304112eb5b0d0b9697f5bf5275fe5d0115b74108cedf4669b5c65683d6387`

Algorithm	Hash digest
SHA256	`bd6d1250a824d88d1bc58d696ead6bdb5b3516335d38ae7a86396f4796d1ae98`
MD5	`ace7f4835cfbcfc0332bcda5aef7e636`
BLAKE2b-256	`7637f2f6a784d902ad62e1df480fd8678ca15f1682020e28215cd074fec4874a`

Algorithm	Hash digest
SHA256	`0f4e82d108c54d9aaf7a90626f9bad0f5d58fa2b950c9e2530863c2bfebe32ac`
MD5	`379a1ebd11a3d38429d9aeec1c413003`
BLAKE2b-256	`c6f7a92f0220fdccdad34ff5cceef5b9253f9b502b8bb831eb7fb5ff72bf670d`

Algorithm	Hash digest
SHA256	`c24be48c2ed1805562b35232e4c5a7d58a30493d1e6440d9967f0ee441b84d80`
MD5	`1bdd4c0241d3fcf1633267ebd709e78e`
BLAKE2b-256	`84dcb4af084889701949139870ae653a27a69b61e6ef819fba0eb8a291c9b67b`

Algorithm	Hash digest
SHA256	`81a8b0f6ef00dc3a40a731ea588b1d890b172b3d832d97e38a47210225b74dfe`
MD5	`8c29e48071b49671594b72123b218a5d`
BLAKE2b-256	`8a08644e0395bb60a7c602c1a7d90b2f56f2c24aefbaa6d787593028ac77bd8e`

Algorithm	Hash digest
SHA256	`a9599a79060ccb4bec5d65a479da20ca71a101086133ede7003a315c54f34ff1`
MD5	`2d70696f636d94dc39b5bd6cdb4a9883`
BLAKE2b-256	`e8b9111cb3c3c354bca76f8011655afdb258c60374be45dcb8bbaceb6e4c7566`

Algorithm	Hash digest
SHA256	`33e2943eb32a9bd0266181cb712efdeff33fe5dd730c5e3b7c8c5b35158da0af`
MD5	`ba21c72ad5e5a44ce24762269010532f`
BLAKE2b-256	`fb3d64efb5c451344fc3de6506712c018b1771111bff072318ca69e22d905632`

Algorithm	Hash digest
SHA256	`0e8d1932760089c9cb16bebf863607ffd021d43c182736a4a63bf1959b18bcf5`
MD5	`06add96001148b78a8d5dd3a9d65c6fd`
BLAKE2b-256	`d7d2ac1a39d001d6f814b3c8236b112157ecce8461f98d7f93da30e2fed42309`

Algorithm	Hash digest
SHA256	`d8cd77e389e3dbc37dc76acd5da28d60ea2bded270ea50564add8a3b2d95fa79`
MD5	`1c53a1d374b5f8a7b7ceeb792973c9fd`
BLAKE2b-256	`d1f85bbd3b5cad5b6b5bbca2bc4c921a371d4fd10dbb4338412b866b6ec59009`

Algorithm	Hash digest
SHA256	`bf32b0652aeee86f019688e22792df0c93ac840a5c9afa0312074ce4abd4737e`
MD5	`793e5ca1d9ba363b5582060debe2514c`
BLAKE2b-256	`56ef32b358419d925da6c7c999db9e38db4282d07cc59047ba8a8590ace467a9`

Algorithm	Hash digest
SHA256	`60c4119b2065aebfe51b5ab723b7a4a38c10069af9cfae606c4eb81bb1dd19b7`
MD5	`ff322f9552763012fe72389690e14cb9`
BLAKE2b-256	`052b06fa6d8643956a6da203450abde3ff7be77d8c2ba38f1f08c4a5a2e81cc0`

Algorithm	Hash digest
SHA256	`ba5f6beb0dd704cddb3773ed737e85714af54df5179610e7e28c7a4e36819ec2`
MD5	`546f03191d0ca1139697433bb9137bb7`
BLAKE2b-256	`001a6f3456db53aecb13db73262596661131576409cd5d598bafdd5c07c9495b`

Algorithm	Hash digest
SHA256	`acecf18428a8f7daf4beea369f42c49005f15b3ac62b802b66502782290c8679`
MD5	`8bc0a8ef6638e68620f003c7278c153e`
BLAKE2b-256	`80bf3a533dd4c141a2494816cbc80c4271322eac589dff94a086ad01d9d4bca2`

Algorithm	Hash digest
SHA256	`c9e021d3d6b86971e066e9e83797c52ee606db6c0b2d0a0abf9f7a68e44255e7`
MD5	`85c19d872aaefe480485aa0e0ccc714a`
BLAKE2b-256	`7e55a82127cb6f7f2dfb59f7caf2960f489ec9ae89ddb87ff20abdc412b6c4d4`

Algorithm	Hash digest
SHA256	`f3fe7ea95a771a91eebdee6ea1c5a6c7bd18a2951136bd86d77bf1bcc5dc94a3`
MD5	`4b0ea75285011cf093d51469ccfde3c1`
BLAKE2b-256	`ce5d3b4d481cdbfef6ab6e21faee0a73791c3d997712a40f99a35e5502570d25`

Algorithm	Hash digest
SHA256	`a53fc915abb7712279d4a1f385bb1ede803982a4442360643ad31d65cee6038d`
MD5	`74974566eb8e11773c005b6689ecee54`
BLAKE2b-256	`7bcc6c89f5c918bc1f5b60cb43d63d7976677f6132dc4f6e7890a8053af9cbae`

Algorithm	Hash digest
SHA256	`e1e79a5ae1a373a6b9803c9f895d299b8f939eb884e97b1314047c1c31190699`
MD5	`6f48c37890377f82de271ad8a9280a2a`
BLAKE2b-256	`5fcefb8fa3e921ba972948247bb5bff43a5c46f01c51b9d92d6a4a10ca6e1643`

Algorithm	Hash digest
SHA256	`5ddc2e30ca6f7635ff0bed62826bc0f32a051778621f9579af4fe8b1148d06b1`
MD5	`f1109bdd98991dcfc1e8461d5db340db`
BLAKE2b-256	`13ae3f83a0106175cdb4257271efbc30f87daa0fe13187168ad5aa0a38a1a7cd`

Algorithm	Hash digest
SHA256	`842683b3b2daf1a26703051cf4a7fc07407c68eb0f9316619002db6646ee45c5`
MD5	`7660f3724f39ae2b4025d68fb8b51d47`
BLAKE2b-256	`8bbab62df6b44edc69d44d0a0e6656ca7572670e2925fd844108f800e8f803a9`

Algorithm	Hash digest
SHA256	`237ccd61e147849417cb9cdf209f63d35e02fc0649bff46c08c9a6557f5f0705`
MD5	`4a77bf66de68f4e0a7d2c6b951c2a6c0`
BLAKE2b-256	`41bd93659f8c9474145b4a29d56c9c9809556e6a2d3f1a8a23b2cbb3bb7569f2`

tree-sitter-languages 1.2.2

Navigation

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Project description

Install

Usage

Demo

License

Project details

Verified details

Maintainers

Unverified details

Project links

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distributions

Built Distributions

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

Algorithm	Hash digest
SHA256	`c8dcef22af5f17ac6303a2c940eb5dc2e404c8da5a09b86db7c8fea0ff01ccd5`
MD5	`b64aa4b4ee63e1697aa190f1ef95f3a4`
BLAKE2b-256	`0d03fedc1cd359c60e6db99e415d25d51ebb1af4be379b43b687d0a20d2678e3`

Algorithm	Hash digest
SHA256	`daa8394e9c085ef3074b688d69f7cdb88c0308019659e2f71341a3812a121d75`
MD5	`a845185e2d0d76acd2938afddec4a3b4`
BLAKE2b-256	`5bc5120f5c918e97d0f7e601b7839ded92b1f865494837ef92df2f14e2f93906`

Algorithm	Hash digest
SHA256	`06b327fa344df0f3069c251085289793ab8012895c120ae224929f130614b3aa`
MD5	`411312d560f14568738f0e0897d075ea`
BLAKE2b-256	`8494acedb46f509b83960513a43f282d520ee19426af12314a52eddbedb92669`

Algorithm	Hash digest
SHA256	`cc0b5d6215649a6c66bbf92a5a7902e55313f141d6183fc3bcce30ab41f96ce1`
MD5	`5727b024e0b0a8f52c756b2d1f36410f`
BLAKE2b-256	`ae06c7d3416fce060f147424d23cef570a0b68ee7e51cf55ef95a627d23d0a90`