like grep but for code: fast and syntax-aware semantic code pattern search for many languages
Project description
semgrep
semgrep is a tool for easily detecting and preventing bugs and anti-patterns in
your codebase. It combines the convenience of grep with the correctness of
syntactical and semantic search. Developers, DevOps engineers, and security engineers
use semgrep to write code with confidence.
Try it now: https://semgrep.live
Overview
Language support:
| Python | Javascript | Go | Java | C | Typescript | PHP |
|---|---|---|---|---|---|---|
| ✅ | ✅ | ✅ | ✅ | ✅ | Coming... | Coming... |
Example patterns:
| Pattern | Matches |
|---|---|
$X == $X |
if (node.id == node.id): ... |
requests.get(..., verify=False, ...) |
requests.get(url, timeout=3, verify=False) |
os.system(...) |
from os import system; system('echo semgrep') |
$ELEMENT.innerHTML |
el.innerHTML = "<img src='x' onerror='alert(`XSS`)'>"; |
$TOKEN.SignedString([]byte("...")) |
ss, err := token.SignedString([]byte("HARDCODED KEY")) |
→ see more example patterns in the semgrep-rules repository
Installation
Install semgrep with Docker:
docker pull returntocorp/semgrep
On OSX, binaries are available via Homebrew:
brew install returntocorp/semgrep/semgrep
On Ubuntu, an install script is available here
Usage
Example Usage
Here is a simple Python example, test.py. We want to retrieve an object by ID:
def get_node(node_id, nodes):
for node in nodes:
if node.id == node.id: # Oops, supposed to be 'node_id'
return node
return None
This is a bug. Let's use semgrep to find bugs like it, using a simple search pattern: $X == $X. It will find all places in our code where the left- and right-hand sides of a comparison are the same expression:
$ docker run --rm -v "${PWD}:/home/repo" returntocorp/semgrep --lang python --pattern '$X == $X' test.py
test.py
rule:python.deadcode.eqeq-is-bad: useless comparison operation `node.id == node.id` or `node.id != node.id`.
3: if node.id == node.id: # Oops, supposed to be 'node_id'
r2c-developed Rules
You can use rules developed by r2c to search for issues in your codebase:
cd /path/to/code
docker run --rm -v "${PWD}:/home/repo" returntocorp/semgrep --config r2c
Custom Rules
You can also create your own rules:
cd /path/to/code
docker run --rm -v "${PWD}:/home/repo" returntocorp/semgrep --generate-config
docker run --rm -v "${PWD}:/home/repo" returntocorp/semgrep
Configuration
For simple patterns use the --lang and --pattern flags. This mode of
operation is useful for quickly iterating on a pattern on a single file or
folder:
docker run --rm -v "${PWD}:/home/repo" returntocorp/semgrep --lang javascript --pattern 'eval(...)' path/to/file.js
To fine-tune your searching, specify the --help flag:
docker run --rm returntocorp/semgrep --help
Configuration Files
For advanced configuration use the --config flag. This flag automagically
handles a multitude of input configuration types:
--config <file|folder|yaml_url|tarball_url|registy_name>
In the absence of this flag, a default configuration is loaded from .semgrep.yml
or multiple files matching .semgrep/**/*.yml.
Pattern Features
semgrep patterns make use of two primary features:
- Metavariables like
$X,$WIDGET, or$USERS_2. Metavariable names can only contain uppercase characters, or_, or digits, and must start with an uppercase character or_- names like$xor$some_valueare invalid. Metavariables are used to track a variable across a specific code scope. - The
...(ellipsis) operator. The ellipsis operator abstracts away sequences so you don't have to sweat the details of a particular code pattern.
For example,
$FILE = open(...)
will find all occurrences in your code where the result of an open() call is assigned
to a variable.
Composing Patterns
You can also construct rules by composing multiple patterns together.
Let's consider an example:
rules:
- id: open-never-closed
patterns:
- pattern: $FILE = open(...)
- pattern-not-inside: |
$FILE = open(...)
...
$FILE.close()
message: "file object opened without corresponding close"
languages: [python]
severity: ERROR
This rule looks for files that are opened but never closed. It accomplishes
this by looking for the open(...) pattern and not a following close()
pattern. The $FILE metavariable ensures that the same variable name is used
in the open and close calls. The ellipsis operator allows for any arguments
to be passed to open and any sequence of code statements in-between the open
and close calls. We don't care how open is called or what happens up to
a close call, we just need to make sure close is called.
For more information on rule fields like patterns and pattern-not-inside
see the configuration documentation.
Equivalences
Equivalences are another key concept in semgrep. semgrep automatically searches
for code that is semantically equivalent. For example, the following patterns
are semantically equivalent
subprocess.Popen(...)
from subprocess import Popen as sub_popen
result = sub_popen("ls")
For a full list of semgrep feature support by language see the
language matrix.
Registry
As mentioned above, you may also specify a registry name as configuration. r2c provides a registry of configuration files. These rules have been tuned on thousands of repositories using our analysis platform.
docker run --rm -v "${PWD}:/home/repo" returntocorp/semgrep --config r2c
Resources
semgreppresentation at HellaSecure and slides- Pattern features documentation
- Configuration files documentation
- Integrations
- Development
- Bug reports
Contribution
semgrep is LGPL-licensed, feel free to help out: CONTRIBUTING.
semgrep is a frontend to a larger program analysis library named pfff. pfff began and was open-sourced at Facebook but is now archived. The primary maintainer now works at r2c. semgrep was originally named sgrep and was renamed to avoid collisons with existing projects.
Commercial Support
semgrep is proudly supported by r2c. We're hiring!
Interested in a fully-supported, hosted version of semgrep? Drop your email and we'll ping you!
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file semgrep-0.6.0.tar.gz.
File metadata
- Download URL: semgrep-0.6.0.tar.gz
- Upload date:
- Size: 31.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.6.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | edd40260fb2be636f8c88fff8cd6f051b4ad52d592fa76e37a8675f7380efdd6 |
|
| MD5 | 80aea583bebe4c2bb20f8c6fc324e388 |
|
| BLAKE2b-256 | f47c425b7829fcf0213415d1a7f281e0fed4321a1af0799fb96c36799e8387dc |
File details
Details for the file semgrep-0.6.0-py36.py37.py38-none-macosx_10_14_x86_64.whl.
File metadata
- Download URL: semgrep-0.6.0-py36.py37.py38-none-macosx_10_14_x86_64.whl
- Upload date:
- Size: 18.8 MB
- Tags: Python 3.6, Python 3.7, Python 3.8, macOS 10.14+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.6.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c25324c413cfc73fc0e9fc3d127e620f58651f92ae6460ca0b9059d026a84805 |
|
| MD5 | 19128c06b9aca9f4bed8f5df9d23639a |
|
| BLAKE2b-256 | c39907ce6ec2b7f25527bfedf1dd8df804e42ca5a8c4bf374deaa818a348b8d1 |
