Property-based testing framework for Open API and GraphQL based apps
Project description
Discover API-breaking payloads, keep API documentation up-to-date, and increase confidence in your API
Documentation: https://schemathesis.readthedocs.io/en/stable/
Chat: https://discord.gg/R9ASRAmHnA
Schemathesis is a specification-based testing tool for OpenAPI and GraphQL apps based on the powerful Hypothesis framework.
Here are the key features:
- OpenAPI & GraphQL: Test a wide range of APIs with ease, regardless of the specification used.
- Positive & Negative Tests: Ensure your API handles valid and invalid inputs, incl. unexpected ones.
- Stateful Testing: Automatically generate sequences of API requests where subsequent requests build on previous ones for testing complex and interdependent scenarios.
- Session Replay: Quickly store and replay test sessions to easily investigate and resolve issues.
- Targeted Testing: Guide data generation towards specific metrics like response time or size. Uncover performance or resource usage issues and optimize API behavior under different conditions.
- Python Integration: Utilize native ASGI/WSGI support for faster testing your Python applications.
- Customization: Tune data generation, API response verification, and testing process to fit your needs.
- CI Integration: Run tests on every code change with Docker image and GitHub Action.
Testimonials
"The world needs modern, spec-based API tests, so we can deliver APIs as-designed. Schemathesis is the right tool for that job."
"Schemathesis is the only sane way to thoroughly test an API."
"The tool is absolutely amazing as it can do the negative scenario testing instead of me and much faster! Before I was doing the same tests in Postman client. But it's much slower and brings maintenance burden."
"Schemathesis is the best tool for fuzz testing of REST API on the market. We are at Red Hat use it for examining our applications in functional and integrations testing levels."
How does it work?
Schemathesis uses your API's schema to generate both valid and invalid test scenarios, helping you verify API compliance and catch potential issues. It also verifies examples from the schema itself.
Schemathesis generates high quality, diverse test data based on novel techniques like Swarm testing or Schema fuzzing, ensuring that your API is thoroughly tested and even the most elusive bugs are uncovered.
It's a versatile tool that works with any language, as long as you have an API schema in a supported format.
Learn more about how it works in our research paper.
Why use Schemathesis?
- Avoid Crashes: Discover API-breaking payloads and avoid crashes, database corruption, and hangs.
- Keep API Documentation Up-to-Date: With Schemathesis, you never have to worry about API consumers using outdated specifications or incorrect payload examples.
- Easy Debugging: Schemathesis provides you with a detailed failure report, along with a single cURL command to help you reproduce the problem instantly.
- Increased Confidence in API Stability: By thoroughly testing your API with Schemathesis, you can have peace of mind knowing that your API is functioning as intended.
- Thorough Testing Coverage: Schemathesis generates a large number of scenarios to test your API against, giving you a comprehensive view of its behavior and potential issues.
- Time-Saving: Schemathesis streamlines API testing, saving your time for other tasks.
Getting started
Schemathesis can be used as a CLI, a Python library, or as a SaaS.
- CLI: Quick and easy way to get started, for those who prefer the command line.
- Python Library: More control and customization, for developers integrating with their codebase.
- SaaS: No setup or installation, if you prefer an all-in-one solution with great visuals. Free tier included.
Installation
python -m pip install schemathesis
This command installs the st entrypoint.
You can also use our Docker image without installing Schemathesis as a Python package:
docker pull schemathesis/schemathesis:stable
Example
Command line
st run --checks all https://example.schemathesis.io/openapi.json
# Or
docker run schemathesis/schemathesis:stable \
run --checks all https://example.schemathesis.io/openapi.json
Python tests
import schemathesis
schema = schemathesis.from_uri("https://example.schemathesis.io/openapi.json")
@schema.parametrize()
def test_api(case):
case.call_and_validate()
Choose CLI for simplicity or Python package for greater flexibility. Both options run extensive tests and report failures with reproduction instructions.
💡 See a complete working example project in the /example directory.💡
GitHub Actions
If you use GitHub Actions, there is a native GitHub app that reports test results directly to your pull requests.
api-tests:
runs-on: ubuntu-20.04
steps:
# Runs Schemathesis tests with all checks enabled
- uses: schemathesis/action@v1
with:
# Your API schema location
schema: "http://localhost:5000/api/openapi.json"
# OPTIONAL. Your Schemathesis.io token
token: ${{ secrets.SCHEMATHESIS_TOKEN }}
Check our GitHub Action for more details.
Let's make it better together 🤝
We're always looking to make Schemathesis better, and your feedback is a crucial part of that journey! If you've got a few minutes, we'd love to hear your thoughts on your experience using Schemathesis.
Just follow this link to let us know what you think 💬
Thanks for helping us make Schemathesis even better! 👍
Commercial support
For assistance with integrating Schemathesis into your company workflows or improving its effectiveness, reach out to our support team at support@schemathesis.io. Additionally, we offer commercial support for those looking for extra assurance and priority assistance.
Contributing
Any contribution to development, testing, or any other area is highly appreciated and useful to the project. For guidance on how to contribute to Schemathesis, see the contributing guidelines.
Additional content
- Deriving Semantics-Aware Fuzzers from Web API Schemas by @Zac-HD and @Stranger6667
- An article about Schemathesis by @Stranger6667
- Effective API schemas testing from DevConf.cz by @Stranger6667
- How to use Schemathesis to test Flask API in GitHub Actions by @lina-is-here
- Testing APIFlask with schemathesis by @pamelafox
- A video from EuroPython 2020 by @hultner
- Schemathesis tutorial with an accompanying video by Red Hat
- Using Hypothesis and Schemathesis to Test FastAPI by @amalshaji
- A tutorial (RUS) about Schemathesis by @Stranger6667
License
This project is licensed under the terms of the MIT license.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file schemathesis-3.19.3.tar.gz.
File metadata
- Download URL: schemathesis-3.19.3.tar.gz
- Upload date:
- Size: 1.4 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.11.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 4c2ce9cd5cad61eee4d229a3c664ec59daa5c3621ff5aa72b317de8667486b78 |
|
| MD5 | 752c1f5301a4a696956b6d4fa2c0a175 |
|
| BLAKE2b-256 | 45e857cc5fd57bcd364321aa5a3f78d5208a58286da37e217161b9e50856a58f |
File details
Details for the file schemathesis-3.19.3-py3-none-any.whl.
File metadata
- Download URL: schemathesis-3.19.3-py3-none-any.whl
- Upload date:
- Size: 207.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.11.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | abd5596c594254df15f86a304ccc1a6b710a33f6a9daf4a0fb5eee6dcfba75fc |
|
| MD5 | 8fb30688c6b4b82668b2217640e7b23b |
|
| BLAKE2b-256 | e057ebd700508181771452276dbd26f94dcaf4b5209bfbcc08e20a15b3fce5ca |
