Expandable and scalable OCR pipeline
Project description
Overview
Nidaba is the central controller for the entire OGL OCR pipeline. It oversees and automates the process of converting raw images into citable collections of digitized texts.
It offers the following functionality:
Grayscale Conversion
Binarization utilizing Sauvola adaptive thresholding or leptonica’s Otsu thresholding with background normalization
Deskewing
Dewarping
Integration of tesseract and ocropus OCR engines
Merging multiple hOCR documents using scoring
As it is designed to use a common storage medium on network attached storage and the celery distributed task queue it scales nicely to multi-machine clusters.
Build
To build Nidaba run
$ pip install .
in the root directory or install using pypi:
$ pip install nibada
Some useful tasks have external dependencies. A good start is:
# apt-get install tesseract-ocr-grc libtesseract3 libleptonica-dev
Tests
Per default no dictionaries and OCR models necessary to runs the tests are installed. To download the necessary files run:
$ python setup.py download
$ python setup.py test
Tests for modules that call external programs, at the time only tesseract and ocropus, will be skipped if these aren’t installed.
Running
First edit (the installed) nidaba.yaml and celery.yaml to fit your needs. Have a look at the docs if you haven’t set up a celery-based application before.
Then start up the celery daemon with something like:
$ celery -A nidaba worker
Next jobs can be added to the pipeline using the nidaba executable:
$ nidaba batch --binarize "sauvola:whsize=10;whsize=20;whsize=30;whsize=40,factor=0.6" --ocr tesseract:eng -- ./input.tiff Preparing filestore....done. Building batch...done. 25d79a54-9d4a-4939-acb6-8e168d6dbc7c
Using the return code the current state of the job can be retrieved:
$ nidaba status 25d79a54-9d4a-4939-acb6-8e168d6dbc7c PENDING
When the job has been processed the status command will return a list of paths containing the final output:
$ nidaba status 25d79a54-9d4a-4939-acb6-8e168d6dbc7c SUCCESS input.tiff -> /home/mittagessen/OCR/97150c41-82a9-4935-8063-9295a2eb2a7f/input_img.rgb_to_gray_binarize.sauvola_10_0.35_ocr.tesseract_eng.tiff.hocr input.tiff -> /home/mittagessen/OCR/97150c41-82a9-4935-8063-9295a2eb2a7f/input_img.rgb_to_gray_binarize.sauvola_20_0.35_ocr.tesseract_eng.tiff.hocr input.tiff -> /home/mittagessen/OCR/97150c41-82a9-4935-8063-9295a2eb2a7f/input_img.rgb_to_gray_binarize.sauvola_30_0.35_ocr.tesseract_eng.tiff.hocr input.tiff -> /home/mittagessen/OCR/97150c41-82a9-4935-8063-9295a2eb2a7f/input_img.rgb_to_gray_binarize.sauvola_40_0.6_ocr.tesseract_eng.tiff.hocr
Documentation
Want to learn more? Read the Docs
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file nidaba-0.3.11.tar.gz.
File metadata
- Download URL: nidaba-0.3.11.tar.gz
- Upload date:
- Size: 64.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 49f89ab5f83e3bb64e1cd4b2a2c13933733aa50ac0827ecd3c9c414255098929 |
|
| MD5 | d71c3bdf0bfa88347d7ee45418c62769 |
|
| BLAKE2b-256 | a7b2482cb0832544784f6c053e46d116f99705010c287272035cb24926990b02 |
File details
Details for the file nidaba-0.3.11-py2-none-any.whl.
File metadata
- Download URL: nidaba-0.3.11-py2-none-any.whl
- Upload date:
- Size: 44.7 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f5c3756f223816471d19605b4d7e976f291b85e122adee2e17e7a9afcb57f130 |
|
| MD5 | 20eb3f37bffe46c77264b8dfbb2a588d |
|
| BLAKE2b-256 | a99456e4b68cb4ae9e0b3938a17ee2d7d95a37d99d6a322668e4ac346a38e12f |
