A webmining CLI tool & library for python.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique
  • Tags webmining
  • Requires: Python >=3

Project description

Build Status

Minet

minet is a webmining CLI tool & library for python. It adopts a lo-fi approach to various webmining problems by letting you perform a variety of actions from the comfort of your command line. No database needed: raw data files will get you going.

In addition, minet also exposes its high-level programmatic interface as a library so you can tweak its behavior at will.

Features

  • Multithreaded, memory-efficient fetching from the web.
  • Multithreaded, scalable crawling using a comfy DSL.
  • Multiprocessed raw text content extraction from HTML pages.
  • Multiprocessed scraping from HTML pages using a comfy DSL.
  • URL-related heuristics utilities such as extraction, normalization and matching.
  • Data collection from various APIs such as CrowdTangle.

Installation

minet can be installed using pip:

pip install minet

Cookbook

To learn how to use minet and understand how it may fit your use cases, you should definitely check out our Cookbook.

Usage

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique
  • Tags webmining
  • Requires: Python >=3

Release history Release notifications | RSS feed

3.1.0

3.0.0

2.0.8

2.0.7

2.0.6

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.5.1

1.5.0

1.4.1

1.4.0

1.3.5

1.3.3

1.3.2

1.3.1

1.3.0

1.2.2

1.2.1

1.2.0

1.1.10

1.1.9

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.1

1.0.0

1.0.0a55 pre-release

1.0.0a54 pre-release

1.0.0a53 pre-release

1.0.0a52 pre-release

1.0.0a51 pre-release

1.0.0a50 pre-release

1.0.0a49 pre-release

1.0.0a48 pre-release

1.0.0a47 pre-release

1.0.0a46 pre-release

1.0.0a45 pre-release

1.0.0a44 pre-release

1.0.0a43 pre-release

1.0.0a42 pre-release

1.0.0a41 pre-release

1.0.0a40 pre-release

1.0.0a39 pre-release

1.0.0a38 pre-release

1.0.0a37 pre-release

1.0.0a36 pre-release

1.0.0a35 pre-release

1.0.0a34 pre-release

1.0.0a33 pre-release

1.0.0a32 pre-release

1.0.0a31 pre-release

1.0.0a30 pre-release

1.0.0a29 pre-release

1.0.0a28 pre-release

1.0.0a27 pre-release

1.0.0a26 pre-release

1.0.0a25 pre-release

1.0.0a24 pre-release

1.0.0a23 pre-release

1.0.0a22 pre-release

1.0.0a21 pre-release

1.0.0a20 pre-release

1.0.0a19 pre-release

1.0.0a18 pre-release

1.0.0a17 pre-release

1.0.0a16 pre-release

1.0.0a15 pre-release

1.0.0a14 pre-release

1.0.0a13 pre-release

1.0.0a12 pre-release

1.0.0a11 pre-release

1.0.0a10 pre-release

1.0.0a9 pre-release

1.0.0a8 pre-release

1.0.0a7 pre-release

1.0.0a6 pre-release

1.0.0a5 pre-release

1.0.0a4 pre-release

1.0.0a3 pre-release

1.0.0a2 pre-release

1.0.0a1 pre-release

0.67.1

0.67.0

0.66.2

0.66.1

0.66.0

0.65.0

0.64.0

0.63.1

0.63.0

0.62.1

0.62.0

0.61.6

0.61.5

0.61.4

0.61.3

0.61.2

0.61.1

0.61.0

0.60.4

0.60.3

0.60.2

0.60.1

0.60.0

0.59.0

0.58.1

0.58.0

0.57.0

0.56.4

0.56.2

0.56.1

0.56.0

0.55.9

0.55.8

0.55.7

0.55.6

0.55.3

0.55.2

0.55.1

0.55.0

0.54.1

0.54.0

0.53.10

0.53.9

0.53.8

0.53.7

0.53.6

0.53.5

0.53.4

0.53.3

0.53.2

0.53.1

0.53.0

0.52.13

0.52.12

0.52.11

0.52.10

0.52.9

0.52.8

0.52.7

0.52.6

0.52.5

0.52.4

0.52.3

0.52.2

0.52.1

0.52.0

0.51.7

0.51.6

0.51.5

0.51.4

0.51.3

0.51.2

0.51.1

0.51.0

0.50.1

0.50.0

0.49.4

0.49.3

0.49.2

0.49.1

0.49.0

0.48.1

0.48.0

0.47.0

0.46.5

0.46.4

0.46.3

0.46.2

0.46.1

0.46.0

0.45.1

0.45.0

0.44.0

0.43.0

0.42.5

0.42.3

0.42.2

0.42.1

0.42.0

0.41.5

0.41.4

0.41.3

0.41.2

0.41.1

0.41.0

0.40.0

0.39.6

0.39.5

0.39.4

0.39.3

0.39.2

0.39.1

0.39.0

0.38.0

0.37.1

0.37.0

0.36.0

0.35.5

0.35.4

0.35.3

0.35.2

0.35.1

0.35.0

0.34.1

0.34.0

0.33.0

0.32.5

0.32.4

0.32.3

0.32.2

0.32.1

0.32.0

0.31.1

This version

0.31.0

0.30.1

0.30.0

0.29.1

0.29.0

0.28.0

0.27.0

0.26.0

0.25.0

0.24.2

0.24.1

0.24.0

0.23.0

0.22.0

0.21.1

0.21.0

0.20.2

0.20.1

0.20.0

0.19.0

0.18.0

0.17.0

0.16.0

0.15.0

0.14.3

0.14.2

0.14.1

0.14.0

0.13.0

0.12.0

0.11.0

0.10.0

0.9.1

0.9.0

0.8.2

0.8.1

0.8.0

0.7.0

0.6.0

0.5.0

0.4.1

0.4.0

0.3.0

0.2.0

0.1.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

minet-0.31.0.tar.gz (59.0 kB view details)

Uploaded Source

Built Distribution

minet-0.31.0-py3-none-any.whl (88.6 kB view details)

Uploaded Python 3

File details

Details for the file minet-0.31.0.tar.gz.

File metadata

  • Download URL: minet-0.31.0.tar.gz
  • Upload date:
  • Size: 59.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.23.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.9

File hashes

Hashes for minet-0.31.0.tar.gz
Algorithm Hash digest
SHA256 652759d273bb20c111c4af4e1b862dc36494e4fe105254b3222b9eea094af7c5
MD5 ca8de2908b1abe4fa263c1932646184e
BLAKE2b-256 004c5bf3da8753c8761fd40b770c645af75d75af80a43844562b71c33782e47a

See more details on using hashes here.

Provenance

File details

Details for the file minet-0.31.0-py3-none-any.whl.

File metadata

  • Download URL: minet-0.31.0-py3-none-any.whl
  • Upload date:
  • Size: 88.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.23.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.9

File hashes

Hashes for minet-0.31.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1ea7fee27086b9831921467266dcfaef0fe6210333237c878bc2fcf703ef0ad2
MD5 70d8613d35f5f739d966566a6046b0d8
BLAKE2b-256 340cb8bc95d6d95f15dd8b205ef986e8fbe2d5f555aaaf6015caccc0382a8476

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page