A webmining CLI tool & library for python.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique
  • Tags webmining
  • Requires: Python >=3

Project description

Build Status

Minet

minet is a webmining CLI tool & library for python. It adopts a lo-fi approach to various webmining problems by letting you perform a variety of actions from the comfort of your command line. No database needed: raw data files will get you going.

In addition, minet also exposes its high-level programmatic interface as a library so you can tweak its behavior at will.

Features

  • Multithreaded, memory-efficient fetching from the web.
  • Multithreaded, scalable crawling using a comfy DSL.
  • Multiprocessed raw text content extraction from HTML pages.
  • Multiprocessed scraping from HTML pages using a comfy DSL.
  • URL-related heuristics utilities such as extraction, normalization and matching.
  • Data collection from various APIs such as CrowdTangle.

Installation

minet can be installed using pip:

pip install minet

Cookbook

To learn how to use minet and understand how it may fit your use cases, you should definitely check out our Cookbook.

Usage

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique
  • Tags webmining
  • Requires: Python >=3

Release history Release notifications | RSS feed

3.1.0

3.0.0

2.0.8

2.0.7

2.0.6

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.5.1

1.5.0

1.4.1

1.4.0

1.3.5

1.3.3

1.3.2

1.3.1

1.3.0

1.2.2

1.2.1

1.2.0

1.1.10

1.1.9

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.1

1.0.0

1.0.0a55 pre-release

1.0.0a54 pre-release

1.0.0a53 pre-release

1.0.0a52 pre-release

1.0.0a51 pre-release

1.0.0a50 pre-release

1.0.0a49 pre-release

1.0.0a48 pre-release

1.0.0a47 pre-release

1.0.0a46 pre-release

1.0.0a45 pre-release

1.0.0a44 pre-release

1.0.0a43 pre-release

1.0.0a42 pre-release

1.0.0a41 pre-release

1.0.0a40 pre-release

1.0.0a39 pre-release

1.0.0a38 pre-release

1.0.0a37 pre-release

1.0.0a36 pre-release

1.0.0a35 pre-release

1.0.0a34 pre-release

1.0.0a33 pre-release

1.0.0a32 pre-release

1.0.0a31 pre-release

1.0.0a30 pre-release

1.0.0a29 pre-release

1.0.0a28 pre-release

1.0.0a27 pre-release

1.0.0a26 pre-release

1.0.0a25 pre-release

1.0.0a24 pre-release

1.0.0a23 pre-release

1.0.0a22 pre-release

1.0.0a21 pre-release

1.0.0a20 pre-release

1.0.0a19 pre-release

1.0.0a18 pre-release

1.0.0a17 pre-release

1.0.0a16 pre-release

1.0.0a15 pre-release

1.0.0a14 pre-release

1.0.0a13 pre-release

1.0.0a12 pre-release

1.0.0a11 pre-release

1.0.0a10 pre-release

1.0.0a9 pre-release

1.0.0a8 pre-release

1.0.0a7 pre-release

1.0.0a6 pre-release

1.0.0a5 pre-release

1.0.0a4 pre-release

1.0.0a3 pre-release

1.0.0a2 pre-release

1.0.0a1 pre-release

0.67.1

0.67.0

0.66.2

0.66.1

0.66.0

0.65.0

0.64.0

0.63.1

0.63.0

0.62.1

0.62.0

0.61.6

0.61.5

0.61.4

0.61.3

0.61.2

0.61.1

0.61.0

0.60.4

0.60.3

0.60.2

0.60.1

0.60.0

0.59.0

0.58.1

0.58.0

0.57.0

0.56.4

0.56.2

0.56.1

0.56.0

0.55.9

0.55.8

0.55.7

0.55.6

0.55.3

0.55.2

0.55.1

0.55.0

0.54.1

0.54.0

0.53.10

0.53.9

0.53.8

0.53.7

0.53.6

0.53.5

0.53.4

0.53.3

0.53.2

0.53.1

0.53.0

0.52.13

0.52.12

0.52.11

0.52.10

0.52.9

0.52.8

0.52.7

0.52.6

0.52.5

0.52.4

0.52.3

0.52.2

0.52.1

0.52.0

0.51.7

0.51.6

0.51.5

0.51.4

0.51.3

0.51.2

0.51.1

0.51.0

0.50.1

0.50.0

0.49.4

0.49.3

0.49.2

0.49.1

0.49.0

0.48.1

0.48.0

0.47.0

0.46.5

0.46.4

0.46.3

0.46.2

0.46.1

0.46.0

0.45.1

0.45.0

0.44.0

0.43.0

0.42.5

0.42.3

0.42.2

0.42.1

0.42.0

0.41.5

0.41.4

0.41.3

0.41.2

0.41.1

0.41.0

0.40.0

0.39.6

0.39.5

0.39.4

0.39.3

0.39.2

0.39.1

0.39.0

0.38.0

0.37.1

0.37.0

0.36.0

0.35.5

0.35.4

0.35.3

0.35.2

0.35.1

0.35.0

0.34.1

0.34.0

0.33.0

0.32.5

0.32.4

0.32.3

0.32.2

0.32.1

0.32.0

0.31.1

0.31.0

0.30.1

0.30.0

0.29.1

0.29.0

This version

0.28.0

0.27.0

0.26.0

0.25.0

0.24.2

0.24.1

0.24.0

0.23.0

0.22.0

0.21.1

0.21.0

0.20.2

0.20.1

0.20.0

0.19.0

0.18.0

0.17.0

0.16.0

0.15.0

0.14.3

0.14.2

0.14.1

0.14.0

0.13.0

0.12.0

0.11.0

0.10.0

0.9.1

0.9.0

0.8.2

0.8.1

0.8.0

0.7.0

0.6.0

0.5.0

0.4.1

0.4.0

0.3.0

0.2.0

0.1.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

minet-0.28.0.tar.gz (50.4 kB view details)

Uploaded Source

Built Distribution

minet-0.28.0-py3-none-any.whl (72.0 kB view details)

Uploaded Python 3

File details

Details for the file minet-0.28.0.tar.gz.

File metadata

  • Download URL: minet-0.28.0.tar.gz
  • Upload date:
  • Size: 50.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5

File hashes

Hashes for minet-0.28.0.tar.gz
Algorithm Hash digest
SHA256 d8b6ec1a92fdcf37aeb4df1d33f64c232d571b46e5ce5e68e6b2e5f86365e78b
MD5 7f5d146e29649580511688ea75c31f59
BLAKE2b-256 38ab05bd2355df7237bd75e8d721a47dfad1feb4e35ff7f7c6135e2b91561e2e

See more details on using hashes here.

Provenance

File details

Details for the file minet-0.28.0-py3-none-any.whl.

File metadata

  • Download URL: minet-0.28.0-py3-none-any.whl
  • Upload date:
  • Size: 72.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5

File hashes

Hashes for minet-0.28.0-py3-none-any.whl
Algorithm Hash digest
SHA256 47ef58e1fda6d5ee0ad07423742f58f090b2e57af05cb86eddd451e2076b9e9a
MD5 d6966ea2af0101a5bb139e7f63d686dc
BLAKE2b-256 c1132c2c478f841beaa93142a65a798ea152d809d777efce003fb611f1cc7568

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page