A webmining CLI tool & library for python.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique, Pauline Breteau
  • Tags webmining
  • Requires: Python >=3

Project description

Build Status

Minet

minet is a webmining CLI tool & library for python. It adopts a lo-fi approach to various webmining problems by letting you perform a variety of actions from the comfort of your command line. No database needed: raw data files will get you going.

In addition, minet also exposes its high-level programmatic interface as a library so you can tweak its behavior at will.

Features

  • Multithreaded, memory-efficient fetching from the web.
  • Multithreaded, scalable crawling using a comfy DSL.
  • Multiprocessed raw text content extraction from HTML pages.
  • Multiprocessed scraping from HTML pages using a comfy DSL.
  • URL-related heuristics utilities such as extraction, normalization and matching.
  • Data collection from various APIs such as CrowdTangle.

Installation

minet can be installed using pip:

pip install minet

Cookbook

To learn how to use minet and understand how it may fit your use cases, you should definitely check out our Cookbook.

Usage

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique, Pauline Breteau
  • Tags webmining
  • Requires: Python >=3

Release history Release notifications | RSS feed

3.1.0

3.0.0

2.0.8

2.0.7

2.0.6

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.5.1

1.5.0

1.4.1

1.4.0

1.3.5

1.3.3

1.3.2

1.3.1

1.3.0

1.2.2

1.2.1

1.2.0

1.1.10

1.1.9

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.1

1.0.0

1.0.0a55 pre-release

1.0.0a54 pre-release

1.0.0a53 pre-release

1.0.0a52 pre-release

1.0.0a51 pre-release

1.0.0a50 pre-release

1.0.0a49 pre-release

1.0.0a48 pre-release

1.0.0a47 pre-release

1.0.0a46 pre-release

1.0.0a45 pre-release

1.0.0a44 pre-release

1.0.0a43 pre-release

1.0.0a42 pre-release

1.0.0a41 pre-release

1.0.0a40 pre-release

1.0.0a39 pre-release

1.0.0a38 pre-release

1.0.0a37 pre-release

1.0.0a36 pre-release

1.0.0a35 pre-release

1.0.0a34 pre-release

1.0.0a33 pre-release

1.0.0a32 pre-release

1.0.0a31 pre-release

1.0.0a30 pre-release

1.0.0a29 pre-release

1.0.0a28 pre-release

1.0.0a27 pre-release

1.0.0a26 pre-release

1.0.0a25 pre-release

1.0.0a24 pre-release

1.0.0a23 pre-release

1.0.0a22 pre-release

1.0.0a21 pre-release

1.0.0a20 pre-release

1.0.0a19 pre-release

1.0.0a18 pre-release

1.0.0a17 pre-release

1.0.0a16 pre-release

1.0.0a15 pre-release

1.0.0a14 pre-release

1.0.0a13 pre-release

1.0.0a12 pre-release

1.0.0a11 pre-release

1.0.0a10 pre-release

1.0.0a9 pre-release

1.0.0a8 pre-release

1.0.0a7 pre-release

1.0.0a6 pre-release

1.0.0a5 pre-release

1.0.0a4 pre-release

1.0.0a3 pre-release

1.0.0a2 pre-release

1.0.0a1 pre-release

0.67.1

0.67.0

0.66.2

0.66.1

0.66.0

0.65.0

0.64.0

0.63.1

0.63.0

0.62.1

0.62.0

0.61.6

0.61.5

0.61.4

0.61.3

0.61.2

0.61.1

0.61.0

0.60.4

0.60.3

0.60.2

0.60.1

0.60.0

0.59.0

0.58.1

0.58.0

0.57.0

0.56.4

0.56.2

0.56.1

0.56.0

0.55.9

0.55.8

0.55.7

0.55.6

0.55.3

0.55.2

0.55.1

0.55.0

0.54.1

0.54.0

0.53.10

0.53.9

0.53.8

0.53.7

0.53.6

0.53.5

0.53.4

0.53.3

0.53.2

0.53.1

0.53.0

0.52.13

0.52.12

0.52.11

0.52.10

0.52.9

0.52.8

0.52.7

0.52.6

0.52.5

0.52.4

0.52.3

0.52.2

0.52.1

0.52.0

0.51.7

0.51.6

0.51.5

0.51.4

0.51.3

0.51.2

0.51.1

0.51.0

0.50.1

0.50.0

0.49.4

0.49.3

0.49.2

0.49.1

0.49.0

0.48.1

0.48.0

0.47.0

0.46.5

0.46.4

0.46.3

0.46.2

0.46.1

0.46.0

0.45.1

0.45.0

0.44.0

0.43.0

0.42.5

0.42.3

0.42.2

0.42.1

0.42.0

0.41.5

0.41.4

0.41.3

0.41.2

0.41.1

0.41.0

0.40.0

0.39.6

0.39.5

0.39.4

0.39.3

0.39.2

0.39.1

0.39.0

0.38.0

0.37.1

0.37.0

0.36.0

0.35.5

0.35.4

0.35.3

0.35.2

0.35.1

This version

0.35.0

0.34.1

0.34.0

0.33.0

0.32.5

0.32.4

0.32.3

0.32.2

0.32.1

0.32.0

0.31.1

0.31.0

0.30.1

0.30.0

0.29.1

0.29.0

0.28.0

0.27.0

0.26.0

0.25.0

0.24.2

0.24.1

0.24.0

0.23.0

0.22.0

0.21.1

0.21.0

0.20.2

0.20.1

0.20.0

0.19.0

0.18.0

0.17.0

0.16.0

0.15.0

0.14.3

0.14.2

0.14.1

0.14.0

0.13.0

0.12.0

0.11.0

0.10.0

0.9.1

0.9.0

0.8.2

0.8.1

0.8.0

0.7.0

0.6.0

0.5.0

0.4.1

0.4.0

0.3.0

0.2.0

0.1.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

minet-0.35.0.tar.gz (60.5 kB view details)

Uploaded Source

Built Distribution

minet-0.35.0-py3-none-any.whl (91.1 kB view details)

Uploaded Python 3

File details

Details for the file minet-0.35.0.tar.gz.

File metadata

  • Download URL: minet-0.35.0.tar.gz
  • Upload date:
  • Size: 60.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.9

File hashes

Hashes for minet-0.35.0.tar.gz
Algorithm Hash digest
SHA256 b522002c51cf96d871b30175fd28ecc1f55aef26e7828cfe9260e942a1ebcdf8
MD5 37c8495b881b4b99f85547b1cf1853a6
BLAKE2b-256 7d6cf4a78b557d447f68ba501bb41f9683336f3c37cb376a3e20ca3ff2ad9126

See more details on using hashes here.

Provenance

File details

Details for the file minet-0.35.0-py3-none-any.whl.

File metadata

  • Download URL: minet-0.35.0-py3-none-any.whl
  • Upload date:
  • Size: 91.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.9

File hashes

Hashes for minet-0.35.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f5fee8071b8a574dad6711c6bcfc7ea9c03f2bdf195513ef38debbce00dd404d
MD5 330813d37fdacc9313b0222dcfc7a9b3
BLAKE2b-256 341704f6c78008ebc681e398fc26ab75d8de71e70ef28c43282000abef915eef

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page