A webmining CLI tool & library for python.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique, Pauline Breteau
  • Tags webmining
  • Requires: Python >=3

Project description

Build Status

Minet

minet is a webmining CLI tool & library for python. It adopts a lo-fi approach to various webmining problems by letting you perform a variety of actions from the comfort of your command line. No database needed: raw data files will get you going.

In addition, minet also exposes its high-level programmatic interface as a library so you can tweak its behavior at will.

Features

  • Multithreaded, memory-efficient fetching from the web.
  • Multithreaded, scalable crawling using a comfy DSL.
  • Multiprocessed raw text content extraction from HTML pages.
  • Multiprocessed scraping from HTML pages using a comfy DSL.
  • URL-related heuristics utilities such as extraction, normalization and matching.
  • Data collection from various APIs such as CrowdTangle.

Installation

minet can be installed using pip:

pip install minet

Cookbook

To learn how to use minet and understand how it may fit your use cases, you should definitely check out our Cookbook.

Usage

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique, Pauline Breteau
  • Tags webmining
  • Requires: Python >=3

Release history Release notifications | RSS feed

3.1.0

3.0.0

2.0.8

2.0.7

2.0.6

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.5.1

1.5.0

1.4.1

1.4.0

1.3.5

1.3.3

1.3.2

1.3.1

1.3.0

1.2.2

1.2.1

1.2.0

1.1.10

1.1.9

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.1

1.0.0

1.0.0a55 pre-release

1.0.0a54 pre-release

1.0.0a53 pre-release

1.0.0a52 pre-release

1.0.0a51 pre-release

1.0.0a50 pre-release

1.0.0a49 pre-release

1.0.0a48 pre-release

1.0.0a47 pre-release

1.0.0a46 pre-release

1.0.0a45 pre-release

1.0.0a44 pre-release

1.0.0a43 pre-release

1.0.0a42 pre-release

1.0.0a41 pre-release

1.0.0a40 pre-release

1.0.0a39 pre-release

1.0.0a38 pre-release

1.0.0a37 pre-release

1.0.0a36 pre-release

1.0.0a35 pre-release

1.0.0a34 pre-release

1.0.0a33 pre-release

1.0.0a32 pre-release

1.0.0a31 pre-release

1.0.0a30 pre-release

1.0.0a29 pre-release

1.0.0a28 pre-release

1.0.0a27 pre-release

1.0.0a26 pre-release

1.0.0a25 pre-release

1.0.0a24 pre-release

1.0.0a23 pre-release

1.0.0a22 pre-release

1.0.0a21 pre-release

1.0.0a20 pre-release

1.0.0a19 pre-release

1.0.0a18 pre-release

1.0.0a17 pre-release

1.0.0a16 pre-release

1.0.0a15 pre-release

1.0.0a14 pre-release

1.0.0a13 pre-release

1.0.0a12 pre-release

1.0.0a11 pre-release

1.0.0a10 pre-release

1.0.0a9 pre-release

1.0.0a8 pre-release

1.0.0a7 pre-release

1.0.0a6 pre-release

1.0.0a5 pre-release

1.0.0a4 pre-release

1.0.0a3 pre-release

1.0.0a2 pre-release

1.0.0a1 pre-release

0.67.1

0.67.0

0.66.2

0.66.1

0.66.0

0.65.0

0.64.0

0.63.1

0.63.0

0.62.1

0.62.0

0.61.6

0.61.5

0.61.4

0.61.3

0.61.2

0.61.1

0.61.0

0.60.4

0.60.3

0.60.2

0.60.1

0.60.0

0.59.0

0.58.1

0.58.0

0.57.0

0.56.4

0.56.2

0.56.1

0.56.0

0.55.9

0.55.8

0.55.7

0.55.6

0.55.3

0.55.2

0.55.1

0.55.0

0.54.1

0.54.0

0.53.10

0.53.9

0.53.8

0.53.7

0.53.6

0.53.5

0.53.4

0.53.3

0.53.2

0.53.1

0.53.0

0.52.13

0.52.12

0.52.11

0.52.10

0.52.9

0.52.8

0.52.7

0.52.6

0.52.5

0.52.4

0.52.3

0.52.2

0.52.1

0.52.0

0.51.7

0.51.6

0.51.5

0.51.4

0.51.3

0.51.2

0.51.1

0.51.0

0.50.1

0.50.0

0.49.4

0.49.3

0.49.2

0.49.1

0.49.0

0.48.1

0.48.0

0.47.0

0.46.5

0.46.4

0.46.3

0.46.2

0.46.1

0.46.0

0.45.1

0.45.0

0.44.0

0.43.0

0.42.5

0.42.3

0.42.2

0.42.1

0.42.0

0.41.5

0.41.4

0.41.3

0.41.2

0.41.1

0.41.0

0.40.0

0.39.6

0.39.5

0.39.4

0.39.3

0.39.2

0.39.1

0.39.0

0.38.0

0.37.1

0.37.0

0.36.0

0.35.5

0.35.4

This version

0.35.3

0.35.2

0.35.1

0.35.0

0.34.1

0.34.0

0.33.0

0.32.5

0.32.4

0.32.3

0.32.2

0.32.1

0.32.0

0.31.1

0.31.0

0.30.1

0.30.0

0.29.1

0.29.0

0.28.0

0.27.0

0.26.0

0.25.0

0.24.2

0.24.1

0.24.0

0.23.0

0.22.0

0.21.1

0.21.0

0.20.2

0.20.1

0.20.0

0.19.0

0.18.0

0.17.0

0.16.0

0.15.0

0.14.3

0.14.2

0.14.1

0.14.0

0.13.0

0.12.0

0.11.0

0.10.0

0.9.1

0.9.0

0.8.2

0.8.1

0.8.0

0.7.0

0.6.0

0.5.0

0.4.1

0.4.0

0.3.0

0.2.0

0.1.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

minet-0.35.3.tar.gz (60.5 kB view details)

Uploaded Source

Built Distribution

minet-0.35.3-py3-none-any.whl (91.1 kB view details)

Uploaded Python 3

File details

Details for the file minet-0.35.3.tar.gz.

File metadata

  • Download URL: minet-0.35.3.tar.gz
  • Upload date:
  • Size: 60.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.9

File hashes

Hashes for minet-0.35.3.tar.gz
Algorithm Hash digest
SHA256 34b84f7c2673ff64c1c3879e200c2d448be2f4958049e007429bcb64cb29cad0
MD5 aa171b014f2e8b10cb31e4be7fb54daf
BLAKE2b-256 c385f5cc28be9cd0a827838677a59e1013932633cd6c0e9a3d78b71006ab1a51

See more details on using hashes here.

Provenance

File details

Details for the file minet-0.35.3-py3-none-any.whl.

File metadata

  • Download URL: minet-0.35.3-py3-none-any.whl
  • Upload date:
  • Size: 91.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.9

File hashes

Hashes for minet-0.35.3-py3-none-any.whl
Algorithm Hash digest
SHA256 a8c0fd9a392e3ef796c1a778c5765f5d4582f5238b58d7d88ef2519ff2c29d65
MD5 621696c0835f9d7882d7ec322eebbeac
BLAKE2b-256 04fd6521bda45c4424e8a5029d1ef7ced5279b9c0802b397ec54db307367a0c6

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page