A webmining CLI tool & library for python.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique, Pauline Breteau
  • Tags webmining
  • Requires: Python >=3

Project description

Build Status

Minet

minet is a webmining CLI tool & library for python. It adopts a lo-fi approach to various webmining problems by letting you perform a variety of actions from the comfort of your command line. No database needed: raw data files will get you going.

In addition, minet also exposes its high-level programmatic interface as a library so you can tweak its behavior at will.

Features

  • Multithreaded, memory-efficient fetching from the web.
  • Multithreaded, scalable crawling using a comfy DSL.
  • Multiprocessed raw text content extraction from HTML pages.
  • Multiprocessed scraping from HTML pages using a comfy DSL.
  • URL-related heuristics utilities such as extraction, normalization and matching.
  • Data collection from various APIs such as CrowdTangle.

Installation

minet can be installed using pip:

pip install minet

Cookbook

To learn how to use minet and understand how it may fit your use cases, you should definitely check out our Cookbook.

Usage

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for Yomguithereal from gravatar.com Yomguithereal

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: MIT
  • Author: Jules Farjas, Guillaume Plique, Pauline Breteau
  • Tags webmining
  • Requires: Python >=3

Release history Release notifications | RSS feed

3.1.0

3.0.0

2.0.8

2.0.7

2.0.6

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.5.1

1.5.0

1.4.1

1.4.0

1.3.5

1.3.3

1.3.2

1.3.1

1.3.0

1.2.2

1.2.1

1.2.0

1.1.10

1.1.9

1.1.8

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.1

1.0.0

1.0.0a55 pre-release

1.0.0a54 pre-release

1.0.0a53 pre-release

1.0.0a52 pre-release

1.0.0a51 pre-release

1.0.0a50 pre-release

1.0.0a49 pre-release

1.0.0a48 pre-release

1.0.0a47 pre-release

1.0.0a46 pre-release

1.0.0a45 pre-release

1.0.0a44 pre-release

1.0.0a43 pre-release

1.0.0a42 pre-release

1.0.0a41 pre-release

1.0.0a40 pre-release

1.0.0a39 pre-release

1.0.0a38 pre-release

1.0.0a37 pre-release

1.0.0a36 pre-release

1.0.0a35 pre-release

1.0.0a34 pre-release

1.0.0a33 pre-release

1.0.0a32 pre-release

1.0.0a31 pre-release

1.0.0a30 pre-release

1.0.0a29 pre-release

1.0.0a28 pre-release

1.0.0a27 pre-release

1.0.0a26 pre-release

1.0.0a25 pre-release

1.0.0a24 pre-release

1.0.0a23 pre-release

1.0.0a22 pre-release

1.0.0a21 pre-release

1.0.0a20 pre-release

1.0.0a19 pre-release

1.0.0a18 pre-release

1.0.0a17 pre-release

1.0.0a16 pre-release

1.0.0a15 pre-release

1.0.0a14 pre-release

1.0.0a13 pre-release

1.0.0a12 pre-release

1.0.0a11 pre-release

1.0.0a10 pre-release

1.0.0a9 pre-release

1.0.0a8 pre-release

1.0.0a7 pre-release

1.0.0a6 pre-release

1.0.0a5 pre-release

1.0.0a4 pre-release

1.0.0a3 pre-release

1.0.0a2 pre-release

1.0.0a1 pre-release

0.67.1

0.67.0

0.66.2

0.66.1

0.66.0

0.65.0

0.64.0

0.63.1

0.63.0

0.62.1

0.62.0

0.61.6

0.61.5

0.61.4

0.61.3

0.61.2

0.61.1

0.61.0

0.60.4

0.60.3

0.60.2

0.60.1

0.60.0

0.59.0

0.58.1

0.58.0

0.57.0

0.56.4

0.56.2

0.56.1

0.56.0

0.55.9

0.55.8

0.55.7

0.55.6

0.55.3

0.55.2

0.55.1

0.55.0

0.54.1

0.54.0

0.53.10

0.53.9

0.53.8

0.53.7

0.53.6

0.53.5

0.53.4

0.53.3

0.53.2

0.53.1

0.53.0

0.52.13

0.52.12

0.52.11

0.52.10

0.52.9

0.52.8

0.52.7

0.52.6

0.52.5

0.52.4

0.52.3

0.52.2

0.52.1

0.52.0

0.51.7

0.51.6

0.51.5

0.51.4

0.51.3

0.51.2

0.51.1

0.51.0

0.50.1

0.50.0

0.49.4

0.49.3

0.49.2

0.49.1

0.49.0

0.48.1

0.48.0

0.47.0

0.46.5

0.46.4

0.46.3

0.46.2

0.46.1

0.46.0

0.45.1

0.45.0

0.44.0

0.43.0

0.42.5

0.42.3

0.42.2

0.42.1

0.42.0

0.41.5

0.41.4

0.41.3

0.41.2

0.41.1

0.41.0

0.40.0

0.39.6

0.39.5

0.39.4

0.39.3

0.39.2

0.39.1

0.39.0

0.38.0

0.37.1

0.37.0

0.36.0

0.35.5

0.35.4

0.35.3

0.35.2

0.35.1

0.35.0

0.34.1

0.34.0

0.33.0

0.32.5

0.32.4

This version

0.32.3

0.32.2

0.32.1

0.32.0

0.31.1

0.31.0

0.30.1

0.30.0

0.29.1

0.29.0

0.28.0

0.27.0

0.26.0

0.25.0

0.24.2

0.24.1

0.24.0

0.23.0

0.22.0

0.21.1

0.21.0

0.20.2

0.20.1

0.20.0

0.19.0

0.18.0

0.17.0

0.16.0

0.15.0

0.14.3

0.14.2

0.14.1

0.14.0

0.13.0

0.12.0

0.11.0

0.10.0

0.9.1

0.9.0

0.8.2

0.8.1

0.8.0

0.7.0

0.6.0

0.5.0

0.4.1

0.4.0

0.3.0

0.2.0

0.1.0

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

minet-0.32.3.tar.gz (59.4 kB view details)

Uploaded Source

Built Distribution

minet-0.32.3-py3-none-any.whl (89.8 kB view details)

Uploaded Python 3

File details

Details for the file minet-0.32.3.tar.gz.

File metadata

  • Download URL: minet-0.32.3.tar.gz
  • Upload date:
  • Size: 59.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.9

File hashes

Hashes for minet-0.32.3.tar.gz
Algorithm Hash digest
SHA256 c344d8d5df237dd0d16924f867354e331c1b38d855646e19df4ba944f64c1053
MD5 b487566b4b115798b560e4c11af93bfd
BLAKE2b-256 d6af0d89ca69ae391274fc56d0ce4356fd7b934cdd1616ecdab1cc740565797d

See more details on using hashes here.

Provenance

File details

Details for the file minet-0.32.3-py3-none-any.whl.

File metadata

  • Download URL: minet-0.32.3-py3-none-any.whl
  • Upload date:
  • Size: 89.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/40.6.2 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.9

File hashes

Hashes for minet-0.32.3-py3-none-any.whl
Algorithm Hash digest
SHA256 b9ec3f4c86432aac35509028d5603ab833cb6114d5ff0853afd8e77fad742e2c
MD5 1870b6790c9f5d5a2e8d3893cb23cfb0
BLAKE2b-256 d9c23fc454d853ddac7069262ab9cc3e36e93a1d93bd1f92b86e7a786d59d4e1

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page