Skip to main content

Nasy Crawler Framework -- Never had such a pure crawler.

Project description

Table of Contents

Prologue

Never had such a pure crawler like this nacf.

Although I often write crawlers, I don’t like to use huge frameworks, such as scrapy, but prefer simple requests+bs4 or more general requests_html. However, these two are inconvenient for a crawler. E.g. Places, such as error retrying or parallel crawling, need to be handwritten by myself. It is not very difficult to write it while writing too much can be tedious. Hence I started writing this nacf (Nasy Crawler Framework), hoping to simplify some error retrying or parallel writing of crawlers.

Packages

Table 1: Packages
Package Version Description
requests-html 0.9.0 HTML Parsing for Humans.

Development Process

TODO Http Functions

DONE Get

CLOSED: [2018-12-25 Tue 17:36]

NEXT Post

TODO Bugs

DONE Fix an error from inspect.Parameter which caused the function parallel down. :err:1:

CLOSED: [2018-12-26 Wed 20:26]

Epoligue

History

Version 0.1.2

  • Data: <2018-12-26 Wed>
  • Fixed: inspect.Parameter error in last version.

Version 0.1.1

  • Data: <2018-12-26 Wed>
  • Ignored: An error caused by inspect.Parameter
  • Help Wanted: Can someone help me about the Parameter?

Version 0.1.0

  • Date: <2018-12-23 Sun>
  • Commemorate Version: First Version
    • Basic Functions.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nacf-0.1.2.tar.gz (13.5 kB view details)

Uploaded Source

Built Distribution

nacf-0.1.2-py3-none-any.whl (36.0 kB view details)

Uploaded Python 3

File details

Details for the file nacf-0.1.2.tar.gz.

File metadata

  • Download URL: nacf-0.1.2.tar.gz
  • Upload date:
  • Size: 13.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/0.12.10 CPython/3.7.1 Darwin/18.2.0

File hashes

Hashes for nacf-0.1.2.tar.gz
Algorithm Hash digest
SHA256 7d4cea95a4c50a63c171eaca0e3ee2f1e20e44459d959f10a1659c279f1d98ef
MD5 f032de855051c41f1523a71f63eec216
BLAKE2b-256 a9cd2482025a17b8b9b1fc82c2822c6908c2f00935b8def13ffdf00b3334e190

See more details on using hashes here.

File details

Details for the file nacf-0.1.2-py3-none-any.whl.

File metadata

  • Download URL: nacf-0.1.2-py3-none-any.whl
  • Upload date:
  • Size: 36.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/0.12.10 CPython/3.7.1 Darwin/18.2.0

File hashes

Hashes for nacf-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 4ffb1a60ca02187902afaca5013f89e4e826b66afac7339112fa02f53c006b5b
MD5 c45ead1ed466308583c2ace435853d0f
BLAKE2b-256 5471e98a3e6e1d8920e41eb6e30301f735f78d59181c657c5b937748029e2b36

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page