Skip to main content

Nasy Crawler Framework -- Never had such a pure crawler.

Project description

Table of Contents

Prologue

Never had such a pure crawler like this nacf.

Although I often write crawlers, I don’t like to use huge frameworks, such as scrapy, but prefer simple requests+bs4 or more general requests_html. However, these two are inconvenient for a crawler. E.g. Places, such as error retrying or parallel crawling, need to be handwritten by myself. It is not very difficult to write it while writing too much can be tedious. Hence I started writing this nacf (Nasy Crawler Framework), hoping to simplify some error retrying or parallel writing of crawlers.

Packages

Table 1: Packages
Package Version Description
requests-html 0.9.0 HTML Parsing for Humans.

Development Process

TODO Http Functions

DONE Get

CLOSED: [2018-12-25 Tue 17:36]

NEXT Post

TODO Bugs

TODO Fix an error from inspect.Parameter which caused the function parallel down.

Epoligue

History

Version 0.1.1

  • Data: <2018-12-26 Wed>
  • Ignored: An error caused by inspect.Parameter
  • Help Wanted: Can someone help me about the Parameter?

Version 0.1.0

  • Date: <2018-12-23 Sun>
  • Commemorate Version: First Version
    • Basic Functions.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nacf-0.1.1.tar.gz (13.4 kB view details)

Uploaded Source

Built Distribution

nacf-0.1.1-py3-none-any.whl (36.1 kB view details)

Uploaded Python 3

File details

Details for the file nacf-0.1.1.tar.gz.

File metadata

  • Download URL: nacf-0.1.1.tar.gz
  • Upload date:
  • Size: 13.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/0.12.10 CPython/3.7.1 Darwin/18.2.0

File hashes

Hashes for nacf-0.1.1.tar.gz
Algorithm Hash digest
SHA256 1a355a2db5bcc5a7b00d4ee81191e62073035fdb05e6d7139df007bf9dae5a53
MD5 fdc1cbb88ec5706898fb996a5b28e9f1
BLAKE2b-256 1bc9dc833b07b735f3df100251007ddd1a2131cad147c3ecc88595102616fd2a

See more details on using hashes here.

File details

Details for the file nacf-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: nacf-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 36.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/0.12.10 CPython/3.7.1 Darwin/18.2.0

File hashes

Hashes for nacf-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 9b1216b0986358212588e6ead8ba35751ef5762f0a71fb06cdad6fb06cef1aca
MD5 17a5044978b0788f6859c163c496385a
BLAKE2b-256 e90ac7c3ac57a4fd8b9b78fe68abd85fbe6ee347b32122cd3731ff8d750c2080

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page