Skip to main content

Downloads RSS/Atom feeds and converts them to Maildir messages

Project description

https://travis-ci.org/wosc/rsspull.png

rsspull downloads and parses RSS and Atom feeds, converts posts into email messages, and stores them in Maildirs. This way you can read your feeds using an email client (I use claws, so I can access mailing lists, newsgroups and feeds all in one application). The heavy lifting is performed by the awesome feedparser library.

Installation

rsspull requires at least Python 3.3. You can install it from PyPI like this:

$ pip install ws.rsspull

You need to create a configuration file in ~/.rsspull/config, like:

[global]
target = ~/Maildir/rss
target_type = maildir
logfile = ~/.rsspull/log
workers = 1
user_agent = optional custom header

For low-volume applications you can also send the emails via SMTP instead:

[global]
target = test@example.com
target_type = smtp://my-smtp-server.example.com

List the feeds you want to pull in an OPML file at ~/.rsspull/feeds.opml:

<?xml version="1.0" encoding="utf-8"?>
<opml version="1.1">
  <head>
    <title>feeds.opml</title>
    <ownerName>rsspull</ownerName>
    <ownerEmail>rsspull@localhost</ownerEmail>
  </head>
  <body>
    <outline text="tech">
      <outline text="ongoing" xmlUrl="http://www.tbray.org/ongoing/ongoing.atom" />
    </outline>
    <outline text="general">
      <outline text="heisec" xmlUrl="http://www.heise.de/security/news/news.rdf" />
      <outline text="trac_example" xmlUrl="https://example.com/trac/timeline?milestone=on&amp;ticket=on&amp;changeset=on&amp;wiki=on&amp;max=10&amp;daysback=90&amp;format=rss" auth="user:password"/>
    </outline>
  </body>
</opml>

Notes about the format:

  • You can group your feeds (using nested <outlines>), this does not matter to rsspull.

  • The text attribute is used as the folder name, relative to the maildir setting in ~/.rsspull/config. If you run rsspull on an IMAP server, you can create folder hierarchies by using something like text=".rss.tech.ongoing" (check you IMAP server documentation how it represents folder hierarchies. The example with dots works for Courier, others might use actual subfolders, and so on).

  • Basic authentication is supported with the auth attribute.

  • You can use file:// URLs.

Then simply run:

$ rsspull

to download the feeds.

You can pass rsspull --confdir /path/to/config to use a different location than ~/.rsspull.

Features

  • Uses If-Modified-Since HTTP headers to avoid downloading a feed that has not changed.

  • Addds a Content-Location header to each message that contains the URL of that post. I bound the following script to [return] in claws to open the current entry in a webbrowser:

    #!/bin/bash
    URL=`sed -ne '/^Content-Location/s/.*: //p' $1 | head -n 1`
    if [ -n "$URL" ]; then
        mozilla $URL &> /dev/null
    fi
  • Entries are converted to multipart messages, one with the original HTML and one converted into markdown-like plaintext (via html2text), so most of the time opening in a browser isn’t even necessary since you can read the post right in the email client.

  • Can use several worker threads to download and parse feeds to increase performance, since quite some time is spent waiting for downloads to complete (the workers setting in ~/.rsspull/config).

  • Has been in daily usage since 2007, so it definitely Works For Me(tm).

CHANGES

2.5.0 (2023-04-16)

Changes

  • Make user-agent header configurable (useragent)

2.4.0 (2023-04-04)

Changes

  • Drop Python-2 support (py2)

  • Switch from setup.py to pyproject.toml (wheel)

2.3.1 (2023-03-20)

  • Escape ‘/’ in feed names to avoid prevent file names

2.3.0 (2018-04-03)

  • Make SMTP server configurable.

2.2.1 (2018-03-30)

  • Fix Python-3 unicode issues.

2.2 (2018-03-24)

  • Support sending messages via SMTP as an alternative to writing into Maildir.

  • Add commandline option to set config directory.

2.1 (2018-03-06)

  • Switch from urllib2 to requests.

  • Make Python-3 compatible.

2.0 (2014-02-22)

  • Switch from spawning curl to using urllib2.

  • Package as egg.

1.2 (2013-10-03)

  • Use stdlib xml parser instead of python-xml.

1.1 (2009-05-23)

  • Write messages directly to Maildir instead of using procmail.

  • Send messages as multipart, text and html.

1.0 (2007-03-31)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ws.rsspull-2.5.0.tar.gz (47.3 kB view details)

Uploaded Source

Built Distribution

ws.rsspull-2.5.0-py3-none-any.whl (48.4 kB view details)

Uploaded Python 3

File details

Details for the file ws.rsspull-2.5.0.tar.gz.

File metadata

  • Download URL: ws.rsspull-2.5.0.tar.gz
  • Upload date:
  • Size: 47.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-httpx/0.23.3

File hashes

Hashes for ws.rsspull-2.5.0.tar.gz
Algorithm Hash digest
SHA256 91d52bcdbcf30ef12ccc3cadf9d167b3c4bde3da5da10bbb203b794391194dff
MD5 031f0ccaa7c59e72a2dc14399711fb18
BLAKE2b-256 df3dbfefc9acd87f6222b9e940b20b81d6ac241b636686426ade7fdeb4d8a33c

See more details on using hashes here.

File details

Details for the file ws.rsspull-2.5.0-py3-none-any.whl.

File metadata

  • Download URL: ws.rsspull-2.5.0-py3-none-any.whl
  • Upload date:
  • Size: 48.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: python-httpx/0.23.3

File hashes

Hashes for ws.rsspull-2.5.0-py3-none-any.whl
Algorithm Hash digest
SHA256 b5d45d575b263144251693649c5c8ad4925fdc9f839820bc2349d0a5c425e68b
MD5 3ae08efa8f4edfbef13d0a049f49931c
BLAKE2b-256 f31d465e3c28785c8f0fa0c7e8a03e9b2fc37e14c1e0522db887952c31d2cbcb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page