Reading RSS feeds via email on the CLI

Tim Chase

2019-07-10

Overview

In this article we install rss2email (along with its dependencies), configure it to send mail via the local MDA (usually sendmail), add some RSS feeds, and then read them via mail/mailx.

This assumes that you are comfortable in a CLI environment, and that you already have Python3 (3.2 or higher) already installed on your Unix-like machine (Linux, BSD, or Mac; I don't cover doing this on Windows).

Getting started

Create a virtualenv named `rss/` to isolate the install

This creates a directory called rss/ in the current directory:

$ python3 -m venv rss

You can put this directory anywhere you want. I have a ~/virtualenvs/ directory in which I keep them all.

Activate the virtual environment

To activate the virtual environment:

$ . rss/bin/activate

This should change the prompt so it has the prefix (rss) and when we install packages with pip they end up in here, rather than messing with our system install. Additionally, you don't need superuser privileges to install.

Test the virtual environment

On most systems, the above should work just fine. However, on some systems, the directory/mount-point containing $HOME gets marked with noexec (or might lack the wxallowed permission on OpenBSD) preventing executables in the $HOME directory from running. If your system doesn't allow you to run things from $HOME you'll have to talk to your system administrator. But activating the virtual environment and typing

(rss)$ which python3

at the command-line should return the path to your $VIRTUALENVDIR/rss/bin/python3 and running

(rss)$ python3

should bring up the Python REPL rather than give an error. (use control+d to exit the REPL)

Install `rss2email` and dependencies

We only need rss2email, however, if we want to launch any URLs contained in the RSS content, urlscan lets us pipe a message from our mail program, scan it for URLs, and launch them in our $BROWSER so we'll install urlscan at this time as well.

(rss)$ pip install rss2email urlscan

Identify yourself to `rss2email`

(rss)$ r2e new ${USER}@localhost

By default, rss2email stores the configuration in $XDG_CONFIG_HOME/rss2email.cfg and the JSON-format feed database in $XDG_DATA_HOME/rss2email.json (which is the right place for them) but you can override these using the --data other/path/to/data.json and --config other/path/to/config parameters.

(rss)$ r2e --data ~/.rss2email.json \
>  --config ~/.rss2email.cfg \
>  new ${USER}@localhost

Modify the configuration

Edit your rss2email.cfg and make the following changes

If your local mail program requires a valid email address or domain-name, change the from = … to either your local account from = demo@localhost or something like from = rss2email@yourdomain.com or from = example+rss2email@gmail.com.
Change the use-8bit = False to use-8bit = True so that it sends the raw 8-bit data rather than encoding it as base64. If you use a mailer other than mail(1)/mailx(1), you might not need this. However, mail(1) doesn't decode base64, making it a pain to read.

The rss2email site lists other options that you might want to twiddle such as

trust-guid & trust-link: controlling whether items to consider items "new" based on URL or on GUID (if an item gets an update, theoretically the GUID should change but the URL should not, so you would want to set this based on whether you want to see updates)
date-header & date-header-order: which date rss2email should associate with the email
html-mail: including an HTML part (if you MUA supports rich HTML messages you might want this, but if you'll use mail(1) you'll want to turn this off)
digest: determines whether rss2email should combine all new articles for a particular feed into a single email message-digest or not

Add your feeds

Use the add command to add feeds by URL. This takes a feed-name and the URL of the RSS-feed:

(rss)$ r2e add "ExampleFeed" http://example.com/rss.xml

You can do this multiple times to add multiple feeds.

If you have an existing collection of RSS feeds you can export it as OPML (the common RSS interchange method) from your previous feed-reader and import it into rss2email using

(rss)$ r2e opmlimport yourfeeds.opml

Catching up

If you have added multiple feeds, the first time you run rss2email it will bring down a deluge of unread articles. For the list of feeds that I get, this would have resulted in about a thousand new emails. To prevent this, you can tell rss2email to fetch all of the feeds but not send emails, catching you up to the latest on all of them by using the -n option for the run command:

(rss)$ r2e run -n

Updating feeds

Once you have optionally caught up with your feeds you'll want to pull down any new articles using the run sub-command (this time without the -n option):

(rss)$ r2e run

This will fetch each RSS feed and send each new article as an email to your local system mailbox. If you prefer to send them to a remote mailbox, you'll have to dig into the mailer settings in your rss2email.cfg file to adjust the email-protocol and possibly the smtp-* or imap-* settings to change how rss2email delivers the messages.

While you can poll your RSS feeds on whatever schedule you like, I prefer to gather mine once early in the day so it doesn't interrupt me (or tempt me to check for more mail/feeds) throughout the day. Thus I use crontab -e and have added the line

15 5 * * * $VIRTUALENVDIR/rss/bin/r2e run

This polls my RSS feeds daily at 5:15am, leaving them in my inbox to check when I start my day. If you prefer to check them hourly during working hours, you might instead use

15 8-16 * * * $VIRTUALENVDIR/rss/bin/r2e …

to run at 8:15am, 9:15am, 10:15am, … 4:15pm. Alternatively, perhaps you want to check them before work, before lunch, and before leaving for the day

45 7,12,16 * * * $VIRTUALENVDIR/rss/bin/r2e …

By default, cron will mail you the output, so you may want to swallow it by appending 2>/dev/null or redirect it to a log-file with 2>$HOME/.rss2email.log in case you need it.

Reading your RSS feeds

Now that you have the articles in your inbox, you can use your favorite MUA to read them. For purposes here, I demonstrate using mail(1)/mailx(1) which works well in any terminal and even on a paper-copy output device. It should also come installed out-of-the-box on the BSDs and various flavors of Linux (I'm not sure about MacOS). But you could use mutt/neomutt, alpine, thunderbird, claws-mail, or any of a number of other mail programs that can read a local mbox file (usually stored in /var/spool/$USER or wherever $MAIL points to).

To begin to read the RSS feeds fire up mail(1) (which you can do either within your rss virtual environment or not)

(rss)$ mail