Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

First, I'm no NAS-pro and I'm no Unix-pro. Second, I read the other RSS-downloader question, but rsstail is a C-source and tvrss is a GUI-program. I also checked out NetVibes, but I can't sort by title.

I bought a Zyxel NSA310 and thought it would be a good idea to have an RSS reader/downloader on it. There is a broadcatcher on it, but it doesn't store the RSS-items and I don't need that additional logic where downloads are initiated based on the RSS-items.

It should download the RSS-feeds and store the contents in readable files. When I get home from a trip I want to see all RSS-items of those feeds and not just the latest 30 or something. Since I use PuTTY to connect to my NAS I can't use any GUI-program. As far as the output is concerned I don't know what to look for, simple text would be enough though. To me it makes sense to sort the RSS-items by title and by date.

To let you know what's on the NAS:

root@NAS:~# uname -a
Linux NAS #4 Fri Jul 20 18:10:42 CST 2012 armv5tel GNU/Linux
root@NAS:~# busybox
BusyBox v1.17.2 (2012-07-20 17:55:00 CST) multi-call binary.
Copyright (C) 1998-2009 Erik Andersen, Rob Landley, Denys Vlasenko
and others. Licensed under GPLv2.
See source distribution for full notice.

Usage: busybox [function] [arguments]...
   or: function [arguments]...

        BusyBox is a multi-call binary that combines many common Unix
        utilities into a single executable.  Most people will create a
        link to busybox for each function they wish to use and BusyBox
        will act like whatever it was invoked as.

Currently defined functions:
        [, [[, acpid, addgroup, adduser, adjtimex, arp, ash, awk, basename,
        beep, bzip2, cat, catv, chat, chgrp, chmod, chown, chroot, chrt, chvt,
        cksum, clear, cmp, cp, crond, crontab, cryptpw, cttyhack, cut, date,
        dd, deallocvt, delgroup, deluser, depmod, devmem, df, diff, dirname,
        dmesg, dnsdomainname, du, echo, ed, egrep, eject, env, expand, expr,
        false, fgrep, flock, free, freeramdisk, fsck, fsync, getopt, getty,
        grep, gunzip, gzip, halt, head, hexdump, hostname, hwclock, id,
        ifconfig, init, insmod, install, ionice, ip, ipaddr, ipcrm, ipcs,
        iplink, iproute, iprule, kill, killall, less, linuxrc, ln, logger,
        login, losetup, ls, lsmod, lspci, lsusb, lzop, lzopcat, man, microcom,
        mkdir, mke2fs, mkfs.ext2, mknod, mkpasswd, mkswap, mktemp, modinfo,
        modprobe, more, mountpoint, mv, netstat, nslookup, openvt, passwd,
        pidof, ping, ping6, pivot_root, poweroff, printenv, printf, ps, pwd,
        raidautorun, rdev, readahead, readlink, reboot, reset, resize, rev, rm,
        rmdir, rmmod, route, rtcwake, script, scriptreplay, sed, seq, setfont,
        setlogcons, setsid, sh, sha256sum, sha512sum, showkey, sleep, sort,
        split, start-stop-daemon, strings, su, swapoff, swapon, sync, tac,
        tail, tar, tee, telnet, telnetd, test, time, timeout, top, touch, tr,
        traceroute, traceroute6, true, tty, ttysize, ubiattach, ubidetach,
        udhcpc, udpsvd, uname, unexpand, uniq, unlzop, unxz, unzip, uptime,
        usleep, vi, volname, wall, wc, which, who, whoami, xargs, xz, xzcat,
        yes, zcat, zcip

I have no cc/gcc or make. I can't find a package installer neither.

Google found stuff for me coded in Python using feedreader, but those require distutils, which I don't have and I don't seem to be able to install that.

The only solution I can think of right now is using wget in connection with crontab. However, I'd have to do all the parsing and all in shell-script (since I don't know Python or Perl). That seems inconvenient.

What can I do/try? Any help is appreciated.

share|improve this question
as an update: I'm writing an RSS-reader with awk... Wish me luck ;) – sjngm Mar 30 '13 at 18:43
up vote 1 down vote accepted

Here's what seems to work for me. I had this running with crontab for a while and it works for me. It supports RSS and Atom I think. However, if you intend to use it, consider it a starting point. If you adapt it and the change is useful, feel free to come back and tell the community.

The result is a directory structure with the feed name being the directory name and the various entries written as files. Those files contain the unchanged contents and at the bottom a link to the source. The filenames consist of the item title and its timestamp, the date of the file is set to that timestamp. That way you can sort them by name or time easily.


function removeFirstTag(str) {
    sub("[^>]*>", "", str)
    return str
function extractCdata(str) {
    gsub(".*<!\\[CDATA\\[|]]>.*", "", str)
    return str
function cleanup(str) {
    return extractCdata(removeFirstTag(str))

    written = 0
    existed = 0
    if (feedTitle) {
        system("mkdir -p '" outDir "/" feedTitle "'")

/<\/description>|<\/content>/ {
    indescr = 0
/<\/summary>/ {
    insummary = 0
/<\/item>|<\/entry>/ {
    gsub(/\//, "-", title)
    fname = outDir "/" feedTitle "/" title " - " dateSec
    fnameEnc = fname
    gsub(/\$/, "\\$", fnameEnc)
    gsub(/"/, "\\\"", fnameEnc)
    if (!description) {
        description = summary
    if (link != "") {
        description = description "\nLink: " link
    } else if (guid != "") {
        description = description "\nLink: " guid

    write = system(binPath "/stat \"" fnameEnc "\" -c \"%u\" 1> /dev/null 2>&1")
    if (write == 1) {
        print description > fname
        system(binPath "/touch \"" fnameEnc "\" -d \"" date "\"")
        written = written + 1
    } else {
        existed = existed + 1

    if (indescr == 1) {
        description = description "\n" $0

/<title[ >]/ {
    if (!feedTitle) {
        feedTitle = cleanup($0)
        system("mkdir -p '" outDir "/" feedTitle "'")
    } else {
        title = cleanup($0)
/<item[ >]|<entry[ >]/ {
    title = ""
    description = ""
    summary = ""
    date = ""
    link = ""
    guid = ""
/<description[ >]|<content[ >]/ {
    indescr = 1
    description = cleanup($0)
/<summary[ >]/ {
    insummary = 1
    summary = cleanup($0)
/<link[ >]/ {
    link = cleanup($0)
    if (index(link, "magnet") == 1) {
        link = ""
/<guid[ >]|<id[ >]/ {
    guid = cleanup($0)
/<pubDate[ >]|<published[ >]|<dc:date[ >]/ {
    date = cleanup($0)
    binPath "/date -d '" date "' +'%s'" | getline dateSec

    print "Done : " feedTitle " (new: " written "/old: " existed ")"
    if (existed == 0 && written != 0) {
        print "=== NOTE === : All entries are new. You may want to check this feed more often"

The file rss-parser.cfg consists of lines such as feed title is better


parse() {
    echo "Start: $name ($url)"
    wget -q -U "$userAgent" -O rss-news.xml $url
    sed -e 's/
//g' -e 's/<\([a-z/]\)/\n<\1/g' rss-news.xml | \
        $binPath/awk \
            -v "feedTitle=$name" \
            -v "binPath=$binPath" \
            -v "outDir=./data" \
            -f rss-parser.awk | \

userAgent="Opera/9.80 (Windows NT 6.1; Win64; x64; U; de) Presto/2.10.289 Version/12.01"
cd `dirname $0`
if [ -z "$1" ]; then
    echo "================= `$binPath/date +"%d.%m.%Y %H:%M"`"
    start=`$binPath/date +%s`
    while read line
        if [ ! -z "$line" -a "${line:0:1}" != "#" ]; then
            if [ "$name" == "$url" ]; then
            parse "$url" "$name"
    done < "rss-parser.cfg"
    echo "================= Took `$binPath/date -d "$start seconds ago" +%s` seconds"
    parse "$1" "$2"

Without parameters it parses rss-parser.cfg. Otherwise parameters are

./ [<URL> [<feed name>]]

I hope it's useful to anyone...

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.