<- Back

# Improved Youtube RSS/Atom feed

Last modification on 2023-11-20

... improved at least for my preferences ;)

It scrapes the channel data from Youtube and combines it with the parsed Atom
feed from the channel on Youtube.

The Atom parser is based on sfeed, with some of the code removed because it is
not needed by this program.  It scrapes the metadata of the videos from the
channel its HTML page and uses my custom JSON parser to convert the
Javascript/JSON structure.

This parser is also used by the »json2tsv« tool. It has few dependencies.

## Features

* Add the video duration to the title to quickly see how long the video is.
* Filter away Youtube shorts and upcoming videos / announcements: only videos are shown.

* Supports more output formats: Atom, »JSON Feed« or

  »sfeed« Tab-Separated-Value format.

* Easy to build and deploy: can be run as a CGI program as a static-linked
  binary in a chroot.
* Secure: additionally to running in a chroot it can use pledge(2) and unveil(2)
  on OpenBSD to restrict system calls and access to the filesystem.


## How to use

There is an option to run directly from the command-line or in CGI-mode.  When
the environment variable $REQUEST_URI is set then it is automatically run in
CGI mode.


Command-line usage:

        youtube_feed channelid atom
        youtube_feed channelid gph
        youtube_feed channelid html
        youtube_feed channelid json
        youtube_feed channelid tsv
        youtube_feed channelid txt


CGI program usage:

The last basename part of the URL should be the channelid + the output format
extension. It defaults to TSV if there is no extension.
The CGI program can be used with a HTTPd or a Gopher daemon such as geomyidae.

For example:

        Atom XML:     https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.xml
        HTML:         https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.html
        JSON:         https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.json
        TSV:          https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.tsv
        twtxt:        https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.txt
        TSV, default: https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw
        
        Gopher dir:   gopher://codemadness.org/1/feed.cgi/UCrbvoMC0zUvPL8vjswhLOSw.gph
        Gopher TSV:   gopher://codemadness.org/0/feed.cgi/UCrbvoMC0zUvPL8vjswhLOSw


An OpenBSD httpd.conf using slowcgi as an example:

        server "codemadness.org" {
                location "/yt-chan/*" {
                        request strip 1
                        root "/cgi-bin/yt-chan"
                        fastcgi socket "/run/slowcgi.sock"
                }
        }

## Using it with »sfeed«

sfeedrc example of an existing Youtube RSS/Atom feed:

        # list of feeds to fetch:
        feeds() {
                # feed   [basesiteurl] [encoding]
                # normal Youtube Atom feed.
                feed "yt IM" "https://www.youtube.com/feeds/videos.xml?channel_id=UCrbvoMC0zUvPL8vjswhLOSw"
        }


Use the new Atom feed directly using the CGI-mode and Atom output format:

        # list of feeds to fetch:
        feeds() {
                # feed   [basesiteurl] [encoding]
                # new Youtube Atom feed.
                feed "idiotbox IM" "https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.xml"
        }

... or convert directly using a custom connector program on the local system via the command-line:

        # fetch(name, url, feedfile)
        fetch() {
                case "$1" in
                "connector example")
                        youtube_feed "$2";;
                *)
                        curl -L --max-redirs 0 -H "User-Agent:" -f -s -m 15 \
                                "$2" 2>/dev/null;;
                esac
        }
        
        # parse and convert input, by default XML to the sfeed(5) TSV format.
        # parse(name, feedurl, basesiteurl)
        parse() {
                case "$1" in
                "connector example")
                        cat;;
                *)
                        sfeed "$3";;
                esac
        }
        
        # list of feeds to fetch:
        feeds() {
                # feed   [basesiteurl] [encoding]
                feed "connector example" "UCrbvoMC0zUvPL8vjswhLOSw"
        }


## Screenshot using sfeed_curses

## Clone

        git clone git://git.codemadness.org/frontends


## Browse

You can browse the source-code at:

* https://git.codemadness.org/frontends/file/youtube/feed.c.html

* gopher://codemadness.org/1/git/frontends/file/youtube/feed.c.gph

The program is: youtube/feed


## Dependencies

* C compiler.
* LibreSSL + libtls.


## Build and install

        $ make
        # make install


## That's all

I hope by sharing this it is useful to someone other than me as well.