TITLE: Making a Gopherhole
DATE: 2019-11-20
AUTHOR: John L. Godlee
====================================================================
I have heard people on youtube, mostly [DistroTube] and [Hex
DSL], talking about [Gopher]. I also saw it being discussed on
[Hacker News] once.
[DistroTube]: https://www.youtube.com/watch?v=lUBhOgK5zQI
[Hex DSL]: https://www.youtube.com/watch?v=ORgk-AwD7SQ
[Gopher]: https://en.wikipedia.org/wiki/Gopher_(protocol)
[Hacker News]: https://news.ycombinator.com/item?id=13855634
The youtubers and digitial minimalist / linux try-hards have
recently begun lauding Gopher as a sane alternative to browsing the
internet with HTTP and the World Wide Web, which they say are
bloated and beyond redemption since advertisers and data harvesters
have taken over.
Gopher is an internet protocol that was created in 1991. In my mind
it sits alongside FTP more than it does HTTP, in that it is merely a
way of organising file delivery in a heirarchical fashion. It
doesn’t have the flexibility of HTTP, providing only plain text
formatted by certain conventions and a smattering of links to
connect files and pages. Some think this inflexibility is a good
thing, Gopher sites are: simple, transparent, text based (good for
the visually impaired) and consistent. In Gopher it’s hard to
implement most of the more destructive mechanisms present on the
World Wide Web like user tracking, data harvesting and advertising.
Gopher sites are also phenomenally light on resources, providing
just plain text. It’s not that HTTP can’t be fast as well, but there
is a tendency to use all the features that the flexibility of HTTP
can afford. That being said, I think a lot of people are confusing
the World Wide Web with the internet. Gopher is an internet
protocol, but it satisfies a different set of needs to HTTP and they
shouldn’t be compared so readily.
I set up a Gopher site (aka a Gopherhole) on the [Super Dimension
Fortress], which provides its registered and validated users with
gopherspace running on the [gophernicus server software] for free.
It also provides lots of other services run through a shell account
that you can SSH into, such as POP3/IMAP mail, and IRC. They are a
good community of hobbyists which provides a valuable if
computationally limiting service. There are lots of other places to
host a Gopherhole and it’s also possible to self-host, and there are
a few Gopher server softwares out there, the most popular right now
being [pygopherd], I think.
[Super Dimension Fortress]: https://sdf.org/
[gophernicus server software]: http://www.gophernicus.org/
[pygopherd]: https://github.com/jgoerzen/pygopherd
First I created a shell account, logged in, and validated my
membership with a $3 USD donation to the SDF by PayPal. My
validation came through in about 12 hours, but I imagine this varies
a bit. I SSH’d into my shell account with:
ssh username@tty.sdf.org
After entering my password I could then create a gopher with
mkgopher, which creates a directory ~/gopher. Inside this directory
is where I store all my gopher content. mkgopher also allows various
site level options to be configured, such as the site title and the
site description. It’s also important to set file and directory
permissions so that other users can read the material in the
gopherhole. This can be done automatically within mkgopher with the
chmod command, or manually:
find ~/gopher/ -type f -print0 | xargs -0 chmod 644
find ~/gopher/ -type d -print0 | xargs -0 chmod 755
Gophermaps
At the root of ~/gopher there should be a file called gophermap.
This file defines the homepage of your gopherhole, with plain text
and links to reach other content.
An example gophermap looks like this:
iWelcome to gopherspace /
0This is a text file in a link file.txt
9This is a pdf file in a link file.pdf
1This is a link to a directory subdir
iSome more text. /
IAn image img.gif
0A file on another server /gopher/relevance.txt gopher.floodgap.com 70
hA HTTP link to another server URL:http://sdf.lonestar.org/
The numbers, and some letters are called itemtypes. They denote what
type of information the line holds. There are a bunch of itemtypes,
but I think the commonly used itemtypes nowadays are:
Itemtype Content
---------- -----------------
0 Text file
1 Directory
7 Search query
9 Binary file
g GIF image
h HTML URL
i inline text
s Sound file
I Image (not-GIF)
One extra itemtype that I haven’t found documented ANYWHERE is =.
This can be used to start a shell command on the server. For
example, to show the current date and time:
=echo "`date`"
This could come in really useful if hosting a server on your own
machine, where you could point the shell to a script to do basically
anything you want. This makes gopher more extensible than I
originally thought. Maybe if Gopher had remained popular for a long
time, we might have ended up with gopher developers abusing = to
gather data on users and subversively advertise to us.
The full syntax of the whole line is, for a text file for example:
0Description of file<TAB>/path/to/content.md<TAB>domain.org<TAB>port_number
Note that <TAB> must be an ACTUAL tab character, not expanded to
multiple spaces as some text editors ‘helpfully’ adjust it. The 0
defines the line as pointing to a text file. Description of file
will appear in the page as a selectable link. /path/to/content
defines the path to the text file to be opened by the link. File
paths can be defined relative to the current gophermap, so if
file.txt is located in the same directory as gophermap the path can
just be file.txt. If the file is located on the same server as
gophermap, domain.org and port_number can be omitted. If the file is
on a different server, the server domain (domain.org) should be
added and the port number, which is usually 70. So if the address of
the remote file linked is
gopher://gopher.floodgap.com/0/gopher/relevance.txt, the domain name
is gopher.floodgap.com.
As a side note, I’ve found that in gopher browsers, if you want to
visit the rendered version of a gophermap, you can enter the address
like this, with a 1 after the domain:
gopher://gopher.floodgap.com/1/gopher
If you want to view the unrendered version of the gophermap, you can
replace the 1 with a 0.
For a piece of text, the gophermap syntax is similar, but not
exactly the same:
iSome text that will appear on its own<TAB>/
A single slash should be included after a single <TAB> following the
text to be displayed. An i should be included before the text.
I’ve found that the 9 itemtype can cover many non-standard
filetypes, as it basically just prompts the web browser to download
the file.
Directories can be nested below the top level gopherspace directory.
Each of these directories can have their own gophermap, but they
don’t have to. In the top level gophermap the directory can be
called as:
1This is a link to a directory<TAB>subdir
If there is a gophermap in subdir/ it will be opened. This
subdir/gophermap can contain relative links to files the same as the
top level gophermap.
There isn’t a lot more to creating a simple functional gophermap,
the rest is just text formatting to design a well formatted page. I
like to split the top level gophermap into sections with headers
wrapped in == symbols, and to have an ASCII art header at the very
top of the top level gophermap. It’s also customary to limit the
width of a gopher page to 69 characters, but I haven’t found any
technical reason why this would be the case:
i _ _ _ _____ _ _ /
i | | | | | | / ____| | | | /
i | | ___ | |__ _ __ | | | | __ ___ __| | | ___ ___ /
i _ | |/ _ \| '_ \| '_ \ | | | | |_ |/ _ \ / _` | |/ _ \/ _ \ /
i| |__| | (_) | | | | | | || |___ _ | |__| | (_) | (_| | | __/ __/ /
i \____/ \___/|_| |_|_| |_||_____(_) \_____|\___/ \__,_|_|\___|\___| /
iJohn L. Godlee /
0Contact details contact.txt
0CV cv.txt
i==== Phlog posts ================================================= /
0Post 1 post_1.txt
0Post 2 post_2.txt
i==== Recent recipes ============================================== /
0Mac and cheese mac_cheese.txt
0Pizza dough pizza_dough.txt
Browsers
To browse Gopher pages the [Lynx] browser in the terminal is the
most common way I think. Otherwise there are plugins for some web
browsers, like the [Overbite plugin for Firefox]. There are also a
few different online Gopher to HTML proxy services where you type in
the gopher address and the output is rendered in HTML,
e.g. [GopherProxy], [Floodgap’s proxy], or [Gopher Commons].
[Lynx]: https://lynx.invisible-island.net/
[Overbite plugin for Firefox]: https://gopher.floodgap.com/overbite/
[GopherProxy]: https://gopherproxy.meulie.net/
[Floodgap’s proxy]: https://gopher.floodgap.com/gopher/gw
[Gopher Commons]: https://gopher.commons.host/
I like to use the [w3m browser] which I couldn’t get to load
gopher pages by default, but I did find an [awk script on
Bitbucket] which converts Gopher to HTML and then serves it
through w3m. So far it is working pretty well, but I still found
myself loading up Lynx when I was building my page initially to make
sure everything was formatted correctly.
[w3m browser]: http://w3m.sourceforge.net/
[awk script on Bitbucket]: https://bitbucket.org/iamleot/gopher2html/src
Converting a Github-pages blog to Jekyll
I created a shell script which generates a gopherhole with a
directory tree like this:
.
├── contact.txt
├── cv.txt
├── gophermap
├── posts
│ ├── 2017-07-20-ranger-rifle-conf-mac.txt
│ ├── 2017-08-14-bash-prompt.txt
│ ├── 2019-11-10-beamer.txt
│ ├── 2019-11-15-gginext.txt
│ └── gophermap
└── recipes
├── Apricot_orange_blossom_baklava.txt
├── Baked_pumpkin_with_apple.txt
└── gophermap
It’s basically just a load of shell scripts. First I use pandoc to
convert the recipes and blog posts to plain text with something
like:
pandoc --from markdown --to plain --reference-links --reference-location=block -o posts/post_1.txt post_1.md
Then I put the title of each blog post as a link into the gophermap
using sed in a for loop, with something like this:
all=(posts/*.txt)
# Reverse order of posts array
for (( i=${#all[@]}-1; i>=0; i-- )); do
rev_all[${#rev_all[@]}]=${all[i]}
done
# Get 10 most recent posts
recent="${rev_all[@]:0:10}"
# Add recent post links to gophermap
for i in $recent; do
line=$(head -n 1 $i)
printf "0$line\t$i\n" >> gophermap
done
Those were just simplified examples, it’s easy to add header
material from another file or format the link text differently.
Update 2019_12_25
Note that my gopher hole is now hosted on [tilde.club/], at
gopher://tilde.club/1/~johngodlee because scp was being problematic
on SDF. I’ve since shut down my SDF gopherhole.
[tilde.club/]: http://tilde.club/