proxy70

Re: Solderpunk and a new protocol[0]
2019-06-19

Having been less engaged in pubnix and gopher for around two weeks following
the birth of my daughter, I checked in today with the intention of reading
solderpunk's 3 phlog series on a protocol between gopher and the web. I have
talked with sodlerpunk and others about this in the past and was then and
remain excited about the possibility. I am excited for a few reasong:

1. It is always exciting to be around for the birth of something and not
know what will become of it, but have high hopes
2. I LOVE the gopher community, but gopher is not without its shortcomings
3. It fits in with my view that the internet on the whole can still be
anything we want it to be, but we must steer it there and not just be
fed things from large companies. Change CAN come from users, and I do
not believe you have to be a programer to take part. As such, I highly
recommend reading solderpunk's posts and writing your own response!

Now to the response in general:

I. Headers

- Request Headers -

I cannot think of many use cases for request headers and generally agree
with solderpunk's sentiments on the issue. I can see cases where making
a language request could be useful. For example, I know that slugmax
phlogs in english AND french. Since I do not speak French I have not
viewed their French phlogs and do not know if it is duplicate content
in another language or not. If it is duplicate content then a request
header from a client could be useful and make it so slugmax did not need
to keep separate phlogs in their feeds. Alternatively, servers could
implement server-side translation of text/map documents (if this were
to happen I think the server should also isnert a line at the top stating
that translation has occured and give the source language so that a user
may request it if need be). I think that kind of use case would really
be neat. Of course, it would require those running servers to buy in to
such an idea and is really not a protocol level question for the most
part.

- Response Headers -

I very very much agree with the notion that adding open ended headers is
a broken idea that just sets up abuse. I think that the proposal to have
headers set up as:

Is a worthy setup. The status codes will make proper issue handling actually
possible... something I have struggled to be able to communicate and figure
out in Bombadillo over gopher. The MIMEtype approach makes perfect sense to
me for the most part. There is some consideration given in regard to having
a text/map type or just having everything be one type that serves both text
and links. I think I fall into the separation of interests camp.

The idea to have one text format and just scan for links sounds appealing
at first. With a lot of content on gopher, where things are relatively short,
it makes a good amount of sense. However, what happens when a very large
text document is served? I found on gopher at one point a copy of "The
Fellowship of the Ring" as a plain text file. Under a system in which this
could contain links or could not a client would have to parse the whole
document before serving it to the user. While it should not take a crazy
long time to do, it is extra processing that is needless. Having a text/plain
and a text/map differentiates the files sufficiently and would let a client
know not to try parsing the text/plain file. I do not know a lot about how
a server/client determines MIME types. Is it infered by file types
(ie. *.html)? If so, then it could make sense to, by convention, user *.map
or some such for files that we would want parsed as maps. But again, this
is not an area of expertise for me and I will need to do some research into
how a server would know the mimetype of a file. Either way, my concern with
the separation of plain text from map files (that may contain links) is
whether or not to bother parsing the file or just serving it up.

Encodings. This is making me flash back to working with Python 2.x. Dealing
with encodings can be a pain. Sure enough, I checked the russian gopherhole[1]
that solderpunk linked... bombadillo does not display most of the characters.
As such, thank you for pointing that out to me! I think an encoding declaration
defaulting to utf-8 is a great great solution.

II. Links, links, links

I absolutely agree that the itemtype i business is a dirty workaround that
has served its purpose but should be improved upon. I generally like the
idea of a file that has plain text (no need for item i) and has links
on their own lines. Solderpunk suggests links go on their own lines and
are set up as:

I think, for me, we could likely lose the itemtype. The client will get
a mimetype from the server (if the above header conversation plays out as
suggested), and clients will be able to use that to prompt a download or
the display text content or the like. The itemtype always weirded me out
anyway. A user can put whatever they want... so it is an on your honor
kind of thing. This trust has generally played out fine on gopher, but
why bother?

I like the idea of something like:

~! My Link@someurl.co.uk/main.map

Which is the equivalent of:

My thought on the above is that using something at the beginning of the
line to indicate a link simplifies searching for links (you dont need to
read the whole line to know that you are at a link line) when parsing a
document. I am not tied to the ~! combination. It is just a suggestion.
The @ symbol could be any other indicator ( == comes to mind), and was
used in place of tab for the specific reason that lots of people set their
editors to replace tabs with spaces. It is annoying to toggle that on and
off so using a standard textual character (or series of them) makes more
sense to me.

III. Misc

- Directory default file -

It occurs to me that if all files, in solderpunk's description, are both
maps (can contain links) and content... what does a request to a directory
get you (ie. a request that ends in a '/')? For gopher, it usually (though
not always, it depends on the server) will grab a file called 'gophermap'
within that directory, if one is present. Some servers, absent a gophermap,
will provide a gophermap of the files in the directory. So what is the
proposed behavior with this new protocol? Is it to be a part of the protocol
at all or something left to servers to figure out and implement? The web has
largely standardized on index.[htm|html|php|etc]. Would a base level file
name be beneficial, such as index or gophermap? Or would some other behavior
make more sense? I'm not sure the answers to these questions.

- Media Types -

After the above confusion over how MIME/media types work, I did a little
research before finishing this. The wiki article[2] on the subject was quite
helpful. It seems like the media type structures already in place would be
very useful and provide great flexibility for clients. For example 'image/png'.
A client can easily be set for anything in the image tree being a download
item rather than display. This is particularly useful for terminal based
clients. However, a graphical client could decide to display the image to the
user (as its own document, not inline with text). I think this flexibility
will prove to be quite nice. As a result, I definitely have come round to
gneral support of the suggested response header from solderpunk. The one
remaining question I have then is would a new media type be created for the
text format under discussion (plain text plus links)?

- Search/Interaction -

The subject of search is an interesting one. Gopher's type 7 is an odd
but solid enough way of handling things. I feel like there could be a
solution via a new mediatype. Have the server send back a mime that lets the
client know that an information/query request is being made. I would think
that the server response could include any arbitrary text, to be used as
a prompt, and the client would know to let the user enter text after
displaying the response body. The prompt input would then be sent back to
the same server address (possibly by the same method gopher uses...
<url><TAB><user query>) and get a response back with the mediatype
appropriate to that reponse (rather than the search mediatype). This would
enable things like weather reports[3] or wikipedia[4] to still function well.

IV. Closing thoughts

I am pretty excited about this collective project that has been under
discussion for some time. I think it would find users well and I think that
its simplicity overall would allow it to get worked into existing gopher
clients. Wouldn't it be wonderful to just not have to worry what protocol
you were using? Produce your content on the one that makes sense to you and
let the world view it as they see fit. :)

It was not thoroughly discussed in sodlerpunk's writeup, but I do know that
they are big supporters of adding tls as a core part of the protocol. I
think that is a good call.

[0] gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/phlog/protocol-pondering-intensifies.txt
[1] gopher://gopher.pclovers.ru:70/1/rus.koi8
[2] gopher://gopherpedia.com:70/0/Media type
[3] gopher://gopher.floodgap.com:70/7/groundhog/us/zipcode
[4] gopher://gopherpedia.com:70/7/lookup
* I was having trouble using gopherpedia's search while getting the
above media type link. Upon looking into it further, it seems someone
has donked up their root gophermap the link points to 0.0.0.0:70.
Hopefully someone over there realizes it and gets it fixed up. You can
still use the link I provide, it just cannot be accessed from their
main page.