EFFECTIVE JANUARY 1, 2024 ------------------------- Because of the proliferation of web-gopher proxies that don't gate bots, and because operating the site costs us money directly in bandwidth, we're forced to crack down. We get literally tens of thousands of requests a day often from single IP addresses or networks. This isn't acceptable anymore and is a cost we cannot meaningfully sustain. We want bots spidering gopherspace to generate publicly accessible indexes that people can search over Gopher. This is a good use of community resources. Those sorts of spiders, assuming they obey restrictions and keep their access rates reasonable, are to be encouraged. We will support them. Bots that generate publicly accessible snapshots also fall in this group. On the other hand, spidering away to generate a private index basically costs us and other sites money for something that doesn't improve general gopherspace, and those spiders will be banned. Similarly, misdirected bots that index through misconfigured proxies generate useless traffic, and we may prevent those proxies from accessing Floodgap altogether until this is fixed. You're welcome to disagree, but you don't pay the bills. Nothing personal. ARE YOU HERE BECAUSE YOUR SITE IS BLOCKED? See the list at the end. Contact us at gopher@floodgap.com if you want to be unblocked. Be prepared to explain your bot's or proxy's activities, and do note that we reserve the right to keep it blocked. REQUIREMENTS FOR SPIDERS: - If your site is determined to be spidering, the IP you use for spidering must be accessible on port 70 so we can confirm you are a gopher server. Ideally your index should also be publicly searchable. - If you cannot do this, but have a reason to be white-listed, please E-mail gopher@floodgap.com with a contact person and an explanation. IPs or netblocks will be blocked WITHOUT WARNING (either at the protocol or the packet filter level) unless either or both of these conditions are met. Talk to us first before firing up your bot. BLOCKED SITES LIST This list contains date they were listed, the IP, and the hostname (if relevant). Patterns may appear for complete blocks if activity came from multiple locations. Blocked from portions of Floodgap Groundhog (due to ignoring robots.txt) (all below as of 2024-01-01) *.0x1bi.* *.amazonaws.* *.codevoid.* *.linode.* *.meulie.* *.nonic.cloud *.pongonova.* *.projectsegfau.lt *.titmouse.* *.veloxzone.com.br vulpes.* *.z3bra.* 23.119.120.152 23-119-120-152.lightspeed.sntcca.sbcglobal.net. 46.44.40.97 47.35.88.61 047-035-088-061.res.spectrum.com. 89.163.145.170 envs.net. 93.157.51.111 155.133.142.66 gw3.sd6.gpaas.net. 198.251.84.158 freeirc.org. 159.196.9.199 159-196-9-199.9fc409.mel.nbn.aussiebb.net. 159.196.36.43 164.90.232.244 198.50.210.248 tilde.team. 205.166.94.4 rie.sdf.org. Blocked from Floodgap entirely (at packet filter level) (all below as of 2024-01-01) 2.57.169.13 5.228.163.240 broadband-5-228-163-240.ip.moscow.rt.ru. 5.255.250.40 5-255-250-40.spider.yandex.com. 23.28.76.218 d28-23-218-76.dim.wideopenwest.com. 31.132.35.252 34.201.165.171 ec2-34-201-165-171.compute-1.amazonaws.com. 38.126.157.40 38.126.157.46 47.88.*.* 88.47.in-addr.arpa. 300 IN SOA rdns1.alidns.com. dnsmgr.alibaba-inc.com. 2015012518 1800 600 1814400 300 51.255.65.* 65.255.51.in-addr.arpa. 9753 IN SOA dns11.ovh.net. tech.ovh.net. 2023121803 43200 4320 2419200 86400 51.255.71.* 71.255.51.in-addr.arpa. 10800 IN SOA dns11.ovh.net. tech.ovh.net. 2023123100 43200 4320 2419200 86400 54.36.148.* 148.36.54.in-addr.arpa. 10183 IN SOA dns103.ovh.net. tech.ovh.net. 2022051899 43200 4320 2419200 86400 54.36.149.* 149.36.54.in-addr.arpa. 9830 IN SOA dns103.ovh.net. tech.ovh.net. 2023102400 43200 4320 2419200 86400 64.124.8.* 8.124.64.in-addr.arpa. 3587 IN SOA ns.zayo.com. nopss.zayo.us. 2022051001 14400 3600 604800 3600 64.77.244.139 66.60.68.66 66-68.netblk-66-60-68.yada.net. 69.157.192.150 69.193.186.100 069-193-186-100.biz.spectrum.com. 77.88.*.* 77-88-*-*.spider.yandex.com. 85.12.219.207 89.33.85.209 89.33.85.209.nonic.cloud. 93.158.* 98.180.236.43 ip98-180-236-43.cl.ri.cox.net. 104.227.222.* 222.227.104.in-addr.arpa. 9833 IN SOA NS1.RDNSPOINT.COM. admin.RDNSPOINT.COM. 1695653940 10800 3600 604800 10800 104.236.11.126 104.238.83.196 196.83.238.104.host.secureserver.net. 104.28.152.* 104.28.153.* 104.28.154.* 104.28.155.155 104.28.156.* 104.28.157.* 104.28.157.125 104.28.158.* 104.28.159.* (all) 28.104.in-addr.arpa. 3587 IN SOA cruz.ns.cloudflare.com. dns.cloudflare.com. 2288625504 10000 2400 604800 3600 107.14.227.102 107-014-227-102.inf.spectrum.com. 107.14.227.86 107-014-227-086.inf.spectrum.com. 107.14.227.90 107-014-227-090.inf.spectrum.com. 107.152.190.* 190.152.107.in-addr.arpa. 9833 IN SOA NS1.RDNSPOINT.COM. admin.RDNSPOINT.COM. 1695653955 10800 3600 604800 10800 109.174.10.11 host-109-174-10-11.bb-nsk.sib.mts.ru. 113.5.134.202 117.41.183.82 134.6.212.170 137.74.201.* 201.74.137.in-addr.arpa. 9830 IN SOA dns16.ovh.net. tech.ovh.net. 2023120900 43200 4320 2419200 86400 137.74.207.* 207.74.137.in-addr.arpa. 10187 IN SOA dns16.ovh.net. tech.ovh.net. 2023121401 43200 4320 2419200 86400 138.128.59.* 59.128.138.in-addr.arpa. 9833 IN SOA NS1.RDNSPOINT.COM. admin.RDNSPOINT.COM. 1695653987 10800 3600 604800 10800 138.197.72.* 72.197.138.in-addr.arpa. 1788 IN SOA ns1.digitalocean.com. hostmaster.72.197.138.in-addr.arpa. 1704121359 10800 3600 604800 1800 141.8.144.61 141-8-144-61.spider.yandex.com. 145.239.154.* 154.239.145.in-addr.arpa. 9830 IN SOA dns103.ovh.net. tech.ovh.net. 2024010100 43200 4320 2419200 86400 151.80.27.* 27.80.151.in-addr.arpa. 9753 IN SOA dns109.ovh.net. tech.ovh.net. 2023120400 43200 4320 2419200 86400 159.203.67.* 67.203.159.in-addr.arpa. 1787 IN SOA ns1.digitalocean.com. hostmaster.67.203.159.in-addr.arpa. 1704166481 10800 3600 604800 1800 159.203.127.* 127.203.159.in-addr.arpa. 1787 IN SOA ns1.digitalocean.com. hostmaster .127.203.159.in-addr.arpa. 1704165018 10800 3600 604800 1800 159.65.230.197 nildocabeleleiro.com.br. 162.221.202.17 zonevpshost21.link. 162.221.202.138 zonevpshost21.link. 164.132.161.* 161.132.164.in-addr.arpa. 9751 IN SOA dns10.ovh.net. tech.ovh.net. 2023122900 43200 4320 2419200 86400 164.132.162.* 162.132.164.in-addr.arpa. 9752 IN SOA dns10.ovh.net. tech.ovh.net. 2024010200 43200 4320 2419200 86400 164.90.232.244 167.114.166.234 170.250.251.130 170.250.251.130.hwccustomers.com. 172.83.40.18 174.138.63.* 63.138.174.in-addr.arpa. 830 IN SOA ns1.digitalocean.com. hostmaster.63.138.174.in-addr.arpa. 1703936472 10800 3600 604800 1800 174.138.80.226 185.225.19.18 no-rdns.mivocloud.com. 192.99.20.10 ns559196.ip-192-99-20.net. 195.3.254.48 nat-net.rfei.ru. 198.46.141.98 host.colocrossing.com. 201.29.63.186 201-29-63-186.user3p.veloxzone.com.br. 209.127.127.73 mx72.fundersquad.com. 217.182.*.*