[tex-live] Still troubles when trying to reach some pages of tug/texlive site
Reinhard Kotucha
reinhard.kotucha at web.de
Tue Feb 13 01:31:55 CET 2018
Sorry, I sent the previous mail too early.
On 2018-02-12 at 16:32:28 +0900, Norbert Preining wrote:
> > Sometimes, I can temporarily reach pages such as
>
> There is a block installed on tug that should catch robots. If you
> are hitting to hard and quickly on the server/svn space, you will
> be blocked for some time.
Really?
while true; do wget https://www.tug.org/texlive/lists.html; done
works fine here and
wget -r https://www.tug.org/texlive
too. I don't understand why Denis can't download lists.html at all.
BTW, blocking robots reliably without bothering normal users is a very
difficult task. It's probably better not to rely on the time between
two requests but on the number of requests within a certain amount of
time.
Wget fails when I replace "www.tug.org" with its IP number. This is
good because most robots simply scan IP numbers.
Regards,
Reinhard
--
------------------------------------------------------------------
Reinhard Kotucha Phone: +49-511-3373112
Marschnerstr. 25
D-30167 Hannover mailto:reinhard.kotucha at web.de
------------------------------------------------------------------
More information about the tex-live
mailing list