[Koha] robots.txt

[Sébastien Hinderer]

MJ Ray (2009/11/02 15:14 +0000):
> It could contain any of http://www.robotstxt.org/robotstxt.html

I didn't know this link, thanks.

> It should contain whatever will make robots behave as you want.

That's what I don't know -- how I want them to behave.
My guess is that everything shold be disallowed because no page has a
meaning without arguments... What I would like to know is whether this
guess is correct or not.

> I don't think our libraries currently use one but I've not checked
> specifically for this.

I apache error logs and noticed some clients look for this file so I
thought I could create one so that the error log does not contain this
not too interesting entry.

Cheers,
Sébastien.