2 Nov
2009
2 Nov
'09
4:27 p.m.
It could contain any of http://www.robotstxt.org/robotstxt.html ... I don't think our libraries currently use one but I've not checked specifically for this.
When we were hosting our own Koha installation we had to start excluding search engine bots (Googlebot in particular) because our server was getting hit too hard and it was slowing everything down. I think LibLime blocks everything by default for its customers now. I'd certainly prefer to be able to let Google in. I'd like the contents of the OPAC to be discoverable in search engines. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org