- Koha - lists.katipo.co.nz

Re: [Koha] Koha Digest, Vol 219, Issue 7 - how to avoid high cpu uses due to web crawlers (vinod mishra)
by Amar Londhe 15 Jan '24

15 Jan '24

Hello, I've observed a concerning issue with our Koha server, where multiple bots are causing downtime and significantly increasing CPU usage. Some of the problematic bots include: * PetalBot;+https://webmaster.petalsearch.com/site/petalbot * MJ12bot/v1.4.8; http://mj12bot.com/ * SemrushBot/7~bl; +http://www.semrush.com/bot.html Despite attempting to address this by adding a robots.txt file, it hasn't proven effective in preventing these bots from causing disruptions. Additionally, the dynamic nature of IP addresses makes it challenging to block them individually. Furthermore, I've noticed that the Apache2 server is generating internal requests, and I'm uncertain about the cause and purpose of these requests. ` ::1 - - [15/Jan/2024:12:40:41 +0530] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f (internal dummy connection)" ` I need your expertise to fix the bot issues impacting server performance, high CPU usage, and prevent unauthorized internal requests. Thanks and Regards, Amar Londhe Full-Stack Developer On 11/01/24 4:30 am, koha-request(a)lists.katipo.co.nz wrote: > Send Koha mailing list submissions to > koha(a)lists.katipo.co.nz > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.katipo.co.nz/mailman/listinfo/koha > or, via email, send a message with subject or body 'help' to > koha-request(a)lists.katipo.co.nz > > You can reach the person managing the list at > koha-owner(a)lists.katipo.co.nz > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Koha digest..." > > > Today's Topics: > > 1. how to avoid high cpu uses due to web crawlers (vinod mishra) > 2. Re: how to avoid high cpu uses due to web crawlers > (Nirmit Krishnatray) > 3. Re: how to avoid high cpu uses due to web crawlers (vinod mishra) > 4. Re: how to avoid high cpu uses due to web crawlers > (Wagner, Alexander) > 5. koha-US Board Meeting Minutes for January 10, 2024 > (Kristi Krueger) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 10 Jan 2024 12:51:38 +0530 > From: vinod mishra<mishravk79(a)gmail.com> > To: Koha<Koha(a)lists.katipo.co.nz> > Subject: [Koha] how to avoid high cpu uses due to web crawlers > Message-ID: > <CAGLUwiRDAsH66xeQoznjHXxEiGiiGvdTv8P7w5uihM3H93mU2g(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hello > > I found that an IP 47.76.35.19 is hitting my opac continuously, due to > which CPU use is very high, and it makes the entire Koha opac and staff > client very slow. > > I tried following the links but could not resolve the issue. > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=4042#c3 > https://wiki.koha-community.org/wiki/Koha_Tuning_Guide > > I am also not able to locate the file .htaccess in mu ubuntu 18.04 with > koha 20.04 > Can anyone how to resolve this? > > With Regards, > > Vinod Kumar Mishra, > (Ph.D, MLISC, MA, B.Sc, DCA) > Assistant Librarian, > Biju Patnaik Central Library (BPCL), > NIT Rourkela, > Sundergadh-769008, > Odisha, > India. > Mob:91+9439420860 > URL:https://vinod.itshelp.co.in/ <http://vinod.itshelp.co.in/> > ORCID ID:https://orcid.org/0000-0003-4666-7874 > <http://orcid.org/0000-0003-4666-7874> > Scopus ID: 57223138343 > > *"Spiritual relationship is far more precious than physical. Physical > relationship divorced from spiritual is body without soul" -- Mahatma > Gandhi* > > > ------------------------------ > > Message: 2 > Date: Wed, 10 Jan 2024 07:46:17 +0000 > From: Nirmit Krishnatray<nirmit(a)edutech.com> > To: vinod mishra<mishravk79(a)gmail.com>, Koha > <Koha(a)lists.katipo.co.nz> > Subject: Re: [Koha] how to avoid high cpu uses due to web crawlers > Message-ID:<27ebb0c12a164436a0b59c8be7e46401(a)edutech.com> > Content-Type: text/plain; charset="utf-8" > > Hi sir, > > Try to block the ip that is hitting on your server. > > Best Regards > Nirmit Krishnatray | Associate Manager - Professional Services > DBS Business Center, World Trade Tower, Barakhamba Lane,Connaught Place, > New Delhi – 110001 > M: +91 9003078515 | E:nirmit@edutech.com > Edutech India | LinkedIn | Twitter | Facebook | Youtube > > > -----Original Message----- > From: Koha [mailto:koha-bounces@lists.katipo.co.nz] On Behalf Of vinod mishra > Sent: 10 January 2024 12:52 > To: Koha<Koha(a)lists.katipo.co.nz> > Subject: [Koha] how to avoid high cpu uses due to web crawlers > > Hello > > I found that an IP 47.76.35.19 is hitting my opac continuously, due to which CPU use is very high, and it makes the entire Koha opac and staff client very slow. > > I tried following the links but could not resolve the issue. > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=4042#c3 > https://wiki.koha-community.org/wiki/Koha_Tuning_Guide > > I am also not able to locate the file .htaccess in mu ubuntu 18.04 with koha 20.04 Can anyone how to resolve this? > > With Regards, > > Vinod Kumar Mishra, > (Ph.D, MLISC, MA, B.Sc, DCA) > Assistant Librarian, > Biju Patnaik Central Library (BPCL), > NIT Rourkela, > Sundergadh-769008, > Odisha, > India. > Mob:91+9439420860 > URL:https://vinod.itshelp.co.in/ <http://vinod.itshelp.co.in/> ORCID ID:https://orcid.org/0000-0003-4666-7874 > <http://orcid.org/0000-0003-4666-7874> > Scopus ID: 57223138343 > > *"Spiritual relationship is far more precious than physical. Physical relationship divorced from spiritual is body without soul" -- Mahatma > Gandhi* > _______________________________________________ > > Koha mailing listhttp://koha-community.org Koha(a)lists.katipo.co.nz > Unsubscribe:https://lists.katipo.co.nz/mailman/listinfo/koha > > ------------------------------ > > Message: 3 > Date: Wed, 10 Jan 2024 13:23:31 +0530 > From: vinod mishra<mishravk79(a)gmail.com> > To: Nirmit Krishnatray<nirmit(a)edutech.com> > Cc: Koha<Koha(a)lists.katipo.co.nz> > Subject: Re: [Koha] how to avoid high cpu uses due to web crawlers > Message-ID: > <CAGLUwiTpOGzFrz84VAX5BZvSa7ejZ+riVFmtDEVuy1c-QTGU0A(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Thanks that is the ultimate solution, looking for any other effective > solution too if faced in future. Creating robots.txt file seems easy but > finding which crawler is hitting is difficult with IP > > On Wed, 10 Jan, 2024, 13:16 Nirmit Krishnatray,<nirmit(a)edutech.com> wrote: > >> Hi sir, >> >> Try to block the ip that is hitting on your server. >> >> Best Regards >> Nirmit Krishnatray | Associate Manager - Professional Services >> DBS Business Center, World Trade Tower, Barakhamba Lane,Connaught Place, >> New Delhi – 110001 >> M: +91 9003078515 | E:nirmit@edutech.com >> Edutech India | LinkedIn | Twitter | Facebook | Youtube >> >> >> -----Original Message----- >> From: Koha [mailto:koha-bounces@lists.katipo.co.nz] On Behalf Of vinod >> mishra >> Sent: 10 January 2024 12:52 >> To: Koha<Koha(a)lists.katipo.co.nz> >> Subject: [Koha] how to avoid high cpu uses due to web crawlers >> >> Hello >> >> I found that an IP 47.76.35.19 is hitting my opac continuously, due to >> which CPU use is very high, and it makes the entire Koha opac and staff >> client very slow. >> >> I tried following the links but could not resolve the issue. >> >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=4042#c3 >> https://wiki.koha-community.org/wiki/Koha_Tuning_Guide >> >> I am also not able to locate the file .htaccess in mu ubuntu 18.04 with >> koha 20.04 Can anyone how to resolve this? >> >> With Regards, >> >> Vinod Kumar Mishra, >> (Ph.D, MLISC, MA, B.Sc, DCA) >> Assistant Librarian, >> Biju Patnaik Central Library (BPCL), >> NIT Rourkela, >> Sundergadh-769008, >> Odisha, >> India. >> Mob:91+9439420860 >> URL:https://vinod.itshelp.co.in/ <http://vinod.itshelp.co.in/> ORCID ID: >> https://orcid.org/0000-0003-4666-7874 >> <http://orcid.org/0000-0003-4666-7874> >> Scopus ID: 57223138343 >> >> *"Spiritual relationship is far more precious than physical. Physical >> relationship divorced from spiritual is body without soul" -- Mahatma >> Gandhi* >> _______________________________________________ >> >> Koha mailing listhttp://koha-community.org Koha(a)lists.katipo.co.nz >> Unsubscribe:https://lists.katipo.co.nz/mailman/listinfo/koha >> > > ------------------------------ > > Message: 4 > Date: Wed, 10 Jan 2024 10:19:55 +0100 (CET) > From: "Wagner, Alexander"<alexander.wagner(a)desy.de> > To: vinod mishra<mishravk79(a)gmail.com> > Cc: Koha<Koha(a)lists.katipo.co.nz> > Subject: Re: [Koha] how to avoid high cpu uses due to web crawlers > Message-ID:<2002597982.8686470.1704878395276.JavaMail.zimbra(a)desy.de> > Content-Type: text/plain; charset=utf-8 > > Hi! > >> I found that an IP 47.76.35.19 is hitting my opac continuously, due to >> which CPU use is very high, and it makes the entire Koha opac and staff >> client very slow. > This does not look like a legit crawler. So most likely you can't tackle this guy with a robots.txt as most likely it will not respect it anyway. > >> I am also not able to locate the file .htaccess in mu ubuntu 18.04 with >> koha 20.04 >> Can anyone how to resolve this? > `.htaccess` files do not exist by default, you'd have to create it in the appropriate place with proper permissions and ownerships using your favourite text-editor. They are basically folder based firewall rules read by your webserver. IOW you could either use those or have a rule in your apache configs. > > I am no expert in either but on one of our current (non-koha)-systems we use something like > > ``` > > # Turn badips away > RewriteMap hosts-deny "txt:/opt/invenio/var/tmp/hosts-deny.txt" > RewriteCond "${hosts-deny:%{REMOTE_ADDR}|NOT-FOUND}" "!=NOT-FOUND" [OR] > RewriteCond "${hosts-deny:%{HTTP:X-Forwarded-For}|NOT-FOUND}" "!=NOT-FOUND" > RewriteRule .* - [R=429,L] > > ``` > > in the apache configs. This refers to a txt-file in this case in some funny path `/opt/invenio/var/tmp/` called `hosts-deny.txt` that lists the ip-addresses that should be dropped. You could in principle create such a file in some place your apache can see it. This makes it a bit easier to handle unwanted "crawlers" as you just add the offending ips there. > > HTH. >

1 0

Department Name in Item Information
by Rupinder Singh 13 Jan '24

13 Jan '24

Hi, We have been using Koha at our university for the last year. We get allocated a consolidated budget at the beginning of the financial year. And we'd like to spend that budget as per the purchase requests sent by different departments of the university. However we haven't used the acquisition module yet and are not planning to do so. But of all the books we catalog in koha, I also want to save information about the department that requested that book. So that it can be exported via catalog report, so I can see how much of the budget we have spent on each department. And each book item shall have this information for the requesting department in the Items section. Please suggest what the best approach to achieve this could be. Is there any existing field to specify that, or is there any existing field I can make use of to put this information? I have also found certain subfields in 952 item information that are not in use. Can I map a new authorized value to this newly created field? Please, guys, Guide me. Rupinder Singh

2 1

koha-US Board Meeting Minutes for January 10, 2024
by Kristi Krueger 10 Jan '24

10 Jan '24

Hello, Here's a link to the minutes from the koha-US board meeting on Wednesday, January 10, 2024-- https://docs.google.com/document/d/1k-SG3wlk0bA7BD708ZW_Fjmin7CoX1y7tBD3fF3… koha-US meetings are open to all! Board meetings are held on the *SECOND* Wednesday of each month at 8:30 PT / 9:30 MT / 10:30 CT / 11:30 ET. The next koha-US General Meeting will be on Wednesday, February 7 @ 8:30 PT / 9:30 MT / 10:30 CT / 11:30 ET (time converter <https://www.timeanddate.com/worldclock/fixedtime.html?msg=koha-US+February+…> ). If you have additional topics for the agenda, please send them to secretary(a)koha-us.org. Kristi Krueger koha-US Secretary secretary(a)koha-us.org

1 0

how to avoid high cpu uses due to web crawlers
by vinod mishra 10 Jan '24

10 Jan '24

Hello I found that an IP 47.76.35.19 is hitting my opac continuously, due to which CPU use is very high, and it makes the entire Koha opac and staff client very slow. I tried following the links but could not resolve the issue. https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=4042#c3 https://wiki.koha-community.org/wiki/Koha_Tuning_Guide I am also not able to locate the file .htaccess in mu ubuntu 18.04 with koha 20.04 Can anyone how to resolve this? With Regards, Vinod Kumar Mishra, (Ph.D, MLISC, MA, B.Sc, DCA) Assistant Librarian, Biju Patnaik Central Library (BPCL), NIT Rourkela, Sundergadh-769008, Odisha, India. Mob:91+9439420860 URL: https://vinod.itshelp.co.in/ <http://vinod.itshelp.co.in/> ORCID ID: https://orcid.org/0000-0003-4666-7874 <http://orcid.org/0000-0003-4666-7874> Scopus ID: 57223138343 *"Spiritual relationship is far more precious than physical. Physical relationship divorced from spiritual is body without soul" -- Mahatma Gandhi*

3 3

koha-US Board Meeting Reminder for Wednesday, January 10, 2024
by Kristi Krueger 08 Jan '24

08 Jan '24

Hello! The next koha-US Board Meeting is Wednesday, January 10, 2024. When: 8:30a PT / 11:30a ET (time converter <https://www.timeanddate.com/worldclock/fixedtime.html?msg=koha-US+Board+Mee…> ) Where: https://us02web.zoom.us/j/293750042?pwd=SGQzS2pmU3prNFhYRHNaeTFhOHN4QT09 Agenda: https://docs.google.com/document/d/1-Sy6ORSGNnYk6MNov_IhIK0U3THyPFRJaoOoY75… If you have additional topics for the agenda, please send them to secretary(a)koha-us.org. Meetings are held via Zoom video conferencing. A microphone and webcam are recommended but not required. Zoom is available for Windows, Mac, and Linux on desktops, and Android and iOS on mobile devices. For help getting started, visit: https://support.zoom.us/hc/en-us/categories/200101697 These meetings are always free and open to any and all. Kristi Krueger koha-US Secretary secretary(a)koha-US.org

1 0

January's Koha-US Circulation Special Interest Group
by Lauren Denny 08 Jan '24

08 Jan '24

Hi everyone, Join us for the next Koha-US Circulation meeting this Thursday, January 11, at 8 PT / 9 MT / 10 CT / 11 ET / 15 UTC You don't have to be a member of Koha-US to join us, we're hoping you'll come! Bring topics on anything to do with patrons, circulation, notices, OPAC, etc. Some possible ideas this month are patron templates and permissions. Here is the link to the Zoom meeting and https://us02web.zoom.us/j/962703559?pwd=SjJWcW9CRDJvMy8ycVIyTzIzOUdSUT09 Can't make it? No worries--it will be recorded and available on the Koha-US YouTube Channel: https://www.youtube.com/c/kohaUS/playlists (See the Circulation Special Interest Group's playlist for past circulation topics!) Feel free to send me any questions or topics for discussion ahead of time! See you there! Lauren Denny SIL International Consortium Librarian

1 0

Kohacon24 - Learning and Building Together
by Caroline Cyr La Rose 05 Jan '24

05 Jan '24

Dear Koha Community, This is a copy of the email sent to the Kohacon24 mailing list. Please consider subscribing (using the form here https://2024.kohacon.org/) to receive important information directly in your mailbox. We will try to keep the Koha mailing list in the loop, but it is not 100% guaranteed. The information is also available in the "News" section of the website https://2024.kohacon.org/news/ if you would rather visit the website than receive it by email. If you are subscribed to the Kohacon24 mailing list, but haven't received the email, please check your spam folder... Happy New Year! We hope this message finds you in good spirits and ready for an exciting announcement about Kohacon24 – the event we've all been eagerly awaiting. Dates and Venues =============================================================================== Mark your calendars for Kohacon24, scheduled to take place from September 23 to September 27, 2024, in the vibrant city of Montréal, Québec, Canada. The conference will unfold over the five days as follows: two days of insightful presentations, a day dedicated to cultural exploration, and two days filled with engaging workshops and hackfest activities. September 23 and 24: Conferences - Venue: Amphitheatre of the Coeur des sciences de l'UQAM (200, Sherbrooke Ouest, SH-2800). - Coffee breaks and work tables: Salle polyvalente in the same building (200, Sherbrooke Ouest, SH-4800). - Conferences will be live-streamed online (more details to follow). September 25: Cultural Day - Stay tuned for exciting activities! September 26 and 27: Workshops and Hackfest - Venue: Goethe Institut (1626, Saint-Laurent Blvd.). Theme: "Learning and Building Together" =============================================================================== This year's theme is "Learning and building together." We believe that collectively, we can learn and grow, with each participant contributing their unique knowledge and experience as a building block to the broader Koha project. Kohacon embraces the spirit of collaborative learning through conferences and collaborative building in workshops. Call for Speakers =============================================================================== We are thrilled to announce the opening of the call for speakers! We welcome your contributions, either in the form of a presentation, a workshop or a B-side discussion. Presentations ------------------------------------------------------------------------------- A presentation is a session where one or more people share their knowledge on a topic. This year, presentations can be delivered in English or French. This is something new we are trying in order to honour the cultural significance of the French language in Québec. Interpreters will be available to translate presentations into the other language. Submitting a presentation? Please include: - Your full name - Presentation title - Summary for the program - Length (15, 30, or 60 minutes) - Language (English or French) - Contact email A few weeks before the conference, we'll ask for your slides in PDF format to facilitate translation. To get your brain cells going, here are subjects that were mentioned in the answers to the survey. - Ways to use Koha to improve day-to-day operations - Interesting and unusual uses of Koha - Case studies - Workflows (especially acquisitions, ERM, preservation, serials, and ILL) - Updates, new features, and future roadmap - History of Koha - Koha discovery for new users - OPAC customization - Documentation - Translation - Ways for non-developers to get involved - Plugins - CSS / JS tips - Improving and modernizing Koha (new technologies, new needs, funding, wish list of things to improve) - Linked data - Technologies behind Koha (Perl, Vue, Plack, Elasticsearch, etc.) - Koha installation (hardware, docker, automation, resource sharing) - Integration with third-party products, external search engines Workshops ------------------------------------------------------------------------------- Workshops and hackfest are community-led hands-on sessions around a specific topic. One or more person can lead the session and everyone works together towards a common goal. Workshops can be led in English or French, but they will not be translated. Submitting a workshop? Please include: - Your full name and co-leaders (if any) - Workshop title - Summary of the workshop or learning objectives - Language (English or French) - Contact email To get your brain cells going, here are subjects that were mentioned in the answers to the survey. - Helping librarians get involved (singing off, translation, documentation) - Customization with CSS / JS - SQL and reports - Plugin development - Koha testing docker - How to QA B-side ------------------------------------------------------------------------------- The Salle polyvalente, a large room where we will have coffee breaks and where work tables will be installed, is spacious enough for informal gatherings during the conference days. If you have a specific topic for discussion or want to form a working group, let us know, and we'll include it in the program to garner interest. -- We look forward to your participation in making Kohacon24 a memorable and enriching experience for all. Let's learn and build together! Best regards, The Kohacon24 organization team (Caroline, Eric, Marie-Luce, and Olivier) kohacon24(a)inlibro.com <https://inLibro.com/en> Caroline Cyr-La-Rose, M.L.I.S. Librarian | Product Manager P 833-INLIBRO (465-4276) <tel:833-465-4276>, poste 221 E caroline.cyr-la-rose(a)inlibro.com www.inLibro.com <https://inLibro.com/en>

1 0

TT Notice question
by Stephen Graham 04 Jan '24

04 Jan '24

Hi All - not specifically Koha I guess, but maybe someone can explain why this happens. I had some TT in an email notice like: [% IF illrequests.branchcode == "CL" %] College Lane LRC [% ELSIF illrequests.branchcode == "DH" %] De Havilland LRC [% ELSE %] <<illrequests.branchcode>> [% END %] Every time I generated the email it printed CL, which is the value of llrequests.branchcode. I just could not get the first condition to work no matter what I did - it never printed College Lane LRC always CL. However, I changed it to: [% SET localbranch = illrequests.branchcode %] [% IF illrequests.branchcode == localbranch %] College Lane LRC [% ELSIF illrequests.branchcode == localbranch %] De Havilland LRC [% ELSE %] <<illrequests.branchcode>> [% END %] , and this works if I set the illrequests.branchcode to a local variable and then check this. Not sure why I have to do this? The example I have seen on the wiki look like I should be able to test a variable against a string using ==. Just couldn't get it to work. Any ideas? Stephen -------------------------------------- Stephen Graham Library Technology Consultant Library and Computing Services University of Hertfordshire Hatfield, Hertfordshire, AL10 9AB Email: s.graham4(a)herts.ac.uk<mailto:s.graham4@herts.ac.uk> Website: herts.ac.uk

1 0

koha-US General Meeting Minutes for Wednesday, January 03, 2024
by Kristi Krueger 04 Jan '24

04 Jan '24

Hello, Here's a link to the minutes from the koha-US General Meeting held on Wednesday, January 03, 2024 - https://docs.google.com/document/d/1wzwvOvU221sBK4bPPYaFyxzEFVXbR8eitexbCA2…. The recording for the general meeting has been posted, there's a link to the recording in the minutes. koha-US meetings are open to all! The next general meeting will be on Wednesday, 07 February, 2024 at 8:30 am PT / 9:30 am MT / 10:30 am CT / 11:30 am ET. The next board meeting will be on Wednesday, 10 January, 2024 at 8:30 am PT / 9:30 am MT / 10:30 am CT / 11:30 am ET. The February board meeting will be on Wednesday, 14 February, 2024 at 8:30 am PT / 9:30 am MT / 10:30 am CT / 11:30 am ET. Kristi Krueger koha-US Secretary secretary(a)koha-us.org

1 0

Koha 23.05.07 is here.
by Lucas Gass 03 Jan '24

03 Jan '24

Hello everyone, The Koha community is proud to announce the release of Koha 23.05.07. It is a bugfix/maintenance release including 11 bugfixes. The full release notes are available here: https://koha-community.org/koha-23-05-07-released/ Debian packages should be available soon 🎁 Happy new year, -- Lucas Gass Web Developer

1 0