Re: [Koha] Fwd: Dead Koha Resuscitation
On 5/15/23 12:12, David Liddle wrote:
I see now that I overlooked your statement that commands such as 'koha-dump' are not recognized. Do they appear in the directory /usr/sbin/?
No. Not a single koha- command present in that directory.
Do you have automated Koha backups configured that are available in /var/spool/koha/{instance}?
No, though that would have been smart of me. I don't even have a directory /var/spool/koha, much less /var/spool/koha/instance. Appreciate the effort. Respectfully, / Bruce / Bruce A. Metcalf, Librarian The Augustan Library
Bruce, it’s an intriguing problem, since you’ve come from a working system. However, I shouldn’t think about it while I ought to be sleeping. Perhaps others will chime in as the sun works its way across the US. But I do have some more questions: - What other services are running on that server? Are they all still working correctly? - What clues are there in the logs in /var/log/? - Is the apt installer working generally, or do you perhaps have a broken Debian installation? Can you update or install other packages successfully? - Can you access the database server to do a manual dump of the Koha database? On Mon, May 15, 2023 at 8:51 PM Bruce A. Metcalf < library@augustansociety.org> wrote:
On 5/15/23 12:12, David Liddle wrote:
I see now that I overlooked your statement that commands such as 'koha-dump' are not recognized. Do they appear in the directory /usr/sbin/?
No. Not a single koha- command present in that directory.
Do you have automated Koha backups configured that are available in /var/spool/koha/{instance}?
No, though that would have been smart of me. I don't even have a directory /var/spool/koha, much less /var/spool/koha/instance.
Appreciate the effort.
Respectfully, / Bruce / Bruce A. Metcalf, Librarian The Augustan Library
Hello again, Bruce, I had some more thoughts today about your situation. What are the results of the following commands? Clearly your installation is incomplete. dpkg -l koha-common dpkg -s koha-common They are similar, in that the first is a summary and the second is more detailed. For example, one of my systems shows the following with the first: ~# dpkg -l koha-common Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-=============================================== hi koha-common 22.05.12-1 all integrated (physical) library management system And the same system shows this with apt: ~# apt policy koha-common koha-common: Installed: 22.05.12-1 Candidate: 22.05.12-1 Version table: *** 22.05.12-1 500 500 http://debian.koha-community.org/koha 22.05/main amd64 Packages 100 /var/lib/dpkg/status In case you need the commands for grabbing your database: ~# mysql -uroot -p
show databases; (take note of KOHA_DB in the table) quit ~# mysqldump --databases KOHA_DB > koha_db_dump.sql
If your other configuration files – most of which are in /etc/koha/ and /var/lib/koha/ – are missing, you may be able to copy them from within the full-drive backup. Some formats are easier to mount and browse than others. What do you use? On Mon, May 15, 2023 at 10:35 PM David Liddle <david@liddles.net> wrote:
Bruce, it’s an intriguing problem, since you’ve come from a working system. However, I shouldn’t think about it while I ought to be sleeping. Perhaps others will chime in as the sun works its way across the US. But I do have some more questions:
- What other services are running on that server? Are they all still working correctly? - What clues are there in the logs in /var/log/? - Is the apt installer working generally, or do you perhaps have a broken Debian installation? Can you update or install other packages successfully? - Can you access the database server to do a manual dump of the Koha database?
On Mon, May 15, 2023 at 8:51 PM Bruce A. Metcalf <library@augustansociety.org> wrote:
On 5/15/23 12:12, David Liddle wrote:
I see now that I overlooked your statement that commands such as 'koha-dump' are not recognized. Do they appear in the directory /usr/sbin/?
No. Not a single koha- command present in that directory.
Do you have automated Koha backups configured that are available in /var/spool/koha/{instance}?
No, though that would have been smart of me. I don't even have a directory /var/spool/koha, much less /var/spool/koha/instance.
Appreciate the effort.
Respectfully, / Bruce / Bruce A. Metcalf, Librarian The Augustan Library
On 5/16/23 12:31, David Liddle wrote:
What are the results of the following commands? Clearly your installation is incomplete.
dpkg -l koha-common
root@store:/var/lib/mysql# dpkg -l koha-common Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-=============================================== ii koha-common 21.11.19-1 all integrated (physical) library management system
dpkg -s koha-common
root@store:/var/lib/mysql# dpkg -s koha-common Package: koha-common Status: install ok installed Priority: optional Section: web Installed-Size: 574598 Maintainer: Mason James <mtj@kohaaloha.com> Architecture: all Source: koha Version: 21.11.19-1 Depends: ... [many lines deleted]
They are similar, in that the first is a summary and the second is more detailed. For example, one of my systems shows the following with the first:
~# dpkg -l koha-common Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-============-============-=============================================== hi koha-common 22.05.12-1 all integrated (physical) library management system
Which is a pretty fair match. Not at all sure what the latter is supposed to show me. (Full dump on request.)
And the same system shows this with apt:
~# apt policy koha-common koha-common: Installed: 22.05.12-1 Candidate: 22.05.12-1 Version table: *** 22.05.12-1 500 500 http://debian.koha-community.org/koha 22.05/main amd64 Packages 100 /var/lib/dpkg/status
root@store:/var/lib/mysql# apt policy koha-common koha-common: Installed: 21.11.19-1 Candidate: 21.11.19-1 Version table: *** 21.11.19-1 500 500 http://debian.koha-community.org/koha oldoldstable/main i386 Packages 100 /var/lib/dpkg/status Again, pretty much a match. So if these responses are good, where is my problem lurking?
In case you need the commands for grabbing your database:
~# mysql -uroot -p
show databases;
Mine shows: root@store:/var/lib/mysql# mysql -uroot -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 18971 Server version: 10.3.38-MariaDB-0+deb10u1 Debian 10
(take note of KOHA_DB in the table)
No table there to take note of.
~# mysqldump --databases KOHA_DB > koha_db_dump.sql
This command gives me "access denied" for both root and myself.
If your other configuration files – most of which are in /etc/koha/ and /var/lib/koha/ – are missing, you may be able to copy them from within the full-drive backup.
Buncha stuff in both directories. Not sure if they're complete or not.
Some formats are easier to mount and browse than others. What do you use?
I use the automatic backup run by my virtual machine service. It restores just fine, but don't know if it can do file-by-file restores. Thanks for your continuing efforts, glad to test and post other items. Respectfully, / Bruce / Bruce A. Metcalf, Librarian The Augustan Library
On 5/15/23 16:35, David Liddle wrote:
Bruce, it’s an intriguing problem.... But I do have some more questions:
- What other services are running on that server? Are they all still working correctly?
There is another Apache server and a MediaWiki server (which may also run under Apache). Both are working as before with no problems or noticeable slowdowns.
- What clues are there in the logs in /var/log/?
The only logs with any entries are access logs which show that the Chinese and Eastern Europeans are trying to hack the system. I don't think those are relevant. What may be relevant is that TOP does not show that Koha is running, though this could be just that the command isn't obvious.
- Is the apt installer working generally, or do you perhaps have a broken Debian installation? Can you update or install other packages successfully?
The apt installer appears to be working, but it is not updating Koha apps, and there have been several updates since this began. The repository link appears to be correct; I'm (trying to run) on "oldoldstable".
- Can you access the database server to do a manual dump of the Koha database?
Not using any Koha tools or commands. I might be able to do so manually, but am clueless about how to do so. Thanks for your continuing efforts. Respectfully, / Bruce / Bruce A. Metcalf, Librarian The Augustan Library
Bruce, here are some more suggestions: 1. When it comes to the logs, perhaps a command such as these (and similar) will reveal when and how something went wrong: grep -inr --include \*.log 'koha-common' /var/log/ zgrep -in 'koha-common' /var/log/dpkg.log* 2. You should double-check that your installation source file is correct, as covered by https://wiki.koha-community.org/wiki/Koha_on_Debian#Set_up_package_sources. For example, mine shows after the following command: ~# cat /etc/apt/sources.list.d/koha.list deb [signed-by=/usr/share/keyrings/koha-keyring.gpg] https://debian.koha-community.org/koha 22.05 main 3. If APT seems to be working, then you could install 'debsums' (apt install debsums) and see what it reveals with this command: debsums --all --silent dpkg --verify koha-common dpkg --audit koha-common Those are going to look for errors in packages and list the configuration files that have been altered from their default. It listed nicely the files that I already knew I changed for both Koha and Apache. 4. You may want to try the following commands, too: apt --fix-broken install apt --fix-missing install 5. If the Koha installation is incomplete, despite what the system seems to report, then you may want to use 'dpkg' to remove 'koha-common' and then reinstall with 'apt' according to the instructions. dpkg --remove koha-common 6. When it comes to listing the databases, here's what you might expect to see: ~# mysql -uroot -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 348 Server version: 10.5.19-MariaDB-0+deb11u2 Debian 11 Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | koha_library | | mysql | | performance_schema | +--------------------+ 4 rows in set (0.000 sec) MariaDB [(none)]> quit Bye Since our instance is named 'library', you can see that our database is named 'koha_library'. It is customary for the instance creation process to use the 'koha_' prefix for the database name. Does yours show something similar? On Tue, May 16, 2023 at 6:40 PM Bruce A. Metcalf <library@augustansociety.org> wrote:
On 5/15/23 16:35, David Liddle wrote:
Bruce, it’s an intriguing problem.... But I do have some more questions:
- What other services are running on that server? Are they all still working correctly?
There is another Apache server and a MediaWiki server (which may also run under Apache). Both are working as before with no problems or noticeable slowdowns.
- What clues are there in the logs in /var/log/?
The only logs with any entries are access logs which show that the Chinese and Eastern Europeans are trying to hack the system. I don't think those are relevant.
What may be relevant is that TOP does not show that Koha is running, though this could be just that the command isn't obvious.
- Is the apt installer working generally, or do you perhaps have a broken Debian installation? Can you update or install other packages successfully?
The apt installer appears to be working, but it is not updating Koha apps, and there have been several updates since this began. The repository link appears to be correct; I'm (trying to run) on "oldoldstable".
- Can you access the database server to do a manual dump of the Koha database?
Not using any Koha tools or commands. I might be able to do so manually, but am clueless about how to do so.
Thanks for your continuing efforts.
Respectfully, / Bruce / Bruce A. Metcalf, Librarian The Augustan Library
On 5/16/23 15:29, David Liddle wrote:
Yet another command that may shed some light on the status of your installation: ~# systemctl status koha-common
root@store:/etc/koha# systemctl status koha-common ● koha-common.service - LSB: Start required services for each Koha instance Loaded: loaded (/etc/init.d/koha-common; generated) Active: active (running) since Sun 2023-05-14 12:45:09 EDT; 3 days ago Docs: man:systemd-sysv-generator(8) Process: 2082 ExecStart=/etc/init.d/koha-common start (code=exited, status=0/SUCCESS) Tasks: 6 (limit: 4915) Memory: 172.2M CGroup: /system.slice/koha-common.service ├─2408 daemon --name=library-koha-zebra --pidfiles=/var/run/koha/library/ --errlog=/var/log/koha/library/zebra-error.log --output=/var/log/koha/library/zebra- ├─2410 /usr/bin/zebrasrv -v none,fatal,warn -k 1024 -f /etc/koha/sites/library/koha-conf.xml ├─2658 daemon --name=library-koha-worker --errlog=/var/log/koha/library/worker-error.log --stdout=/var/log/koha/library/worker.log --output=/var/log/koha/libr ├─2659 /usr/bin/perl /usr/share/koha/bin/background_jobs_worker.pl ├─2698 daemon --name=library-koha-indexer --errlog=/var/log/koha/library/indexer-error.log --output=/var/log/koha/library/indexer-output.log --pidfiles=/var/r └─2699 /usr/bin/perl /usr/share/koha/bin/migration_tools/rebuild_zebra.pl -daemon -sleep 19 Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable. So, it looks like Koha is running, but it's not responding to calls from Apache? Regards, / Bruce / Bruce A. Metcalf, Librarian The Augustan Library
On 5/16/23 15:10, David Liddle wrote:
Bruce, here are some more suggestions:
1. When it comes to the logs, perhaps a command such as these (and similar) will reveal when and how something went wrong: grep -inr --include \*.log 'koha-common' /var/log/
Results: Binary file /var/log/daemon.log matches
zgrep -in 'koha-common' /var/log/dpkg.log*
This results in many lines like: /var/log/dpkg.log.10.gz:161:2022-04-29 13:51:36 status unpacked koha-common:all 21.05.14-1 And a few like: /var/log/dpkg.log.10.gz:161:2022-04-29 13:51:36 status unpacked koha-common:all 21.05.14-1 /var/log/dpkg.log.1:20:2023-04-12 23:07:03 status half-configured koha-common:all 21.11.17-1 /var/log/dpkg.log.1:22:2023-04-12 23:07:04 status half-installed koha-common:all 21.11.17-1 Needless to say, I have no idea what these messages mean or how to fix them.
2. You should double-check that your installation source file is correct, as covered by https://wiki.koha-community.org/wiki/Koha_on_Debian#Set_up_package_sources. For example, mine shows after the following command: ~# cat /etc/apt/sources.list.d/koha.list deb [signed-by=/usr/share/keyrings/koha-keyring.gpg] > https://debian.koha-community.org/koha 22.05 main
Mine is: deb http://debian.koha-community.org/koha oldoldstable main
3. If APT seems to be working, then you could install 'debsums' (apt install debsums) and see what it reveals with this command: debsums --all --silent
# /etc/koha# debsums --all --silent debsums: changed file /etc/koha/zebradb/etc/default.idx (from koha-common package) debsums: changed file /etc/cron.d/koha-common (from koha-common package) debsums: changed file /etc/koha/zebradb/lang_defs/en/sort-string-utf.chr (from koha-common package) debsums: changed file /etc/default/koha-common (from koha-common package) debsums: changed file /etc/koha/zebradb/etc/word-phrase-utf.chr (from koha-common package) debsums: changed file /etc/koha/koha-sites.conf (from koha-common package) debsums: changed file /etc/cron.daily/mlocate (from mlocate package) plus some unrelated lines of that ilk.
dpkg --verify koha-common
root@store:/etc/koha# dpkg --verify koha-common ??5?????? c /etc/cron.d/koha-common ??5?????? c /etc/default/koha-common ??5?????? c /etc/koha/apache-shared-opac.conf ??5?????? c /etc/koha/koha-sites.conf ??5?????? c /etc/koha/zebradb/etc/default.idx ??5?????? c /etc/koha/zebradb/etc/word-phrase-utf.chr ??5?????? c /etc/koha/zebradb/lang_defs/en/sort-string-utf.chr
dpkg --audit koha-common
This generates no response.
Those are going to look for errors in packages and list the configuration files that have been altered from their default. It listed nicely the files that I already knew I changed for both Koha and Apache. 4. You may want to try the following commands, too: apt --fix-broken install apt --fix-missing install
Both result in no actions.
5. If the Koha installation is incomplete, despite what the system seems to report, then you may want to use 'dpkg' to remove 'koha-common' and then reinstall with 'apt' according to the instructions. dpkg --remove koha-common
root@store:/etc/koha# dpkg --remove koha-common dpkg: warning: 'ldconfig' not found in PATH or not executable dpkg: warning: 'start-stop-daemon' not found in PATH or not executable dpkg: error: 2 expected programs not found in PATH or not executable Note: root's PATH should usually contain /usr/local/sbin, /usr/sbin and /sbin Aha! my $PATH contains no mention of any sbin directory. Let me see if I can add it. Hmm. My ~/.bashrc file is effectively empty, with all lines commented out. It also looks "wrong".
6. When it comes to listing the databases, here's what you might expect to see:
~# mysql -uroot -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 348 Server version: 10.5.19-MariaDB-0+deb11u2 Debian 11
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | koha_library | | mysql | | performance_schema | +--------------------+ 4 rows in set (0.000 sec)
MariaDB [(none)]> quit Bye
This generates: +--------------------+ | Database | +--------------------+ | aug_wiki | | augustan | | information_schema | | koha_library | | mysql | | performance_schema | +--------------------+ The instance name is "library", so that's the 4th entry. "aug_wiki" is from our wiki installation, not sure what "augustan" is. So, we have some new data, but nothing that shows me what needs to be done. I hope someone here can tell me what it is. Regards, / Bruce / Bruce A. Metcalf, Librarian The Augustan Library
participants (2)
-
Bruce A. Metcalf -
David Liddle