Taking patron images out of db into a directory
Hi all, We'd like to write a patch to move patron images out of db into a directory in the server. By this change, we'd like to satisfy following. 1. The size of database backup will be decreased, by this way one may store db backups more. 2. Database performance will increase 3. It will be possible to put patron images in different disk in the server 4. Easily put images gathered from other systems like (university system, personnel system, etc) What is your idea? -- Mengü Yazıcıoğlu Devinim Yazılım Eğitim Danışmanlık Reşit Galip Cad. No: 29/6 Çankaya/ANKARA Tel: (312) 446 86 68 Cep: (532) 701 90 27
We'd like to write a patch to move patron images out of db into a directory in the server.
I think this is a good idea. There's no reason not to offer it as a choice if it's possible to do both. I think an argument could be made that it shouldn't be a choice presented to the librarian and therefore should be a setting in Koha's configuration file rather than, say, a system preference. -- Owen -- Web Developer Athens County Public Libraries (740) 737-6006 https://www.myacpl.org
Greetings,
We'd like to write a patch to move patron images out of db into a directory in the server.
Mmmm... no. People doing backups might forget to supplement the default Koha backup with that specific directory, and come a system failure and a restore later... they discover the photos are no more.
1. The size of database backup will be decreased, by this way one may store db backups more.
Only if the pictures are on a different server.
2. Database performance will increase
Theoretically, yes. But I don't see how one tiny thing would speed it up that much, unless ... (goes to update his branches and see the schema) Oh... okay... yes, perhaps patron images should be in its own linked table, so that borrower.* won't be hindered with patronimage data.
3. It will be possible to put patron images in different disk in the server 4. Easily put images gathered from other systems like (university system, personnel system, etc)
Putting the files in another collection that gets backed up differently would be okay.
What is your idea?
I prefer the notion of in DB or externally controlled by system preferences. I think system preferences, because librarians may not have access to the technical koha-conf.xml configuration settings file. And patron images would likely fall under the responsibility of librarians, not IT people. GPML, Mark Tompsett
Hi Mark and all, I kind of disagree with a few points. I see no harm in storing the patron images optionally outside of the database as suggested by Owen. I don't think the database is a good place to store files actually, storing them outside makes a lot of sense to me. Patron images are currently stored in their own linked table called patronimage. The setting of paths is usually done in the koha-conf.xml file so I think that would be the place for configuration of patron image storage too. Afaik we already have an option to include index files and uploaded files in the backups - I think it should not be hard to add another option. We could make the switch between database and directory storage a pref maybe. Katrin On 10.04.20 21:49, Mark Tompsett wrote:
Greetings,
We'd like to write a patch to move patron images out of db into a directory in the server.
Mmmm... no. People doing backups might forget to supplement the default Koha backup with that specific directory, and come a system failure and a restore later... they discover the photos are no more.
1. The size of database backup will be decreased, by this way one may store db backups more.
Only if the pictures are on a different server.
2. Database performance will increase
Theoretically, yes. But I don't see how one tiny thing would speed it up that much, unless ... (goes to update his branches and see the schema) Oh... okay... yes, perhaps patron images should be in its own linked table, so that borrower.* won't be hindered with patronimage data.
3. It will be possible to put patron images in different disk in the server 4. Easily put images gathered from other systems like (university system, personnel system, etc)
Putting the files in another collection that gets backed up differently would be okay.
What is your idea?
I prefer the notion of in DB or externally controlled by system preferences. I think system preferences, because librarians may not have access to the technical koha-conf.xml configuration settings file. And patron images would likely fall under the responsibility of librarians, not IT people.
GPML, Mark Tompsett _______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
On Fri, Apr 10, 2020 at 3:50 PM Mark Tompsett <mtompset@hotmail.com> wrote:
Mmmm... no.
This kind of comment is completely counterproductive to meaningful cooperation. Please be constructive without being dismissive. -- Owen -- Web Developer Athens County Public Libraries (740) 737-6006 https://www.myacpl.org
Greetings, I wrote:
Mmmm... no.
Owen replied:
This kind of comment is completely counterproductive to meaningful cooperation. Please be constructive without being dismissive.
If that had been all I wrote, then your comment would be valid in my mind. However, you didn't include the rest of the comment which explained why no: "People doing backups might forget to supplement the default Koha backup with that specific directory, and come a system failure and a restore later... they discover the photos are no more." I also said: "Putting the files in another collection that gets backed up differently would be okay." So... over all, I do not think I was dismissive, and apologize if people read it that way. I was sharing worries, because I believe someone (probably me and others), somewhere, somehow will forget to configure something, and then something bad will happen, because they forgot or didn't realize. After all, the original poster did ask, "What is your idea?" which I took to mean, "What do you think?" Sharing worries and gut reactions are valid answers to the question, though as Owen noted not useful for trying to find a good direction. +1 to Tomas' comment about command line utilities to keep sync. Katrin wrote:
Afaik we already have an option to include index files and uploaded files in the backups - I think it should not be hard to add another option.
If we do, that's great. If not, we have a functionality gap to solve. Additionally, system level things are usually beyond the access of librarians and only IT. Katrin addressed my concern:
We could make the switch between database and directory storage a syspref maybe.
+1 to this idea. Hope this clarifies. Peace, Mark Tompsett
El jue., 9 abr. 2020 a las 11:41, Mengu Yazicioglu (<mengu@devinim.com.tr>) escribió:
Hi all,
We'd like to write a patch to move patron images out of db into a directory in the server.
By this change, we'd like to satisfy following.
1. The size of database backup will be decreased, by this way one may store db backups more.
And they would take far less time to finish. Also, using incremental backups would be possible.
2. Database performance will increase
Not sure about that. But the DB usually uses caching for the rows, and with a limited space, so it *could* improve caching. 3. It will be possible to put patron images in different disk in the server
Yes, taking them out of the DB comes with this advantages. 4. Easily put images gathered from other systems like (university
system, personnel system, etc)
It should definitely be using Koha::UploadedFIles. We really want to keep the consistency between the DB and the images. Command-line utilities could be developed to sync new batches, check nothing is missing, etc. Good idea! -- Tomás Cohen Arazi Theke Solutions (http://theke.io) ✆ +54 9351 3513384 GPG: B2F3C15F
Mengu, We did something similar for one clients. It's patrons pictures were on an external server using the borrowernumber as filename (they have a unified id for all of their services) So their is definitively use cases for such a feature. In our case, we just "hacked" the picture URL with their server URL. I think we should have a syspref for the patron pictures base URL, and maybe the key to use as filename (cardnumber, borrowernumber or even an patron attribute) Hope that help ! Eric Bégin Solutions inLibro inc. Le 9 avr. 2020 à 10:42, à 10:42, Mengu Yazicioglu <mengu@devinim.com.tr> a écrit:
Thanks for everyone's comments. We've developed this kind of feature first for 3.20 version with more than 1,600,000 users in the past. Some of our new customers also wanted this feature for new versions and with your comments, we'll start to write it soon. Thanks everyone ! Mengü Yazıcıoğlu Devinim Yazılım Eğitim Danışmanlık Reşit Galip Cad. No: 29/6 Çankaya/ANKARA Tel: (312) 446 86 68 Cep: (532) 701 90 27 On 11.04.2020 03:56, Eric Bégin wrote:
Mengu,
We did something similar for one clients. It's patrons pictures were on an external server using the borrowernumber as filename (they have a unified id for all of their services)
So their is definitively use cases for such a feature.
In our case, we just "hacked" the picture URL with their server URL.
I think we should have a syspref for the patron pictures base URL, and maybe the key to use as filename (cardnumber, borrowernumber or even an patron attribute)
Hope that help !
Eric Bégin Solutions inLibro inc. Le 9 avr. 2020, à 10:42, Mengu Yazicioglu <mengu@devinim.com.tr <mailto:mengu@devinim.com.tr>> a écrit:
The first step: write the tests! El sáb., 11 de abril de 2020 17:47, Mengu Yazicioglu <mengu@devinim.com.tr> escribió:
Thanks for everyone's comments.
We've developed this kind of feature first for 3.20 version with more than 1,600,000 users in the past.
Some of our new customers also wanted this feature for new versions and with your comments, we'll start to write it soon.
Thanks everyone !
Mengü Yazıcıoğlu Devinim Yazılım Eğitim Danışmanlık Reşit Galip Cad. No: 29/6 Çankaya/ANKARA
Tel: (312) 446 86 68 Cep: (532) 701 90 27
On 11.04.2020 03:56, Eric Bégin wrote:
Mengu,
We did something similar for one clients. It's patrons pictures were on an external server using the borrowernumber as filename (they have a unified id for all of their services)
So their is definitively use cases for such a feature.
In our case, we just "hacked" the picture URL with their server URL.
I think we should have a syspref for the patron pictures base URL, and maybe the key to use as filename (cardnumber, borrowernumber or even an patron attribute)
Hope that help !
Eric Bégin Solutions inLibro inc. Le 9 avr. 2020, à 10:42, Mengu Yazicioglu <mengu@devinim.com.tr <mailto:mengu@devinim.com.tr>> a écrit:
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Greetings, Tomas Cohen Arazi wrote:
The first step: write the tests! +1000
By writing the tests first, you know what you are coding towards, and have full test coverage for what you are going to provide a patch for. This makes everything easier for reviewing. The test plan becomes: run the tests, they have full code coverage. GPML, Mark Tompsett P.S. El sáb., 11 de abril de 2020 17:47, Mengu Yazicioglu <mengu@devinim.com.tr> escribió:
We've developed this kind of feature first for 3.20 version with more than 1,600,000 users in the past.
Ouch! We were stuck on that version for the longest time. May you have success in coding and implementing your patch!
Hi,
El sáb., 11 de abril de 2020 17:47, Mengu Yazicioglu <mengu@devinim.com.tr> escribió:
We've developed this kind of feature first for 3.20 version with more than 1,600,000 users in the past.
Ouch! We were stuck on that version for the longest time. May you have success in coding and implementing your patch!
Why do you think that we may not? We're supporting Koha since v.2, and we also did similar thing in 16.11. I just wanted to give the above number of persons to show that taking pictures out of db is a must in some cases. In any size, taking pictures out of db will give more benefits than saving in database. Regards.
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Mengu, I read Mark as saying he's been there as well, stuck on a version for a long time due to some customization. I don't think he's talking about your ability to implement it at all! El lun., 13 abr. 2020 a las 5:57, Mengu Yazicioglu (<mengu@devinim.com.tr>) escribió:
Hi,
El sáb., 11 de abril de 2020 17:47, Mengu Yazicioglu <mengu@devinim.com.tr> escribió:
We've developed this kind of feature first for 3.20 version with more than 1,600,000 users in the past.
Ouch! We were stuck on that version for the longest time. May you have success in coding and implementing your patch!
Why do you think that we may not? We're supporting Koha since v.2, and we also did similar thing in 16.11. I just wanted to give the above number of persons
to show that taking pictures out of db is a must in some cases. In any size, taking pictures out of db will give more benefits than saving in database.
Regards.
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
-- Tomás Cohen Arazi Theke Solutions (http://theke.io) ✆ +54 9351 3513384 GPG: B2F3C15F
Greetings, I wrote:
Ouch! We were stuck on that version for the longest time. May you have success in coding and implementing your patch!
To which Mengu Yazicioglu replied:
Why do you think that we may not? We're supporting Koha since v.2, and we also did similar thing in 16.11.
I apologize, I misread. I thought you were still on 3.20 for that patch. Upgrading patches is not always easy. I was offering best wishes and good will if that was the case. I also apologize for clearly not expressing myself well enough, such that I keep being understood as saying mean things. I'll go back to being silent now. Grace, Peace, Mercy, and Love to everyone... Mark Tompsett
participants (6)
-
Eric Bégin -
Katrin Fischer -
Mark Tompsett -
Mengu Yazicioglu -
Owen Leonard -
Tomas Cohen Arazi