Re: [Koha] GDPR - Statistics and anonymization
Michal, Redirection the discussion back to the list. What do you mean by "we want to see them as results from borrower searches"? What I understood is that libraries want to collect statistics on transactions even when the patron's records have been deleted. So: 1/ Create a patron (it's copied to borrowers and anonymized_borrowers) 2/ Do some transactions (it's in statistics and anonymized_transactions) 3/ Update a patron (it's updated in borrowers and anonymized_borrowers, if option 1 is picked) 4/ Delete the patron (moved from borrowers to deletedborrowers) 6/ Clean/purge the deletedborrowers => anonymized_* tables are still there, statistics still possible 7/ When needed, given parameters, anonymized_* entries will be deleted. Le ven. 22 nov. 2019 à 15:51, Mike D. <black23@gmail.com> a écrit :
Hello, I think that thinks are maybe less complicated. I really like Your idea with hash. But if we anonymize (replace by some generated hash or something similar) name, adress, telephone, e-mai and adress, noticess, we don't need related information about borrowers. Because we cut the rope between data and person. Anonymized borrower records we can set as "anonymized" and store them separately from "live" borrowers, because we want to see them as results from borrower searches. What do You think about this?
Michal
pá 22. 11. 2019 v 15:41 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
2 tables vs 1 table :)
Le ven. 22 nov. 2019 à 15:28, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, what options? I miss a point :-)
Michal
pá 22. 11. 2019 v 15:17 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
Thanks for the help Michal, What about the two options I have?
Le jeu. 21 nov. 2019 à 17:58, Mike D. <black23@gmail.com> a écrit :
Hi Jonathan, I’m volunteer for debate about processes and anon tools and methods. I’m ready to be tester of bugs. Koha is GDPR ready but some points could be improved for easier everyday usage in libraries. Because if something is clear and easy everybody do it without fear and stress.
Thank You
Michal
čt 21. 11. 2019 v 17:14 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
Hello everybody,
I have been contracted by KohaLa to work on some GDPR requirements. The main idea is to "anonymize" patron's data but letting the library access the transactions' statistics.
I am going to present you what I am planning to implement, in order to collect ideas and answers.
There are the following steps I have in mind: 1. Pseudonymization [1] of patron's data 2. Improve deletion of patron related date (tables statistics, old_reserves, deletedborrowers) 3. Add the ability to remove data that have been pseudonymized
I see 2 ways to achieve point 1: * We create 2 tables, 1 for the patrons, 1 for the transactions. - borrowers_anonymized will contain: hash_id, has_cardnumber, branchcode, creation_date, categorycode, bsort1, bsort2, [borrower_attributes] - transaction_anonymized will contain: hash_id, transaction_type, branchcode, itemnumber, holdingbranch, location, itemcallnumber, itemtype, timestamp
hash_id will be generated using the borrowernumber and a key (that will be stored on the server, path in koha-conf)
Pros: Easier to understand and manipulate as it follows existing structure. We track patron's modifications (this is the most important part) Cons: tech part: new config, a new path have to be created (minor)
* We create only 1 table, (nosql-like). It will contain the same data as previously, without the hash_id
Pros: No new config. Data are never updated and we have the values when the transactions has been processed. Cons: Data are not updated :)
About borrower_attributes, the initial specification asks for 2 attributes defined in a syspref. I think it should be configurable, with a join table (Pro: more flexible, Con: SQL requests more complex)
I think we should have the 2 tables and keep a link between the anonymized_patrons and anonymized_transactions tables.
What do you think? I am going to start the implementation very soon in order to plan an integration early in the 20.05 dev cycle.
Regards, Jonathan
[1] https://en.wikipedia.org/wiki/Pseudonymization _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
Hello Jonathan, sorry "we want to see them as results from borrower searches" shoub be "we want to hide them (anonymized borrower records) from all results from searches" Really we need to copy every patron into anonymized_borrowers at time of creation of their records? I think that must be done only in case that we want to anonymize borrower. Not before this day. Because if we will copy all translation into new table we'll maka database bigger wi same data. Do I understand it clear? Michal pá 22. 11. 2019 v 16:08 odesílatel Jonathan Druart < jonathan.druart@bugs.koha-community.org> napsal:
Michal,
Redirection the discussion back to the list.
What do you mean by "we want to see them as results from borrower searches"?
What I understood is that libraries want to collect statistics on transactions even when the patron's records have been deleted. So: 1/ Create a patron (it's copied to borrowers and anonymized_borrowers) 2/ Do some transactions (it's in statistics and anonymized_transactions) 3/ Update a patron (it's updated in borrowers and anonymized_borrowers, if option 1 is picked) 4/ Delete the patron (moved from borrowers to deletedborrowers) 6/ Clean/purge the deletedborrowers => anonymized_* tables are still there, statistics still possible
7/ When needed, given parameters, anonymized_* entries will be deleted.
Le ven. 22 nov. 2019 à 15:51, Mike D. <black23@gmail.com> a écrit :
Hello, I think that thinks are maybe less complicated. I really like Your idea
with hash. But if we anonymize (replace by some generated hash or something similar) name, adress, telephone, e-mai and adress, noticess, we don't need related information about borrowers. Because we cut the rope between data and person. Anonymized borrower records we can set as "anonymized" and store them separately from "live" borrowers, because we want to see them as results from borrower searches. What do You think about this?
Michal
pá 22. 11. 2019 v 15:41 odesílatel Jonathan Druart <
2 tables vs 1 table :)
Le ven. 22 nov. 2019 à 15:28, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, what options? I miss a point :-)
Michal
pá 22. 11. 2019 v 15:17 odesílatel Jonathan Druart <
jonathan.druart@bugs.koha-community.org> napsal:
Thanks for the help Michal, What about the two options I have?
Le jeu. 21 nov. 2019 à 17:58, Mike D. <black23@gmail.com> a écrit :
Hi Jonathan, I’m volunteer for debate about processes and anon tools and
methods. I’m ready to be tester of bugs. Koha is GDPR ready but some points could be improved for easier everyday usage in libraries. Because if something is clear and easy everybody do it without fear and stress.
Thank You
Michal
čt 21. 11. 2019 v 17:14 odesílatel Jonathan Druart <
jonathan.druart@bugs.koha-community.org> napsal:
> > Hello everybody, > > I have been contracted by KohaLa to work on some GDPR requirements. > The main idea is to "anonymize" patron's data but letting the
jonathan.druart@bugs.koha-community.org> napsal: library
> access the transactions' statistics. > > I am going to present you what I am planning to implement, in order to > collect ideas and answers. > > There are the following steps I have in mind: > 1. Pseudonymization [1] of patron's data > 2. Improve deletion of patron related date (tables statistics, > old_reserves, deletedborrowers) > 3. Add the ability to remove data that have been pseudonymized > > I see 2 ways to achieve point 1: > * We create 2 tables, 1 for the patrons, 1 for the transactions. > - borrowers_anonymized will contain: hash_id, has_cardnumber, > branchcode, creation_date, categorycode, bsort1, bsort2, > [borrower_attributes] > - transaction_anonymized will contain: hash_id, transaction_type, > branchcode, itemnumber, holdingbranch, location, itemcallnumber, > itemtype, timestamp > > hash_id will be generated using the borrowernumber and a key (that > will be stored on the server, path in koha-conf) > > Pros: Easier to understand and manipulate as it follows existing structure. > We track patron's modifications (this is the most important part) > Cons: tech part: new config, a new path have to be created (minor) > > * We create only 1 table, (nosql-like). It will contain the same data > as previously, without the hash_id > > Pros: No new config. Data are never updated and we have the values > when the transactions has been processed. > Cons: Data are not updated :) > > About borrower_attributes, the initial specification asks for 2 > attributes defined in a syspref. I think it should be configurable, > with a join table (Pro: more flexible, Con: SQL requests more complex) > > I think we should have the 2 tables and keep a link between the > anonymized_patrons and anonymized_transactions tables. > > What do you think? > I am going to start the implementation very soon in order to plan an > integration early in the 20.05 dev cycle. > > Regards, > Jonathan > > [1] https://en.wikipedia.org/wiki/Pseudonymization > _______________________________________________ > Koha mailing list http://koha-community.org > Koha@lists.katipo.co.nz > https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
The anonymized_borrowers table will not contain all data from borrowers. Only a small part. Also we want to track down the transactions when they happen, it will be much more easier to implement and less error prone. How do you want to retrieve the transactions for a given patron when they are anonymized? Also you want to collect statistics only on this table, it will be impossible to make statistics on this table for "anonymized" patrons, and another table for the ones that are not anonymized yet. Does it make sense? Le ven. 22 nov. 2019 à 16:33, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, sorry "we want to see them as results from borrower searches" shoub be "we want to hide them (anonymized borrower records) from all results from searches"
Really we need to copy every patron into anonymized_borrowers at time of creation of their records? I think that must be done only in case that we want to anonymize borrower. Not before this day. Because if we will copy all translation into new table we'll maka database bigger wi same data. Do I understand it clear?
Michal
pá 22. 11. 2019 v 16:08 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
Michal,
Redirection the discussion back to the list.
What do you mean by "we want to see them as results from borrower searches"?
What I understood is that libraries want to collect statistics on transactions even when the patron's records have been deleted. So: 1/ Create a patron (it's copied to borrowers and anonymized_borrowers) 2/ Do some transactions (it's in statistics and anonymized_transactions) 3/ Update a patron (it's updated in borrowers and anonymized_borrowers, if option 1 is picked) 4/ Delete the patron (moved from borrowers to deletedborrowers) 6/ Clean/purge the deletedborrowers => anonymized_* tables are still there, statistics still possible
7/ When needed, given parameters, anonymized_* entries will be deleted.
Le ven. 22 nov. 2019 à 15:51, Mike D. <black23@gmail.com> a écrit :
Hello, I think that thinks are maybe less complicated. I really like Your idea with hash. But if we anonymize (replace by some generated hash or something similar) name, adress, telephone, e-mai and adress, noticess, we don't need related information about borrowers. Because we cut the rope between data and person. Anonymized borrower records we can set as "anonymized" and store them separately from "live" borrowers, because we want to see them as results from borrower searches. What do You think about this?
Michal
pá 22. 11. 2019 v 15:41 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
2 tables vs 1 table :)
Le ven. 22 nov. 2019 à 15:28, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, what options? I miss a point :-)
Michal
pá 22. 11. 2019 v 15:17 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
Thanks for the help Michal, What about the two options I have?
Le jeu. 21 nov. 2019 à 17:58, Mike D. <black23@gmail.com> a écrit : > > Hi Jonathan, > I’m volunteer for debate about processes and anon tools and methods. I’m ready to be tester of bugs. Koha is GDPR ready but some points could be improved for easier everyday usage in libraries. Because if something is clear and easy everybody do it without fear and stress. > > Thank You > > Michal > > čt 21. 11. 2019 v 17:14 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal: >> >> Hello everybody, >> >> I have been contracted by KohaLa to work on some GDPR requirements. >> The main idea is to "anonymize" patron's data but letting the library >> access the transactions' statistics. >> >> I am going to present you what I am planning to implement, in order to >> collect ideas and answers. >> >> There are the following steps I have in mind: >> 1. Pseudonymization [1] of patron's data >> 2. Improve deletion of patron related date (tables statistics, >> old_reserves, deletedborrowers) >> 3. Add the ability to remove data that have been pseudonymized >> >> I see 2 ways to achieve point 1: >> * We create 2 tables, 1 for the patrons, 1 for the transactions. >> - borrowers_anonymized will contain: hash_id, has_cardnumber, >> branchcode, creation_date, categorycode, bsort1, bsort2, >> [borrower_attributes] >> - transaction_anonymized will contain: hash_id, transaction_type, >> branchcode, itemnumber, holdingbranch, location, itemcallnumber, >> itemtype, timestamp >> >> hash_id will be generated using the borrowernumber and a key (that >> will be stored on the server, path in koha-conf) >> >> Pros: Easier to understand and manipulate as it follows existing structure. >> We track patron's modifications (this is the most important part) >> Cons: tech part: new config, a new path have to be created (minor) >> >> * We create only 1 table, (nosql-like). It will contain the same data >> as previously, without the hash_id >> >> Pros: No new config. Data are never updated and we have the values >> when the transactions has been processed. >> Cons: Data are not updated :) >> >> About borrower_attributes, the initial specification asks for 2 >> attributes defined in a syspref. I think it should be configurable, >> with a join table (Pro: more flexible, Con: SQL requests more complex) >> >> I think we should have the 2 tables and keep a link between the >> anonymized_patrons and anonymized_transactions tables. >> >> What do you think? >> I am going to start the implementation very soon in order to plan an >> integration early in the 20.05 dev cycle. >> >> Regards, >> Jonathan >> >> [1] https://en.wikipedia.org/wiki/Pseudonymization >> _______________________________________________ >> Koha mailing list http://koha-community.org >> Koha@lists.katipo.co.nz >> https://lists.katipo.co.nz/mailman/listinfo/koha
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
Hello Jonathan, I understand. You're right. So, can we document plan for new GDPR improvements on wiki? Next steps? Michal pá 22. 11. 2019 v 16:53 odesílatel Jonathan Druart < jonathan.druart@bugs.koha-community.org> napsal:
The anonymized_borrowers table will not contain all data from borrowers. Only a small part. Also we want to track down the transactions when they happen, it will be much more easier to implement and less error prone. How do you want to retrieve the transactions for a given patron when they are anonymized?
Also you want to collect statistics only on this table, it will be impossible to make statistics on this table for "anonymized" patrons, and another table for the ones that are not anonymized yet.
Does it make sense?
Le ven. 22 nov. 2019 à 16:33, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, sorry "we want to see them as results from borrower searches" shoub be
"we want to hide them (anonymized borrower records) from all results from searches"
Really we need to copy every patron into anonymized_borrowers at time of
creation of their records? I think that must be done only in case that we want to anonymize borrower. Not before this day. Because if we will copy all translation into new table we'll maka database bigger wi same data. Do I understand it clear?
Michal
pá 22. 11. 2019 v 16:08 odesílatel Jonathan Druart <
Michal,
Redirection the discussion back to the list.
What do you mean by "we want to see them as results from borrower
searches"?
What I understood is that libraries want to collect statistics on transactions even when the patron's records have been deleted. So: 1/ Create a patron (it's copied to borrowers and anonymized_borrowers) 2/ Do some transactions (it's in statistics and anonymized_transactions) 3/ Update a patron (it's updated in borrowers and anonymized_borrowers, if option 1 is picked) 4/ Delete the patron (moved from borrowers to deletedborrowers) 6/ Clean/purge the deletedborrowers => anonymized_* tables are still there, statistics still possible
7/ When needed, given parameters, anonymized_* entries will be deleted.
Le ven. 22 nov. 2019 à 15:51, Mike D. <black23@gmail.com> a écrit :
Hello, I think that thinks are maybe less complicated. I really like Your
idea with hash. But if we anonymize (replace by some generated hash or something similar) name, adress, telephone, e-mai and adress, noticess, we don't need related information about borrowers. Because we cut the rope between data and person. Anonymized borrower records we can set as "anonymized" and store them separately from "live" borrowers, because we want to see them as results from borrower searches. What do You think about
Michal
pá 22. 11. 2019 v 15:41 odesílatel Jonathan Druart <
jonathan.druart@bugs.koha-community.org> napsal:
2 tables vs 1 table :)
Le ven. 22 nov. 2019 à 15:28, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, what options? I miss a point :-)
Michal
pá 22. 11. 2019 v 15:17 odesílatel Jonathan Druart <
jonathan.druart@bugs.koha-community.org> napsal:
> > Thanks for the help Michal, > What about the two options I have? > > Le jeu. 21 nov. 2019 à 17:58, Mike D. <black23@gmail.com> a écrit : > > > > Hi Jonathan, > > I’m volunteer for debate about processes and anon tools and methods. I’m ready to be tester of bugs. Koha is GDPR ready but some points could be improved for easier everyday usage in libraries. Because if something is clear and easy everybody do it without fear and stress. > > > > Thank You > > > > Michal > > > > čt 21. 11. 2019 v 17:14 odesílatel Jonathan Druart < jonathan.druart@bugs.koha-community.org> napsal: > >> > >> Hello everybody, > >> > >> I have been contracted by KohaLa to work on some GDPR requirements. > >> The main idea is to "anonymize" patron's data but letting the
> >> access the transactions' statistics. > >> > >> I am going to present you what I am planning to implement, in order to > >> collect ideas and answers. > >> > >> There are the following steps I have in mind: > >> 1. Pseudonymization [1] of patron's data > >> 2. Improve deletion of patron related date (tables statistics, > >> old_reserves, deletedborrowers) > >> 3. Add the ability to remove data that have been pseudonymized > >> > >> I see 2 ways to achieve point 1: > >> * We create 2 tables, 1 for the patrons, 1 for the
> >> - borrowers_anonymized will contain: hash_id, has_cardnumber, > >> branchcode, creation_date, categorycode, bsort1, bsort2, > >> [borrower_attributes] > >> - transaction_anonymized will contain: hash_id,
> >> branchcode, itemnumber, holdingbranch, location, itemcallnumber, > >> itemtype, timestamp > >> > >> hash_id will be generated using the borrowernumber and a key (that > >> will be stored on the server, path in koha-conf) > >> > >> Pros: Easier to understand and manipulate as it follows existing structure. > >> We track patron's modifications (this is the most important
> >> Cons: tech part: new config, a new path have to be created (minor) > >> > >> * We create only 1 table, (nosql-like). It will contain the same data > >> as previously, without the hash_id > >> > >> Pros: No new config. Data are never updated and we have the values > >> when the transactions has been processed. > >> Cons: Data are not updated :) > >> > >> About borrower_attributes, the initial specification asks for 2 > >> attributes defined in a syspref. I think it should be configurable, > >> with a join table (Pro: more flexible, Con: SQL requests more complex) > >> > >> I think we should have the 2 tables and keep a link between the > >> anonymized_patrons and anonymized_transactions tables. > >> > >> What do you think? > >> I am going to start the implementation very soon in order to
jonathan.druart@bugs.koha-community.org> napsal: this? library transactions. transaction_type, part) plan an
> >> integration early in the 20.05 dev cycle. > >> > >> Regards, > >> Jonathan > >> > >> [1] https://en.wikipedia.org/wiki/Pseudonymization > >> _______________________________________________ > >> Koha mailing list http://koha-community.org > >> Koha@lists.katipo.co.nz > >> https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
Hello Michal, My next steps are the 3 ones I listed in my first post. I have already started to implement it and will open the bug reports in the next couple of days. Thanks for your interest :) Cheers, Jonathan Please note that Koha is the only ILS known to be powerful ;-) Le ven. 22 nov. 2019 à 21:06, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, I understand. You're right. So, can we document plan for new GDPR improvements on wiki? Next steps?
Michal
pá 22. 11. 2019 v 16:53 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
The anonymized_borrowers table will not contain all data from borrowers. Only a small part. Also we want to track down the transactions when they happen, it will be much more easier to implement and less error prone. How do you want to retrieve the transactions for a given patron when they are anonymized?
Also you want to collect statistics only on this table, it will be impossible to make statistics on this table for "anonymized" patrons, and another table for the ones that are not anonymized yet.
Does it make sense?
Le ven. 22 nov. 2019 à 16:33, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, sorry "we want to see them as results from borrower searches" shoub be "we want to hide them (anonymized borrower records) from all results from searches"
Really we need to copy every patron into anonymized_borrowers at time of creation of their records? I think that must be done only in case that we want to anonymize borrower. Not before this day. Because if we will copy all translation into new table we'll maka database bigger wi same data. Do I understand it clear?
Michal
pá 22. 11. 2019 v 16:08 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
Michal,
Redirection the discussion back to the list.
What do you mean by "we want to see them as results from borrower searches"?
What I understood is that libraries want to collect statistics on transactions even when the patron's records have been deleted. So: 1/ Create a patron (it's copied to borrowers and anonymized_borrowers) 2/ Do some transactions (it's in statistics and anonymized_transactions) 3/ Update a patron (it's updated in borrowers and anonymized_borrowers, if option 1 is picked) 4/ Delete the patron (moved from borrowers to deletedborrowers) 6/ Clean/purge the deletedborrowers => anonymized_* tables are still there, statistics still possible
7/ When needed, given parameters, anonymized_* entries will be deleted.
Le ven. 22 nov. 2019 à 15:51, Mike D. <black23@gmail.com> a écrit :
Hello, I think that thinks are maybe less complicated. I really like Your idea with hash. But if we anonymize (replace by some generated hash or something similar) name, adress, telephone, e-mai and adress, noticess, we don't need related information about borrowers. Because we cut the rope between data and person. Anonymized borrower records we can set as "anonymized" and store them separately from "live" borrowers, because we want to see them as results from borrower searches. What do You think about this?
Michal
pá 22. 11. 2019 v 15:41 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal:
2 tables vs 1 table :)
Le ven. 22 nov. 2019 à 15:28, Mike D. <black23@gmail.com> a écrit : > > Hello Jonathan, > what options? I miss a point :-) > > Michal > > pá 22. 11. 2019 v 15:17 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal: >> >> Thanks for the help Michal, >> What about the two options I have? >> >> Le jeu. 21 nov. 2019 à 17:58, Mike D. <black23@gmail.com> a écrit : >> > >> > Hi Jonathan, >> > I’m volunteer for debate about processes and anon tools and methods. I’m ready to be tester of bugs. Koha is GDPR ready but some points could be improved for easier everyday usage in libraries. Because if something is clear and easy everybody do it without fear and stress. >> > >> > Thank You >> > >> > Michal >> > >> > čt 21. 11. 2019 v 17:14 odesílatel Jonathan Druart <jonathan.druart@bugs.koha-community.org> napsal: >> >> >> >> Hello everybody, >> >> >> >> I have been contracted by KohaLa to work on some GDPR requirements. >> >> The main idea is to "anonymize" patron's data but letting the library >> >> access the transactions' statistics. >> >> >> >> I am going to present you what I am planning to implement, in order to >> >> collect ideas and answers. >> >> >> >> There are the following steps I have in mind: >> >> 1. Pseudonymization [1] of patron's data >> >> 2. Improve deletion of patron related date (tables statistics, >> >> old_reserves, deletedborrowers) >> >> 3. Add the ability to remove data that have been pseudonymized >> >> >> >> I see 2 ways to achieve point 1: >> >> * We create 2 tables, 1 for the patrons, 1 for the transactions. >> >> - borrowers_anonymized will contain: hash_id, has_cardnumber, >> >> branchcode, creation_date, categorycode, bsort1, bsort2, >> >> [borrower_attributes] >> >> - transaction_anonymized will contain: hash_id, transaction_type, >> >> branchcode, itemnumber, holdingbranch, location, itemcallnumber, >> >> itemtype, timestamp >> >> >> >> hash_id will be generated using the borrowernumber and a key (that >> >> will be stored on the server, path in koha-conf) >> >> >> >> Pros: Easier to understand and manipulate as it follows existing structure. >> >> We track patron's modifications (this is the most important part) >> >> Cons: tech part: new config, a new path have to be created (minor) >> >> >> >> * We create only 1 table, (nosql-like). It will contain the same data >> >> as previously, without the hash_id >> >> >> >> Pros: No new config. Data are never updated and we have the values >> >> when the transactions has been processed. >> >> Cons: Data are not updated :) >> >> >> >> About borrower_attributes, the initial specification asks for 2 >> >> attributes defined in a syspref. I think it should be configurable, >> >> with a join table (Pro: more flexible, Con: SQL requests more complex) >> >> >> >> I think we should have the 2 tables and keep a link between the >> >> anonymized_patrons and anonymized_transactions tables. >> >> >> >> What do you think? >> >> I am going to start the implementation very soon in order to plan an >> >> integration early in the 20.05 dev cycle. >> >> >> >> Regards, >> >> Jonathan >> >> >> >> [1] https://en.wikipedia.org/wiki/Pseudonymization >> >> _______________________________________________ >> >> Koha mailing list http://koha-community.org >> >> Koha@lists.katipo.co.nz >> >> https://lists.katipo.co.nz/mailman/listinfo/koha
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
Great, thanks! Can You add me to CC of bugs? I'm black23@gmail.com on Bugzilla. Thank You. Michal út 26. 11. 2019 v 21:22 odesílatel Jonathan Druart < jonathan.druart@bugs.koha-community.org> napsal:
Hello Michal,
My next steps are the 3 ones I listed in my first post. I have already started to implement it and will open the bug reports in the next couple of days. Thanks for your interest :)
Cheers, Jonathan Please note that Koha is the only ILS known to be powerful ;-)
Le ven. 22 nov. 2019 à 21:06, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, I understand. You're right. So, can we document plan for new GDPR
improvements on wiki? Next steps?
Michal
pá 22. 11. 2019 v 16:53 odesílatel Jonathan Druart <
The anonymized_borrowers table will not contain all data from borrowers. Only a small part. Also we want to track down the transactions when they happen, it will be much more easier to implement and less error prone. How do you want to retrieve the transactions for a given patron when they are anonymized?
Also you want to collect statistics only on this table, it will be impossible to make statistics on this table for "anonymized" patrons, and another table for the ones that are not anonymized yet.
Does it make sense?
Le ven. 22 nov. 2019 à 16:33, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, sorry "we want to see them as results from borrower searches" shoub
be "we want to hide them (anonymized borrower records) from all results from searches"
Really we need to copy every patron into anonymized_borrowers at time
of creation of their records? I think that must be done only in case that we want to anonymize borrower. Not before this day. Because if we will copy all translation into new table we'll maka database bigger wi same data. Do I understand it clear?
Michal
pá 22. 11. 2019 v 16:08 odesílatel Jonathan Druart <
jonathan.druart@bugs.koha-community.org> napsal:
Michal,
Redirection the discussion back to the list.
What do you mean by "we want to see them as results from borrower
searches"?
What I understood is that libraries want to collect statistics on transactions even when the patron's records have been deleted. So: 1/ Create a patron (it's copied to borrowers and
anonymized_borrowers)
2/ Do some transactions (it's in statistics and anonymized_transactions) 3/ Update a patron (it's updated in borrowers and anonymized_borrowers, if option 1 is picked) 4/ Delete the patron (moved from borrowers to deletedborrowers) 6/ Clean/purge the deletedborrowers => anonymized_* tables are still there, statistics still possible
7/ When needed, given parameters, anonymized_* entries will be deleted.
Le ven. 22 nov. 2019 à 15:51, Mike D. <black23@gmail.com> a écrit :
Hello, I think that thinks are maybe less complicated. I really like Your
idea with hash. But if we anonymize (replace by some generated hash or something similar) name, adress, telephone, e-mai and adress, noticess, we don't need related information about borrowers. Because we cut the rope between data and person. Anonymized borrower records we can set as "anonymized" and store them separately from "live" borrowers, because we want to see them as results from borrower searches. What do You think about
Michal
pá 22. 11. 2019 v 15:41 odesílatel Jonathan Druart <
jonathan.druart@bugs.koha-community.org> napsal:
> > 2 tables vs 1 table :) > > Le ven. 22 nov. 2019 à 15:28, Mike D. <black23@gmail.com> a écrit : > > > > Hello Jonathan, > > what options? I miss a point :-) > > > > Michal > > > > pá 22. 11. 2019 v 15:17 odesílatel Jonathan Druart < jonathan.druart@bugs.koha-community.org> napsal: > >> > >> Thanks for the help Michal, > >> What about the two options I have? > >> > >> Le jeu. 21 nov. 2019 à 17:58, Mike D. <black23@gmail.com> a écrit : > >> > > >> > Hi Jonathan, > >> > I’m volunteer for debate about processes and anon tools and methods. I’m ready to be tester of bugs. Koha is GDPR ready but some points could be improved for easier everyday usage in libraries. Because if something is clear and easy everybody do it without fear and stress. > >> > > >> > Thank You > >> > > >> > Michal > >> > > >> > čt 21. 11. 2019 v 17:14 odesílatel Jonathan Druart < jonathan.druart@bugs.koha-community.org> napsal: > >> >> > >> >> Hello everybody, > >> >> > >> >> I have been contracted by KohaLa to work on some GDPR requirements. > >> >> The main idea is to "anonymize" patron's data but letting
> >> >> access the transactions' statistics. > >> >> > >> >> I am going to present you what I am planning to implement, in order to > >> >> collect ideas and answers. > >> >> > >> >> There are the following steps I have in mind: > >> >> 1. Pseudonymization [1] of patron's data > >> >> 2. Improve deletion of patron related date (tables statistics, > >> >> old_reserves, deletedborrowers) > >> >> 3. Add the ability to remove data that have been
> >> >> > >> >> I see 2 ways to achieve point 1: > >> >> * We create 2 tables, 1 for the patrons, 1 for the
> >> >> - borrowers_anonymized will contain: hash_id, has_cardnumber, > >> >> branchcode, creation_date, categorycode, bsort1, bsort2, > >> >> [borrower_attributes] > >> >> - transaction_anonymized will contain: hash_id,
> >> >> branchcode, itemnumber, holdingbranch, location, itemcallnumber, > >> >> itemtype, timestamp > >> >> > >> >> hash_id will be generated using the borrowernumber and a key (that > >> >> will be stored on the server, path in koha-conf) > >> >> > >> >> Pros: Easier to understand and manipulate as it follows existing structure. > >> >> We track patron's modifications (this is the most important
> >> >> Cons: tech part: new config, a new path have to be created (minor) > >> >> > >> >> * We create only 1 table, (nosql-like). It will contain the same data > >> >> as previously, without the hash_id > >> >> > >> >> Pros: No new config. Data are never updated and we have the values > >> >> when the transactions has been processed. > >> >> Cons: Data are not updated :) > >> >> > >> >> About borrower_attributes, the initial specification asks for 2 > >> >> attributes defined in a syspref. I think it should be configurable, > >> >> with a join table (Pro: more flexible, Con: SQL requests more complex) > >> >> > >> >> I think we should have the 2 tables and keep a link between
jonathan.druart@bugs.koha-community.org> napsal: this? the library pseudonymized transactions. transaction_type, part) the
> >> >> anonymized_patrons and anonymized_transactions tables. > >> >> > >> >> What do you think? > >> >> I am going to start the implementation very soon in order to plan an > >> >> integration early in the 20.05 dev cycle. > >> >> > >> >> Regards, > >> >> Jonathan > >> >> > >> >> [1] https://en.wikipedia.org/wiki/Pseudonymization > >> >> _______________________________________________ > >> >> Koha mailing list http://koha-community.org > >> >> Koha@lists.katipo.co.nz > >> >> https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
*Honorable Friends * *ABCD Software is a power full Library Automation Software* *FOR PRESENTATION Please Google Search: Rasheed ABCD* *https://www.slideshare.net/Rasheed1976/presentation-by-rasheedahmedmarch2011 <https://www.slideshare.net/Rasheed1976/presentation-by-rasheedahmedmarch2011>* *ABCD* stands for "Automatización de Bibliotecas y Centros de Documentación" (Spanish), which means: Library and Documentation Centers Automation. Its development was promoted and coordinated by BIREME, with the support of VLIR. *ABCD* is web-based integrated library management software comprising the main basic library functions. This kind of library application is a long held aspiration for the ISIS community, since the first MS-DOS version came out more than 20 years ago. Several library automation systems were developed during this period and are still in operation worldwide. BIRME EMP previous system was limited to the circulation services. The main characteristics of ABCD are the coverage of the main library functions, its web centrality and its development and maintenance under the methodology of Free and Open Source Software. *Main functions* - Definition of any number of new databases (similar to Winisis), which includes: FDT, PFT, FST, and worksheets directly on the Web, or copying from existing ones either from the Web or from Winisis on a local hard disk, - Cataloguing of books and serials, independently of the format: MARC, LILACS, AGRIS, etc. - End-user searching (OPAC), - Loans circulation, - Acquisitions, - Library services like SDI, barcode printing, quality control, etc. - Compatible with CDS/ISIS database technology for the bibliographic databases, i.e. reading ISIS-databases and making use of ISIS Formatting Language for producing output and indexing of records; - Run on both Windows and Linux platforms; - Use of MARC-21 cataloging formats and other current standards or protocols (Dublin Core, METS, Z39.50...); - Published as Free and Open Source Software (FOSS) with the accompanying tools for the developer community; - Multi-lingual; *Rasheed Ahmed* On Fri, Nov 22, 2019 at 6:09 PM Jonathan Druart < jonathan.druart@bugs.koha-community.org> wrote:
Michal,
Redirection the discussion back to the list.
What do you mean by "we want to see them as results from borrower searches"?
What I understood is that libraries want to collect statistics on transactions even when the patron's records have been deleted. So: 1/ Create a patron (it's copied to borrowers and anonymized_borrowers) 2/ Do some transactions (it's in statistics and anonymized_transactions) 3/ Update a patron (it's updated in borrowers and anonymized_borrowers, if option 1 is picked) 4/ Delete the patron (moved from borrowers to deletedborrowers) 6/ Clean/purge the deletedborrowers => anonymized_* tables are still there, statistics still possible
7/ When needed, given parameters, anonymized_* entries will be deleted.
Le ven. 22 nov. 2019 à 15:51, Mike D. <black23@gmail.com> a écrit :
Hello, I think that thinks are maybe less complicated. I really like Your idea
with hash. But if we anonymize (replace by some generated hash or something similar) name, adress, telephone, e-mai and adress, noticess, we don't need related information about borrowers. Because we cut the rope between data and person. Anonymized borrower records we can set as "anonymized" and store them separately from "live" borrowers, because we want to see them as results from borrower searches. What do You think about this?
Michal
pá 22. 11. 2019 v 15:41 odesílatel Jonathan Druart <
2 tables vs 1 table :)
Le ven. 22 nov. 2019 à 15:28, Mike D. <black23@gmail.com> a écrit :
Hello Jonathan, what options? I miss a point :-)
Michal
pá 22. 11. 2019 v 15:17 odesílatel Jonathan Druart <
jonathan.druart@bugs.koha-community.org> napsal:
Thanks for the help Michal, What about the two options I have?
Le jeu. 21 nov. 2019 à 17:58, Mike D. <black23@gmail.com> a écrit :
Hi Jonathan, I’m volunteer for debate about processes and anon tools and
methods. I’m ready to be tester of bugs. Koha is GDPR ready but some points could be improved for easier everyday usage in libraries. Because if something is clear and easy everybody do it without fear and stress.
Thank You
Michal
čt 21. 11. 2019 v 17:14 odesílatel Jonathan Druart <
jonathan.druart@bugs.koha-community.org> napsal:
> > Hello everybody, > > I have been contracted by KohaLa to work on some GDPR requirements. > The main idea is to "anonymize" patron's data but letting the
jonathan.druart@bugs.koha-community.org> napsal: library
> access the transactions' statistics. > > I am going to present you what I am planning to implement, in order to > collect ideas and answers. > > There are the following steps I have in mind: > 1. Pseudonymization [1] of patron's data > 2. Improve deletion of patron related date (tables statistics, > old_reserves, deletedborrowers) > 3. Add the ability to remove data that have been pseudonymized > > I see 2 ways to achieve point 1: > * We create 2 tables, 1 for the patrons, 1 for the transactions. > - borrowers_anonymized will contain: hash_id, has_cardnumber, > branchcode, creation_date, categorycode, bsort1, bsort2, > [borrower_attributes] > - transaction_anonymized will contain: hash_id, transaction_type, > branchcode, itemnumber, holdingbranch, location, itemcallnumber, > itemtype, timestamp > > hash_id will be generated using the borrowernumber and a key (that > will be stored on the server, path in koha-conf) > > Pros: Easier to understand and manipulate as it follows existing structure. > We track patron's modifications (this is the most important part) > Cons: tech part: new config, a new path have to be created (minor) > > * We create only 1 table, (nosql-like). It will contain the same data > as previously, without the hash_id > > Pros: No new config. Data are never updated and we have the values > when the transactions has been processed. > Cons: Data are not updated :) > > About borrower_attributes, the initial specification asks for 2 > attributes defined in a syspref. I think it should be configurable, > with a join table (Pro: more flexible, Con: SQL requests more complex) > > I think we should have the 2 tables and keep a link between the > anonymized_patrons and anonymized_transactions tables. > > What do you think? > I am going to start the implementation very soon in order to plan an > integration early in the 20.05 dev cycle. > > Regards, > Jonathan > > [1] https://en.wikipedia.org/wiki/Pseudonymization > _______________________________________________ > Koha mailing list http://koha-community.org > Koha@lists.katipo.co.nz > https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
participants (3)
-
Jonathan Druart -
Mike D. -
Rasheed A.