Hi Jesse, first: of course you can join koha-devel and you are welcome there :) I am not aware of any conscious/intentional change in Koha and I know that we've always been importing and exporting records with combined diacritics (NFC) into Koha to avoid display and editing issues (German umlauts etc.) When you edit the records in Koha, do they present with NFC or NFD? (you can tell by removing a diacritic, does it require one or two steps?) How were the records added to Koha? Can you replicate the behaviour on another installation for example on a sandbox? Hope this helps, Katrin On 03.09.23 05:11, Jesse Savage wrote:
This is probably a question for a developer, but I'm not one, so don't really want to join the koha-devel list (and they probably wouldn't want me to), and I know a lot of developers frequent this list. My apologies to anyone who might not be interested in this question.
I recently got a new Windows laptop on which I installed (I think) a newer version of WSL/Ubuntu, and I see that records exported with the "MARC (Unicode/UTF-8)" option (as *utf8 files, which look to be basically *mrc) apparently use NFD (or "decomposed") encodings for Unicode characters (in my case, mainly Spanish and French titles) and thus don't display properly in *less*, *grep*, or *cat* (the diacritic follows the standard-Latin character, rather than integrated with it) as do files encoded with NFC characters, and the characters also can't be *grep*'d with searches like "grep $'\u16A0'." The program I use to update records for uploading also outputs NFD, even if the records it takes for input contain NFC. (I'm waiting on the answer to the present questions to see whether that program has an option hidden somewhere to output NFC instead, which I'd prefer.)
So, is this a systematic change in koha? *Should I ensure that *.mrc files for batch uploading ALWAYS include only NFD characters, or do the underlying processes standardize NFC vs. NFD?* The system on which the catalog lives (but which I don't administer) currently has koha 23.05.00.000 and runs on "SMP Debian 5.10.179-1 (2023-05-12) x86_64". Please feel free to respond directly if you feel the answer won't interest anybody else.
Thanks very much in advance! Jesse --------------------- Jesse Savage (pronouns he, him, his) jessava@gmail.com _______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha