[Koha] Integrating Vision AI / GPT for cataloging in Koha

[Karam Qubsi]

Hi Victor,


>  The idea could become a plugin?


I think they way it looks now ( in the demo)  can indeed be inside Koha as
a plugin, but if we look at the bigger picture of AI integration with
library services, and the need of vector stores to build RAG based systems,
for that benefits I found it more efficient to build this as a
separate tool and integrate with Koha through Rest API ,

The main limitation here is Mysql /MariaDB with Vector stores, I'm not
aware of any good stable implementation of vector stores in any of those 2
DBs servers, or at least I'm not familiar with that if it exists.

Vector stores can help us build  things like ( Which I think I will build
in later phases if librarians found this useful )

   - Chatbot, for both librarians and patrons, can be aware of library
   collections and rules. and what is available now ( live ) in the library
   and what is checked out. ( we should have a plugin here to update vector DB
   with every update of bib or item status )
   - Smart Search all library content in any language, you can ask the AI
   search assistant to find something using your own words / language, and
   because of storing data as vector embeddings, your query will be converted
   into an embeddings first, then matched with library content, and you will
   get results of everything related even if you didn't use the words used in
   that language in your search!. but it is relevant to your search term.
   - Authority control ( authors, subjects ..) , if these records are
   stored as vector embeddings, then the cataloger ( or the AI cataloger)  can
   easily find all relevant controlled authority records related to the
   material being cataloged now in library catalog history, so selection of
   new subject heading will be more consistent.



On Thu, Sep 19, 2024 at 10:44 PM Victor Barroso Oliveira <
vbovictor at gmail.com> wrote:

> The idea could become a plugin?
>
> Em qui., 19 de set. de 2024 às 11:32, Karam Qubsi <karamqubsi at gmail.com>
> escreveu:
>
>> Hi Victor,
>>
>> The process is like this:
>>
>>    1. Cataloger will upload a group of images for the book.
>>    2. in backend MarcAI ( what I call it for now) will use some
>>    predefined prompts along with the provided images as prompts.
>>    3. All those prompts are submitted to either OpenAI  or Google
>>    Gemini. For Arabic books, I use both together, but for English and
>>    Latin-based languages, I think OpenAI alone is sufficient.
>>    4. The responses from the AI models are restricted to follow some
>>    JSON schema defined in MarcAI.
>>    5. AI Models will reply following the defined json schema, which will
>>    be a valid Marc21 record in custom json format.
>>    6. then back to UI, cataloger now can see and edit the marc21 record
>>    if needed.
>>    7. Cataloger can click add to Koha, which uses Koha RestAPI with
>>    OAuth2.0 . ( this will require cataloger to fill in client_id and
>>    client_secret, only one time)
>>    8. Book is added to Koha, ( if duplicate the AddBiblio endpoint will
>>    return an error of duplicate title with bib id )
>>
>>
>> Actually this is the current flow, few librarians friends suggested other
>> interesting idea, where they can scan ISBN using barcode scanner, ( so a
>> 100 books can be scanned in minutes ) or upload a file of ISBNs, and then
>> MarcAI will do web search, and group data from z39.50 or other sources,
>> then we pass that to AI model, and generate marc21 records based on all
>> those information, I will try to explore this scenario next weekend.
>>
>> Thanks for asking.
>> Karam.
>>
>>
>> On Thu, Sep 19, 2024 at 9:48 PM Victor Barroso Oliveira <
>> vbovictor at gmail.com> wrote:
>>
>>> Could you detail the process?
>>>
>>> Em qui., 19 de set. de 2024 às 10:10, Karam Qubsi <karamqubsi at gmail.com>
>>> escreveu:
>>>
>>>> Hello Koha community,
>>>>
>>>> I’ve been working on a side project experimenting with AI, and I’ve
>>>> built
>>>> something that looks interesting to me. I’d love to see if other
>>>> members of
>>>> the Koha community are interested in something similar.
>>>>
>>>> You can watch this 35-second demo I recorded:
>>>>
>>>> https://youtu.be/vnvxmjTUmGQ?si=-YHqLoyp0VeqRL6L
>>>>
>>>> If you have any questions, feel free to ask.
>>>>
>>>> Thanks!
>>>>
>>>> --
>>>> *Karam Qubsi*
>>>> _______________________________________________
>>>>
>>>> Koha mailing list  http://koha-community.org
>>>> Koha at lists.katipo.co.nz
>>>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>>>
>>>
>>
>> --
>> *Karam Qubsi*
>>
>>

-- 
​*Karam Qubsi*