[Koha] Koha indexing - files

[Paul A]

At 01:26 PM 2/20/2015 +1300, Robin Sheat wrote:
>Chad Roseburg schreef op do 19-02-2015 om 15:14 [-0800]:
> > We are considering mounting the index files to a separate high-performance
> > disk array in hopes of improving performance during the indexing process --
> > has anyone experimented with something like this?
>
>It'd be worth seeing where the slowdown in the indexing actually is: is
>the exporting the slow part, or the merging into the index?
>
>Though, I've never tried with zebra indicies on a high speed disk, I'd
>be interested to see some real benchmarks.

Not absolute, but... our latest server uses SSD (raided 240 Gig Kingstons) 
rather than HDD (raided 500 Gig Seagates) with very slightly faster 8-core 
CPU, same 18 Gigs RAM; and the "read speed" (counting by 100's) of a total 
re-index is about twice as fast, with authorities faster than biblios. 
However, the "Cleaning" stage is disappointingly about the same speed, 
maybe a tad faster but barely perceptible. Memcached enabled on both boxes, 
same version of Zebra.  I get the *impression* that Zebra sorts (or does 
some other process) during the final write stage, otherwise we'd have seen 
an improvement.

However, we rarely use the full re-index, having got the Cron job fully 
sorted every minute (including minor tweaking of 
biblio-zebra-indexdefs.xsl) -- and that is as near instantaneous as anyone 
could wish.

Best -- Paul

>Also, more RAM might be
>useful for disk caching, and you could experiment with disk fsync()
>commit options (i.e. by making it not require data to be written back
>before returning.) This has risk though, but if you're careful, it could
>well be acceptable, for example just needing to a full reindex in case
>of a power outage.
>
>--
>Robin Sheat
>Catalyst IT Ltd.
>✆ +64 4 803 2204
>GPG: 5FA7 4B49 1E4D CAA4 4C38  8505 77F5 B724 F871 3BDF
>
>_______________________________________________
>Koha mailing list  http://koha-community.org
>Koha at lists.katipo.co.nz
>http://lists.katipo.co.nz/mailman/listinfo/koha

---
Maritime heritage and history, preservation and conservation,
research and education through the written word and the arts.
<http://NavalMarineArchive.com> and <http://UltraMarine.ca>