[Koha] Koha indexing - files

[Chad Roseburg]

Thanks for the info!

Our servers have SSDs and plenty of RAM. So it sounds like possibly putting
the index files in tmp on a separate disk might add some performance
improvement ...though the SSDs by themselves will probably be the biggest
improvement.

On Thu, Feb 19, 2015 at 4:48 PM, Paul A <paul.a at navalmarinearchive.com>
wrote:

> At 01:26 PM 2/20/2015 +1300, Robin Sheat wrote:
>
>> Chad Roseburg schreef op do 19-02-2015 om 15:14 [-0800]:
>> > We are considering mounting the index files to a separate
>> high-performance
>> > disk array in hopes of improving performance during the indexing
>> process --
>> > has anyone experimented with something like this?
>>
>> It'd be worth seeing where the slowdown in the indexing actually is: is
>> the exporting the slow part, or the merging into the index?
>>
>> Though, I've never tried with zebra indicies on a high speed disk, I'd
>> be interested to see some real benchmarks.
>>
>
> Not absolute, but... our latest server uses SSD (raided 240 Gig Kingstons)
> rather than HDD (raided 500 Gig Seagates) with very slightly faster 8-core
> CPU, same 18 Gigs RAM; and the "read speed" (counting by 100's) of a total
> re-index is about twice as fast, with authorities faster than biblios.
> However, the "Cleaning" stage is disappointingly about the same speed,
> maybe a tad faster but barely perceptible. Memcached enabled on both boxes,
> same version of Zebra.  I get the *impression* that Zebra sorts (or does
> some other process) during the final write stage, otherwise we'd have seen
> an improvement.
>
> However, we rarely use the full re-index, having got the Cron job fully
> sorted every minute (including minor tweaking of
> biblio-zebra-indexdefs.xsl) -- and that is as near instantaneous as anyone
> could wish.
>
> Best -- Paul
>
>  Also, more RAM might be
>> useful for disk caching, and you could experiment with disk fsync()
>> commit options (i.e. by making it not require data to be written back
>> before returning.) This has risk though, but if you're careful, it could
>> well be acceptable, for example just needing to a full reindex in case
>> of a power outage.
>>
>> --
>> Robin Sheat
>> Catalyst IT Ltd.
>> ✆ +64 4 803 2204
>> GPG: 5FA7 4B49 1E4D CAA4 4C38  8505 77F5 B724 F871 3BDF
>>
>> _______________________________________________
>> Koha mailing list  http://koha-community.org
>> Koha at lists.katipo.co.nz
>> http://lists.katipo.co.nz/mailman/listinfo/koha
>>
>
> ---
> Maritime heritage and history, preservation and conservation,
> research and education through the written word and the arts.
> <http://NavalMarineArchive.com> and <http://UltraMarine.ca>
>
> _______________________________________________
> Koha mailing list  http://koha-community.org
> Koha at lists.katipo.co.nz
> http://lists.katipo.co.nz/mailman/listinfo/koha
>



-- 
Chad Roseburg
Automation Dept.
North Central Regional Library