[Koha] 008 Fixed length data elements: problems with imported records in 3.12

[Elaine Bradtke]

Something is happening to our 008 fields, when we edit the 008 field
of an imported record (converted from our old database - this doesn't
seem to be a problem with Z39.50 downloads).  I don't know if it's
always been a problem, or if it's just come up since we moved to 3.12.
Our cataloguer only recently reported it , but we don't often need to
edit 008 fields.

Here's an example of the imported 008:
130515s2006####stka###gr#####000#0#eng#d

If I edit the record and open the 008 editing window  Koha
automatically changes it to:
130515s2006

And this is what it looks like after clearing the field and entering
the information manually:
131030s2006    stka   gr     000 0 eng d

The problem seems to be with the #  characters in the imported 008.
Koha slots the information it has into the spaces, working from left
to right.  When it gets to the first # something happens.  It skips
over the # character(s) and continues to slot in information,
redistributing the other characters in the remaining spaces from left
to right.

Once the data is redistributed, if the information in a given position
doesn't match what is allowed in that position then Koha uses the
default (usually blank which is what happened in the above example).
If a character does match what's allowed, then Koha uses it. . . which
can make for some bizarre entries because  'a' in one position means
something very different in another.

But it also seems to hold on to these incorrect characters, even when
you think you have changed them using the editing window. While the
field editor displays the corrected information, after you click OK
and return to the MARC display,  the characters are still in the wrong
positions.

In order to properly correct the faulty 008 field, I have to manually
delete all the incorrect information  from the main MARC display
(rather than using the field editor) and then enter the correct
information either in the MARC display or using the field editor.

I think I need to file a bug report . . .? or is it two bugs? - 1) the
# character and 2) the field editor not correctly updating the
information???

Aside from the quirks described above, I have a couple questions:

How important is the 008 field  in the long term? I know it does
influence a few display and search functions in Koha.
Is it  worth the effort to export the whole catalogue and try to fix
this with batch edits?
How do I avoid this problem with the next set of uploads?

-- 
Elaine Bradtke
Data Wrangler
VWML
English Folk Dance and Song Society | http://www.efdss.org
Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY
Tel    +44 (0) 20 7485 2206 (This number is for the English Folk Dance
and Song Society in London, England. If you wish to phone me
personally, send an e-mail first. I work off site)
--------------------------------------------------------------------------
Registered Company No. 297142
Charity Registered in England and Wales No. 305999
---------------------------------------------------------------------------
"Writing about music is like dancing about architecture"
--Elvis Costello (Musician magazine No. 60 (October 1983), p. 52)