Thanks very much for your well-though out and reasoned response, Gerry.  And
thanks for the appropriately placed "grins" =)

Given it was diazo being replaced that were copies in normal use and the
silver masters were retained, I agree this was a logical decision.  Maybe
the only thing I would have done differently was to consider scanning from
the silver masters to get a bit better image

As to a couple of your other specific responses:.



On 1/29/07, Gerry McFatridge <[log in to unmask]> wrote:
>
>
> >How is budgeting being handled to pay for the cost of periodic
> migration and validation
> >of the digital images over time to meet retention?
>
> Good question, and I don't have an answer on that but let me give a
> couple of comments:
>
> We are operating on an enterprise document imaging/management system so
> our costs are based upon the fees charged by our IT folks so I can't say
> that is broken down as a constituent of that annual fee. I'll have to
> ask about that and see what I can find out.
>
> As far as migration I don't see that any future migration of our image
> files is likely to be a real concern. Certainly, I do not buy into the
> "migration of digital images every 3-5 years". I'm not sure I understand
> the basis for such a statement.



I think this *IS* specifically related to consideration of the use of native
file formats or proprietary formats AND storage of data on removable media,
such as CDs, DVDs, diskettes, optical platters, and some forms of magnetic
tape.

We use TIFF files and I expect the TIFF format to live forever + 10
> years <grin>. If we were using some vendors proprietary file format that
> would worry me but as we are using a widespread international standard
> file format I sleep ok. I assume that at some point there may be the
> development of a digital format that offers many advantages over TIFF
> and if/when it becomes an internationally accepted standard there may be
> a desire to convert existing data to that - at that time I would expect
> that conversion from TIFF to whatever would be a minor endeavor and a
> minor budgetary expense. Of course my vision of the future could be very
> wrong and not only will we not all be driving hover cars and viewing
> files on holographic displays but such file conversion could cost an arm
> and a leg.



Yeah, well... Lots of people put stock in TIFF and even PDF or more to the
point PDF/A, but the jury is still out as to how "open source" Adobe wants
to make either.   And if Adobe continues to maintain control of it over
time.  I know the intent of PDF/A is that source code is being held "in
escrow", but there have been some articles recently on problems with Adobe
formats:

http://www.computerworld.com/blogs/node/4273
http://www.publish.com/article2/0,1895,2082262,00.asp

...but Adobe pretty quickly addressed these and they seem to actively do
so.

While I'm a supporter of the Open Source movement, I think we've also seen
how some in the market place try to exploit even the best of intentions...

http://business.bostonherald.com/technologyNews/view.bg?articleid=179372
"Under threat of further EU action, Microsoft in October said it had made
changes to Vista's search service and security system and had an
international standards organization look at its new PDF-type file
format."

(GEEZ... can't these M$ guys develop ANYTHING on their own?!?!?)  =^P


Validation could be a concern. Bit rot does occasionally happen - as to
> whether that would completely corrupt an image file or just cause random
> pixel dropin/dropouts I do not know.  In any case we will still have the
> archival silver for microfilmed items and stored paper for
> non-microfilmed items. I can't honestly say that we (does anyone?) have
> a process in place to view/verify every digital image once every xx
> years. I do not know if a routine process of running checksum
> validations during the wee hours of the morning to try and identify any
> corrupted files would be a worthwile effort (any comments anyone has on
> that matter would be appreciated!!)



You're completely right here, there is a limit to the parameters everyone
has to set as to how accurate you have to be, but then again, if you make a
decision to convert something that you KNOW is accurate to begin with and
then don't validate the accuracy of it once you convert it (especially when
you're a public entity) you KNOW you'll hear about it later!  Don't forget
the decision regarding the US Census, I believe it was in 2000, where the
data was gathered on Scantron-type sheets, scanned and collected in a
database and a decision was made to NOT microfilm the sheets and some of the
data was corrupted?

I would say that validation of film stuck away in some archival storage
> facility may be a concern also. Does anyone periodically validate their
> decades old stored away microfilm?



When I used to work for a public utility, we DEFINITELY did. Each year when
we took new film to deep storage, we randomly sampled film being stored.  We
had a DOT requirement for pipeline and other transmission records to be
stored for 75 years beyond destruction of a facility, and we diligently
validated the quality of the film, along with an eyeball check of the
images.

I know we ASSUME that under proper
> storage conditions film will not deteriorate for a good long while but
> you never know when some file warehouse worker might be a little
> careless with their lunchtime PB&J and drop some apricot preserves into
> a box of film.



=) Yeah, a great image...  But we had pretty strict policies of who was
allowed access and under what conditions, and we had logs that were verified
and validated for access controls.

Plus, I doubt that most people store their film or other
> physical files in truly state of the art climate controlled, chemical
> and vermin free environments.



You'd be surprised how many do... and I'm surprised even more at how many
don't.  When I was a Consultant, specializing in vital records program
development, disaster preparedness and recovery, I wrote many a finding on
inadequate storage and was surprised at how many organizations were willing
to accept the risk over the cost of remediation to correct inadequate
conditions... but then again, how many people continue store their
information assets with "...the global leader..." ??   =)

Is it really hard to imagine any government entity doing something
> without the benefit of a proper analysis beforehand <grin>?



Yeah, that really WAS A "LEAP OF FAITH" comment wasn't it?  And can you
believe I made it without a NET??? =)

>Given the need for computer equipment for the content to reside on,
> >the communications requirements for transmitting rather large files,
> >the processing power to search the repository, the displays to read
> >the converted images, the time to index the content when converted,
> >the QC for the images and the converted images on each generation over
> >the future, and the cost to upgrade systems to avoid obsolescence and
> media degradation...
> >at minimum, I would think retaining the film as a backup would have
> been cost effective.
>
> All of our staff already have desktop pc's supported by a netwoking
> infrastructure designed to handle many varied business apps. As we were
> already implementing a document imaging/management system the desire to
> have all our files available electronically so they could be readily
> accessed by folks in various locations made the cost worthwile.



If the hardware and communications network were already deployed in all
locations, then I have to agree it was simply an incremental cost for more
storage... and maybe a bit more processing power. But there had to be a base
cost for the scanning/imaging, indexing and QC and depending on the volume,
it must have been rather significant. And some conversion/migration WILL
happen over time, even with a disk based system (RAID, SAN, NAS or whatever)

Again,
> in my case what we disposed of were only diazo copies of the silver
> masters so in effect we do have that backup.



And this was my main point, and you've clearly stated a couple of time that
it was done.

Thank you for your comments and questions Larry. Unlike most people on
> the list I am not a records management professional. I only became
> involved with my departments records management function (apart from
> being a user of our files for many decades) about a year ago. I am
> primarily a "computer guy" who was re-assigned to help get our document
> imaging program back on track and provide some tech support/supervision
> for our departmental records management function so I have been having
> to learn and deal with a lot of things outside of my normal pc geek
> realm. The RECMGMT list has been quite useful in that regard.



And again, thank you for taking the time to provide a thorough response.
For a "computer guy", you're alright!!  =)  I'm happy to have folks like you
drinking at the same trough of knowledge as us RIM types....

Larry

-- 
Larry Medina
Danville, CA
RIM Professional since 1972

List archives at http://lists.ufl.edu/archives/recmgmt-l.html
Contact [log in to unmask] for assistance