I think both Dan and Larry make good points. The error rate here is not
simply the accuracy of the bitstream, which for all intents is perfect, as
Dan notes. The error rate is compounded by user error, particularly when
transferring from older, perhaps deprecated media types to newer ones and
from proprietary to other types. For example, someone attempting to migrate
500 floppy disks to a single CD may lose more information, all else being
equal, than someone doing a CD-to-CD media refresh. Similarly, someone
converting their data from an old FileMagic or Paradox-based solution on CDs
or 200MB Zip drives to a current solution built on top of MS SQL Server
2000, for which the images are then written out to DVDs, there is a
significantly greater potential for data loss due to user error and
potential data incompatibilities. In the aggregate this might approach 1-2%.

One previous point that Dan and Larry made, regarding TCO for film vs.
digital. Cost to film can be significantly higher because the amount of data
is so much higher. An organization of 1000 users may receive as much as
200GB+ of email a month (Forrester, Gartner). If even 1% of that is
important enough to keep for more than 10 years, that's still 2GB of
messages that would have to be printed or filmed, or perhaps 100,000
messages per month at 20KB per (Osterman). That's a lot of printing or
filming, a lot of stuff to store, and if you want to then make a use copy
from the master copy the volume doubles. It's just not feasible.

Larry's points on electronic migration were spot-on, as well. It is
expensive, for the reasons listed in my first paragraph as well as the fact
that it has to be done every 5-10 years. It's not the media that goes bad,
it's the hardware and especially the software.

So what do we do? We as end users, consultants, and even vendors need to
start pressuring the software vendors for backwards compatibility and
standards-based interoperability (XML anyone?), the hardware vendors for
backwards compatibility (see Blu-Ray type technologies such as Sony PDD and
Plasmon UDO vs. red-laser based CD and DVD), and the media vendors to make
sure their media is high-quality.

I don't know if there is a place for microfilm that cannot be readily filled
by digital, subject to the limitations above (and they are significant). I
think film remains a comfortable solution for organizations that want to do
archival, but I don't believe most Fortune 500 or Global 2000 companies are
seriously looking at investing in film these days because of the sheer
volumes of information involved. Happy to hear opposing view....

My .02 on a beautiful Colorado morning,

Jesse Wilkins
CDIA+, EDP, LIT/ERM, ICP
Principal
IMERGE Consulting
[log in to unmask]


>From: Dan Elam <[log in to unmask]>
>Reply-To: Records Management Program <[log in to unmask]>
>To: [log in to unmask]
>Subject: Re: Microfilm Alternatives
>Date: Fri, 15 Oct 2004 04:51:00 -0400
>
>>types of numbers you're working with, and you're writing to CD-R or DVD-R
>>media, and you're replacing/refreshing your data once every 5 years,
>>you're going to go through 15 cycles.  And the process is NOT
>>"lossless".  No one has come up with exact figures as to how much loss is
>>acceptable, but it's not uncommon to experience 1-2% loss per migration...
>
>
>Larry, I'll put together the economic analysis for you in a week or two,
>but in the meantime, I couldn't let this go unchallenged (despite your
>other good points.)  It is inaccurate to state that digital conversion is
>not lossless and grossly inaccurate to say that there is a "1-2% loss per
>migration".
>
>If you use CD's  - which are less reliable than a standard magnetic disk
>but more reliable than non-DAT magnetic tape over short terms - there are
>two types of redundancy built into the CD.  I won't bore you with the math
>and details of how it works, but the *uncorrected* bit error rate is about
>1 in 10^(-12) or one per trillion bits.  An 8.5x11" 200 dpi image should
>have 3.74M pixels (raw).  Some of our older research suggests than only
>about 29% of the image contains "significant" information (i.e.,
>information used for actual reader interpretation).  The numbers may have
>changed a bit, but it's probably still in the ballpark.  Let's be generous
>and say that 0.1% of the pixels could change an interpretation.  The math
>works out pretty even so that the changes of a single meaningful pixel
>change are just under 1 in 10 trillion.  (Want some perspective?  The
>estimate for number of stars in the universe is 10^(11) or ten times less
>than the error rate.)  Hardly a 1-2% loss.
>
>List archives at http://lists.ufl.edu/archives/recmgmt-l.html
>Contact [log in to unmask] for assistance

List archives at http://lists.ufl.edu/archives/recmgmt-l.html
Contact [log in to unmask] for assistance