Dan Elam <[log in to unmask]> wrote:
>Ok. I'll crawl out from under my rock and weigh in on this one - seeing as
>I've been vilified over the topic before.
No plans to vilify you, but I think some perspective might be needed here from an RIM, rather than a consultant. =) And seeing as I was one for 9 years, and have been an RIM for 32, I think I can differentiate between the two and see both sides of the issue.
>Can film last 500 years? Yes. Is it likely to do so? No. Probably
>everyone on this list has seen film that deteriorated. The argument that
>film is an analog source and better suited to long term 'preservation'
>isn't without it's merits, but it's a red herring.
Film, processed and stored properly will definitely outlast any digital media that currently exists. Period. If we've seen film deteriorate, it's been because of improper processing and storage, or storage in concert with other items that didn't belong in the same environment. Do we NEED it to last 500 years? I don't think anyone is calling for the "LE500" to necessarily be met, but meeting 50-100 years can be a beautiful thing for some static documents/records with extended retention requirements for a number of reasons.
1) If the digital "use copy" is damaged, destroyed or otherwise no longer available, you can always re-generate a "use copy" from a film based image... and typically cheaper, faster and of better quality than from the "source document", provided you retained it after the initial conversion.
2) Film, if appropriately indexed, can be searched and images are relatively quick and cheap to produce compared to re-generating from paper in the event the digital images stored on media have deteriorated and you hadn't been periodically checking the quality of the images.
3) If you're converting on a multi-purpose machine (such as a Kodak Archive Writer) it's simple and inexpensive to capture the image on film at the same time and the film can be stored as your Archival copy of the images, while the digital copy can be burned to CD, OD or whatever digital media form for regular use.
Film *IS* better suited to long-term preservation because of it's image permanence. With any form of media, without even giving consideration to the concerns about file formats and application obsolescence, you *WILL* have to periodically migrate the information to prevent loss due to media degradation and/or media obsolescence due to periodic hardware changes (as mentioned by Dan later in his post).
What's important here is to consider the data itself, and define "long term". When I think of long-term in a personal sense, I'm looking at 10 years or more... from a business perspective (in my current work situation)... I'm talking about 75 years and longer. If these are the types of numbers you're working with, and you're writing to CD-R or DVD-R media, and you're replacing/refreshing your data once every 5 years, you're going to go through 15 cycles. And the process is NOT "lossless". No one has come up with exact figures as to how much loss is acceptable, but it's not uncommon to experience 1-2% loss per migration... now, consider that over 15 migrations and it can get to be a bit problematic.
In addition, it isn't inexpensive to make the migrations, especially if you have large volumes of data. And again, we're not taking into account the possibility of needing to convert formats, deal with obsolete operating systems, applications, etc. over the 75 year time frame. And along with the migration, there's the QC effort that needs to be done.
At the recent ARMA Conference, a speaker from a media manufacturer gave a presentation and stated that he burns two copies of his records simultaneously, ensures they are on media from different batches, performs a QC verification exercise to ensure the data is identical on the two copies and then stores on in a remote, environmentally sound location... and he replaces critical information every 2-3 years, and goes through the same steps when he does. AND HE WORKS FOR SOMEONE THAT MAKES THE MEDIA!!!
>1. Analog vs. Digital doesn't matter. I know it's heresy to the pro-film
>world, but the reality is that more information will be produced on digital
>in 2004 than all the analog in recorded history. The fact is that digital
>is here to stay and that vast amounts of information of critical
>information are stored only in a digital form. If a nuclear bomb goes off
>in NY, all of the financial records like bank accounts and mutual fund
>holdings aren't going to be restored from an analog source - they are going
>to be restored from digital back-ups of the databases. Furthermore, if a
>company did try to restore the insane amounts of information from analog,
>it would take too long and pose additional challenges (e.g., run on the
>banks). The bottom line is that, as a society, we have committed to
>digital data. The issue is no longer whether digital is the right
>media: the question should be how to do we best manage and preserve the
>digital information.
While you're correct that the volume of "born digital" information is increasing exponentially and little if any of transactional data ever has a "paper life", there are many other classes of information that exist. Most of the financial data you've mentioned has a required retention of 7 years or less, some of it that pertains to partnerships and corporations has a maximum potential retention of up to 25 years, but it doesn't matter because the IT Administrators who are managing this information are completely unaware of the retention requirements and in many cases aren't even ensuring it's being managed appropriately for the periods their organizations are REQUIRED to retain it.
Lots of the disaster recovery plans and business continuity plans being developed for IT shops are more concerned with the "live data" than the legacy/historical data... all you need to do is look at an IT Administrator's definition of "archived data" and you'll find this out. In their world, you archive things based on the use pattern of the data, not on the value of the information or the regulatory or other requirement to retain it. But, I digress....
>2. Eye-Readable is a poor metric. The idea of eye-readable isn't really
>important unless we're talking about a return to middle ages where we are
>assuming that we lose our digital technology. If we really get to that
>point, I would suggest that having a microfilm copy of your FileNET
>software maintenance contract isn't all that important. Digital use
>technology isn't any different from the optical magnification technology
>used to read a piece of film (oh wait, it is - digital technology has
>advanced much more than optical magnification technology). If you don't
>have optical magnification technology you aren't going to be sitting around
>squinting into the sun so you read the tiny little frame of film. (Boy,
>what a visual!)
What metric would you prefer? Basically, from the RIM perspective, the primary concern is being able to have access to the required information at the time it's needed. If the metric is how quick a disk spins up and the image can be portrayed and the situation is you're a Public Utility in an area that has been hit by a massive hurricane and the power is out... I think eye-readable is a pretty good metric. I recall a classic situation with one of the Southern Utilities who had used a system of paper-based "plat map books" for years for their transmission and distribution trucks for troubleshooting and had converted to a digital system to save money, then when a hurricane struck, they couldn't gain access to the images and generate maps for the crews to determine the routing from substations and transformers to the subdivisions and had to send the source data out of state to have the books re-generated before the crews could begin their work. As I remember, the solution a!
fter this
incident was to generate microfiche for the crews to have along with a battery powered hand reader with a small screen in the event of emergencies.
I think in this instance, eye-readable was a pretty decent metric, and the visual of power company crews huddling around a microfiche reader screen is more powerful than them sitting in trucks unable to assist their customers.
>3. Media strategy is important - media itself is not. CDs
>deteriorate. They delaminate. They lose bits. Sometimes they lose the
>entire CD. Like film, the deterioration process is slow until it reaches a
>certain point and then the loss of data accelerates. The smart thing to do
>is to examine your media and *know* that you will have to convert from time
>to time. Even if the media survives, you have to factor in whether you
>have hardware and software drivers to read the data. (Quick show off
>hands: how many have a 5.25" drive on the computer they are using right
>now? No. Ok, how many have a 3.5" floppy on the same machine?) We
>generally advise people to review their media, drives, data formats, and
>software drivers every 5 years. If the decision isn't made to upgrade at
>that point, review every three years until the data is
>converted. Preservation or even long term storage does require planning as
>a part of the media strategy.
On this issue, I'm in agreement with you. Not specifically on the stated timetables for reviewing media, hardware and application based data, but that the key to a preservation strategy is planning and active review of your data. I disagree somewhat that the media isn't important. As a part of developing a preservation strategy, one needs to consider the media used to store records, data and information on and the selection of a media should be consistent with the required retention period for the information. CDs and DVDs may be plenty acceptable for transitory data that can be discarded in 2-5 years, but if you have long-term data, they aren't an acceptable alternative, unless you have a large budget for conversion and migration, or a very limited volume of data.
>4. Film costs more in the long run. Like it or not, cost is an important
>consideration (if it wasn't, people would be storing the paper records -
>not film - in fire proof, hermetically-sealed vaults). The difference in
>costs for film vs. digital is pretty substantial. So substantial that the
>interest on the difference in capital easily pays for the inevitable media
>conversions.
Well, we could debate this for a long time, too. Film is less expensive than conversions to retain viable file formats and 15 migrations and bit-parity checks of a massive volume of records over a 75 year time frame. And paper is a bit more difficult to index and review than film (not to speak of storing the equivalent volume of information on film versus paper), which is why people use it as an alternative to paper for long-term storage of analog images.
And while you may think that the "interest on the difference in capital easily pays for the inevitable media conversions" it DOESN'T pay for the loss of information that can potentially occur during the multiple conversions and the even more simplistic process of migrations when they have to take place repeatedly over a long time frame. It's pretty difficult to budget for replacing lost data when you can't easily identify it or locate the source documents.
>The bottom line is that there is no reasonable economic or scientific
>reason to produce new film today. Demonstrations such as putting video on
>film just strike me as hilarious: who thinks it's a novel idea to put
>analog video on film? Thomas Edison? Going to put digital video on
>film? Then we are back to the same issues with file formats and being able
>to decipher and play it back: all with a cost of more than 50x what it
>would have been to store digitally in the first place.
Now here's something I like... a "bottom line" consultant! =) I think I've provided some reasonable economic considerations for utilizing film as a media form for long-term preservation even in today's paradigm. They may not be "scientific" but they're logical when you're considering film as a medium for archival storage an re-generating digital media from if you have to ensure it's available in the same format for 50, 75 or even more years into the future.
I also think it's safe to say that even with the work NARA is doing on the Electronic Records Archives (ERA) Project and the studies they're doing with the San Diego Supercomputer Center (SDSC) they aren't abandoning the concept of using film as an archival media for images of sensitive and vital records. I don't think we'll see an electronic image of the Constitution or Bill of Rights being the "go-to" format in the near future. =)
>Crawling back under my rock.
No need to crawl... walk PROUDLY.
Larry
List archives at http://lists.ufl.edu/archives/recmgmt-l.html
Contact [log in to unmask] for assistance
|
