Preserving history in obsolete digital formats
Digital Domesday Book lasts 15 years not 1000. This March 3, 2002 article illustrates the hazards of entrusting your digitized bits to a dying format. By December 2002, researchers successfully retrieved the bits. The Domesday book, created in 1086, is a record of the state of Britain at that time.
The digital archive, and how it got locked:
The special computers developed to play the 12in video discs of text, photographs, maps and archive footage of British life are - quite simply - obsolete.
As a result, no one can access the reams of project information - equivalent to several sets of encyclopaedias - that were assembled about the state of the nation in 1986. By contrast, the original Domesday Book - an inventory of eleventh-century England compiled in 1086 by Norman monks - is in fine condition in the Public Record Office, Kew, and can be accessed by anyone who can read and has the right credentials. ‘It is ironic, but the 15-year-old version is unreadable, while the ancient one is still perfectly usable,’ said computer expert Paul Wheatley. ‘We’re lucky Shakespeare didn’t write on an old PC.’ [Read More]
The above article discusses how all digital storage is in danger of becoming obsolete, and notes other examples where data—only decades old, not 900 years old—is lost.
How the Domesday Book got unlocked.
The team at Leeds University and the University of Michigan in the US say they have now found a way to access this rich digital archive.
They have developed software that emulates the obsolete Acorn Microcomputer system and the video disc player. [Read More]
This is why using standard formats for archiving media clips is so important.
For audio, WAV files are what we should be archiving today.
For photographs, the attractiveness of RAW camera files is alluring, I’ll admit to storing RAW files, but there may be issues down the road. Currently there are at least five applications - including from Microsoft and Adobe - that can read my Nikon RAW files. Otherwise, saving images in TIFF, lossless JPEG2000, or very-high resolution JPEGs if that’s how they came makes sense.
One of the challenges as I understand the British problem was that it was not only the raw data that was difficult to access, it was also the “flow” through the data—the metadata and programming—that was embedded in the original application. That represented a substantial portion of the effort. In that regard, we need to be careful about our Flash movies, PowerPoint slide shows, and especially the less-pervasive formats for making presentations.
Thanks for weighing in on WAV. Better n redbook audio CD? or is this a both/and vs. either/or? I recall your discussion of what kind of machinery will be around 100 years from now; probability for Audio CD is high, correct? My thoughts are, do both: WAV/AIF as data files, and AudioCDs as files in physical media.
Re: photos. Interesting points you make. I’m so much of a photoshop head that I store stuff in .psd formats. Haven’t yet jumped onto the RAW bandwagon. hi-rez jpeg, psd, tif: those are my main formats.
But the biggie that’s still in motion is digital video. Both software file format and hardware. It hurts my head just to consider the permutations. But because I’m going to be talking at Vloggercon next week, head-hurt I must.
In that regard, we need to be careful about our Flash movies, PowerPoint slide shows, and especially the less-pervasive formats for making presentations.
heh. I was just looking at Keynote… in preparation for the aforementioned speaking at Vloggercon. This Domesday entry is one research snippet for what I’ll be talking about. Digital video formatting—longterm, is an issue I wish to raise; I’ve no idea how it’ll go.
Red book CD is fine for standard 44,100 samples per second, 16 bit audio files. The archival storage “standard” for the moment is 96,000 samples per second and 24 bits, so that doesn’t fit on a red book CD. For individuals, red book CDs (standard audio CDs) are an excellent choice on gold-reflective-layer phthalocyanine dye CD-Rs, with at least two copies stored in different geological/geographical areas.
I think AIF should be fine - I apologize for my Windows-centric post.
As to PSD, that will work as long as Adobe PhotoShop is around. I save a bunch of stuff in PSD. Does GIMP read PSDs?
Motion video and film scans are best stored in a lossless format. JPEG2000/lossless is a good choice - and there is a motion-JPEG2000 version. There are people in the archival community pushing for that, but support is still small.
MPEG 3 video is widely used, and at 40-50 Mb/s looks very good. DV compression at 25 and 50 Mb/s is used as well (both for Standard Def TV). High Def and film storage consumes a LOT of space.
Remeber, HD TV baseband digital is 1.5 Gb/s!
35mm film is being scanned at 4K pixels wide, by whatever the height is for square pixels. Some maybe even higher.
Yes, everyone’s head hurts. I’ll let you in on a little secret. The predicitibility of the long-term stability of a given reel of tape (dedicated audio and video - data tape is better) is not very good.
Check out my tape aging section of my Blog at
There are some not-so-pretty pictures there.
PSD files and The Gimp. I’ll hafta check. Doc M (my significant other) has The Gimp on his linux box, but is in the midst of deadline hell, so I can’t check right away.
gold-reflective-layer phthalocyanine dye CD-Rs == Mitsui Gold/MAM-A, correct? (never mind, Google will help me figure it out) And it has . It is. The nice thing about the geographic dispersal is that if you distribute these among family members, you HAVE done geographic dispersal. I’ll put that on my To Do list, tho.. the geographic dispersal.
By the way, I’ve gotta do a review of archival pens to write on CD/DVD media. I wrote the Sharpie folks to ask what’s in their new “archival” pen—does it have the bad chemicals in it? The response I got back was, “Sorry Ma’am, that’s proprietary.” Which, to me, means, “Sorry, Sharpie, I cannot recommend you.”
Lossless video—zoiks!—here, I’m thinking about what you do with home and personal video converted to digital format. And how an individual stores that. Of which more later.
I’ve been marking CDs and DVDs with this for about a year and a half now…
It is probably safer than a pen and I use it BEFORE burning so if it has done any damage, the damage will most likely be detected during the burn.
An old thread on a timeless topic!
When I started thinking about interactive family history projects I eventually chose Adobe’s PDF format. At this point PDF is as close to a digital archival format as we have have. You could store everthing as text but that would be a bit of a stretch for your images, sound files and video files. With the right Acrobat reader, PDF files can be viewed and printed successfully from any computer platform anywhere. PDFs can include audio, video and hyperlinks. Adobe has gone to great lengths to insure backwards compantibility in PDF files and for the most part they’ve succeeded.
I’ve done several interactive family history projects in PDF format. You can download major sections of one project here:
Like everything else digital, even with a good archival format PDF files at the mercy of the storage medium, however.