Monday, 23 September 2019

backup - What medium should be used for long term, high volume, data storage (archival)?


This question was inspired by https://superuser.com/questions/374386/how-to-store-and-preserve-lots-of-data. There have been other similar questions, but none with the same criteria.


This is two questions in one.



  1. How do you store financial/critical records that should survive anything but a fire and should be available for decades?

  2. Lets say I want to store family photos/videos and want people do be able to find them in storage 100 years from now and still be able to use them. How would this be done?


Criteria



  1. Long term means 30+ years guaranteed. 100+ years average. [If this is not practical, use the closest solution]

  2. High volume means a couple terabytes.

  3. Answers can be 'no-compromise/industrial' solutions or practical solutions for the home office/small business user.

  4. Media will not be active during the timespan. (i.e., if you suggest hard drives, they will not be spinning).

  5. Further, there is no expectation of needing to read these archives. They are there for emergency or "for future generations" purposes.

  6. Should not require maintenance (if at all possible).


My thoughts:



  1. CD-R/DVD-Rs have proven to me, even in the short term, to be a terrible medium for backups. They seem to be very fragile and seem to lose their data a very short time even when in pristine condition.

  2. I can't help but think that storing data on a couple of 1TB hdd's and then expecting them to spin up correctly a decade or two later to be a terrible idea. Am I wrong?

  3. Industrial tape drives seem like a viable option?



Answer



Paper


Other than archival ink on archival paper in sealed storage, no current medium is proven to last an average 100 years without any sort of maintenance.


Archival Paper



Older papers were made from materials such as linen and hemp, and so are naturally alkaline. or acid free, therefore lasting hundreds of years. 20th century paper and most modern paper is usually made from wood pulp, which is often acidic and does not keep for long periods.



Archival Inks



These permanent, non-fading inks are resistant to light, heat and water, and contain no impurities that can affect the permanence of paper or photographic materials. Black Actinic Inks are chemically stable and feature an inorganic pigment that has no tendency to absorb impurities like other ink pigments can.



Redundant storage


Torvalds once said



Only wimps use tape backup: _real_ men just upload their important stuff on ftp, and let the rest of the world mirror it



Which suggests you should not rely on a single copy on a single medium.


Not magnetic media?


http://www.zdnet.com/blog/perlow/the-bell-tolls-for-your-magnetic-media/9364?tag=content;siu-container



  • Typical example of irretrievable degradation of magnetic media.

  • Issues of hardware and software (and data formats)


Not specialized systems



In 2002, there were great fears that the discs would become unreadable as computers capable of reading the format had become rare and drives capable of accessing the discs even rarer. Aside from the difficulty of emulating the original code, a major issue was that the still images had been stored on the laserdisc as single-frame analogue video,



http://en.wikipedia.org/wiki/BBC_Domesday_Project#Preservation


Long Term Personal storage


http://www.zdnet.com/blog/storage/long-term-personal-data-storage/376



  • both the media AND the format can become unreadable.

  • print on acid-free paper with pigment inks and store in a cool, dry and dark place.

  • The first problem is picking data formats for maximum longevity.

  • Avoid using proprietary formats

  • USCSF is transferring all their original tapes - many in now-obsolete formats like BetaSP and VHS - to the 75Mbit motionJPEG2000 format


No comments:

Post a Comment

How can I VLOOKUP in multiple Excel documents?

I am trying to VLOOKUP reference data with around 400 seperate Excel files. Is it possible to do this in a quick way rather than doing it m...