Microsoft’s new 10,000-year data storage medium: glass

Of course the other problem is how much do you bootstrap? I mean do you start with fire and the wheel? Smelting copper? Electricity? Quantum mechanics? I would think the "Ideal" archival material would have many layers starting with human readable that could also include instructions to read the next layer.
Example....
First layer: Information important to "primitive" people and instructions to grind glass to make a magnifier.
2nd layer to small to read clearly without a magnifier. Similar information but instructions on making a microscope.
3rd layer to small to read without a microscope, more information and instructions....

Not sure where we could find this magical material though.
The Omnissiah
 
Upvote
0 (0 / 0)
Upvote
1 (1 / 0)

EnPeaSea

Ars Scholae Palatinae
4,992
From a purely cost point of view - that laser to write the data and the gear needed to read it back are expensive.

Combined with the slow speed something like this needs dramatic cost reductions in very very complicated and niche gear to be even remotely viable for anything but the most important and least dense data.

EDIT: I'd love to discuss this with the folks who downvoted me. 🤷‍♂️
Is that you, John "There is no evidence anyone wants (the mouse) as an input device" Dvorak? :p
 
Upvote
1 (2 / -1)
Combined with the slow speed something like this needs dramatic cost reductions in very very complicated and niche gear to be even remotely viable for anything but the most important and least dense data.

I didn't downvote you but nobody is suggesting this isn't a niche tech. Having something for the most important data (and moderate not low density) would be great. So you likely got downvotes by setting up intentional or otherwise as a strawman.
 
Upvote
7 (8 / -1)

Dzov

Ars Legatus Legionis
16,028
Subscriptor++
CRC Error at sector: 836864268543
Retry Abort

Edit: My assumption is the down votes are from people too young to understand the reference. If you know, you know.
One possible reason for a downvote is that optical storage since CDs have included various error correction schemes to account for a certain level of scratches. Granted, if the data is too damaged, good luck.

Also, the worst is when disk 21 of your 25 disk install is bad.
 
Upvote
0 (2 / -2)
One possible reason for a downvote is that optical storage since CDs have included various error correction schemes to account for a certain level of scratches. Granted, if the data is too damaged, good luck.

Also, the worst is when disk 21 of your 25 disk install is bad.
I recall that in Monkey Island this joke was patched out purely to save the tech support hotline the headache. It's a shame they didn't patch it back into the remake, long after anyone would have actually believed it.
 
Upvote
1 (1 / 0)

Golgatha

Ars Scholae Palatinae
1,376
The medium may last forever... if nobody cracks it.
And in practice, 20 years later you'll probably struggle to find drives to read it.

If you want to preserve data, keep copying it.
Yes, I think a lot of folks miss the bit about being able to access the data after it's written. The "drive" needed to read the glass medium requires lots of expertise to setup and maintain, not to mention is prohibitively expensive.

Same thing with optical media for home use. This medium isn't prohibitively expensive. However, that disc might last your entire lifetime, but will the optical drive? Not to mention, will a computer exist that has the right interface for said optical drive?
 
Upvote
2 (2 / 0)

HoboWhisperer

Wise, Aged Ars Veteran
138
Okay, true, there’s things generating petabytes of data. But no human being does, and neither do the vast majority of corporations and other entities.

Most people probably generate at most of few hundred GB of truly original data in their lifetimes unless they take a lot of video.
That was my thought too. Yes, we can generate Petabytes of data easily. But I imagine that data we intend to archive for 10,000 years will be curated to some degree - not just raw data dumps.
 
Upvote
2 (2 / 0)

Control Group

Ars Legatus Legionis
19,268
Subscriptor++
That was my thought too. Yes, we can generate Petabytes of data easily. But I imagine that data we intend to archive for 10,000 years will be curated to some degree - not just raw data dumps.
You’re no doubt right, but as others have mentioned, some of the most valuable information historians have found has been stuff no one would have thought to archive at the time. The Roman curses at Bath, for example (chosen only because I’ve been there). I doubt anyone doing serious data archival at the time would have committed them to laser etched glass. But “Docimedis has lost two gloves and asks that the thief responsible should lose their minds and eyes in the goddess' temple.” gives a lot of insight into what “normal life” was like there and then.

So I suspect that, from a historian/anthropologist point of view, the best outcome would be everyone has access to archival storage and uses it for whatever they want. And a huge chunk of it is lost/destroyed, but whatever’s left ends up being a representative sample of what life was like for people of our era. A way for an anthropologist to do better than describing modern US fashion as “they wore blue trousers”.
 
Upvote
8 (8 / 0)

plaidflannel

Ars Praetorian
440
Subscriptor
I am reminded of the novel Lucifer's Hammer by Larry Niven and Jerry Pournelle, in which the Earth is struck by a comet (nicknamed The Hammer). Expecting the destruction of civilization and imminent extinction of humans, the survivors wanted to preserve as much human history and knowledge as possible. Their hope was that after millions of years, a new intelligent species would evolve on Earth that would discover and learn to read that archive. The only storage medium they believed could survive for millions of years was glass. They placed the archive on the top of the highest mountain in Australia (IIRC), a place they expected the new species would eventually visit.

The book was nominated in 1978 for the Hugo Award fro Best Novel.
 
Upvote
1 (2 / -1)

Architect_of_Insanity

Ars Tribunus Militum
2,136
Subscriptor++
LTO is a terrible medium for long term archival.
The tapes may be rated for "30 years" but in ~10 years they won't be manufacturing drives capable of reading current tapes.
E.g. LTO-10 drives can't even read LTO-9 tapes.
Wow, something must have changed - I was always under the impression that LTO was W/R N-1 gen, R was N-2 generations.
 
Upvote
0 (0 / 0)
Wow, something must have changed - I was always under the impression that LTO was W/R N-1 gen, R was N-2 generations.

LTO-10 is the first one that can't even read or write the prior generation but the 2 prior generation for reading was lost starting in LTO-8. LTO-8 can RW LTO-7 but can't read LTO-6. Prior to that it was as you describe (i.e. LTO-7 can RW both LTO-7 & LTO-6 and read only LTO-5).

https://en.wikipedia.org/wiki/Linear_Tape-Open#Compatibility
 
Last edited:
Upvote
6 (6 / 0)
In 10,000 years we will either be far past the singularity and have no need for something like that as we could recall all data in all of human history or be using wooden clubs to fight off other and breaking the demon machines and everything associated with them.
How exactly do you believe "the singularity" can recover data that was never recorded? Some math isn't reversible. 2+2=4 Now, let's start with the end of that equation. 4. What produced it? Is there any way to tell? I know there's a postulate that if we simply measured EVERY single particle and their momentum, we could "reverse" all the math of solved physics and figure out literally everything that's ever occurred. That... makes a lot of assumptions. No matter how advanced a computer, it can't measure every particle in the universe at once, not least because simultaneity breaks down at the universal scale, but also because of the uncertainty principle preventing that from being doable in the first place, and basic true randomness at the quantum level making some things literally irreversible even in principle. But, also, there's no way to say that a singularity can just "solve" physics completely. No intellect, no matter how advanced, can do such without observations.

The notion that we can just count on a future god to create heaven at some arbitrary point isn't based on anything more than wishful thinking, with no basis in reality. A single drop of evidence is worth an ocean of speculation, after all. Heck, I'm one of the few here who doesn't actually think there'd be a meaningful difference in a copy of a person, and that it's literally generating the same consciousness, and that continuity of consciousness only matters from the perspective OF said consciousness, and I still think this is a pipe dream. But, fortunately my belief isn't required for this particular faith. If it happens, all it means is I was wrong, I'm still in virtual heaven with the rest of you... but I have SEVERE doubts on it ever being possible in ANY point in the distant future, and I will live my life under that assumption.
 
Upvote
0 (2 / -2)

FabiusCunctator

Ars Scholae Palatinae
899
Subscriptor
Dr. Peter Gainsford (aka kiwihellenist) wrote at length about it here, including explaining the source of the myth of its destruction setting back civilization: Carl Sagan on a segment of Cosmos in 1980.
What a fascinating read that is! The following particularly jumped out at me WRT the specific topic at hand (emphasis mine):

From KiwiHellenist:
The survival of ancient books isn't something that depends on one repository: that would put them at the mercy of regime changes, shifts in governmental priorities, funding. Books survive if they were copied, repeatedly. The story of ancient books being lost isn't a story of library fires: it's a story of economics, long-term cultural developments, and above all, format shifts.

IOW: not much has changed in 2000+ years!
 
Upvote
8 (8 / 0)

Waco

Ars Tribunus Militum
2,212
Subscriptor
I am working with tape archival at EB range.

If your tape software is serious about data preservation, the tape drive will read and verify each written tape block of your data as you are writing it (Logical Block Protection feature) this is a feature that has been around for a while on TS11XX and LTO drives (even in the last generations of 10K drives).
Therefore written data is always tested by design at 0 performance cost contrary to disk.

Migrating data every 5 years is a bit complicated at EB scale as the cost of media is several order of magnitude higher than the infrastructure cost: today 10 years may be a more reasonable target but this is more likely to increase further as medium amortization cost and exponentially growing amounts of data requires to keep data (and drives and libraries) longer and longer until we reach the 30 years data duration wall.

Then I guess this problem is for future generations of tape archive service managers: not for me to solve, just for me to warn.
We're in the multi-hundred PB range. We're to the point of doing erasure coding across tapes to ensure we can actually read every bit on the 5-10 year cadence to move to new media.

We do verify as written but on something without a lot of churn unless you're constantly reading tapes in the background (which presents its own challenges and risks) it's a sliding scale of tape age and bit error rates.

I wish I could trust tapes to actually be readable at the 10-15 year range without a lot of overhead and risk.
 
Upvote
2 (2 / 0)

Waco

Ars Tribunus Militum
2,212
Subscriptor
I didn't downvote you but nobody is suggesting this isn't a niche tech. Having something for the most important data (and moderate not low density) would be great. So you likely got downvotes by setting up intentional or otherwise as a strawman.
I don't disagree - I want this to succeed.

I'm just being realistic on the timelines for it being viable. When your media reader/writer costs orders of magnitude more than the existing technology and runs orders of magnitude slower...that's a long ramp. There aren't a lot of external pressures to bring those technologies dramatically forward at this point.
 
Upvote
1 (2 / -1)
I don't disagree - I want this to succeed.

I'm just being realistic on the timelines for it being viable. When your media reader/writer costs orders of magnitude more than the existing technology and runs orders of magnitude slower...that's a long ramp. There aren't a lot of external pressures to bring those technologies dramatically forward at this point.

LTO-10 drives aren't exactly cheap. If this hit the market at $50k for the drive and glass slabs for $50 each there would be a market. Even $100k and $200 there would be a small market.
 
Upvote
1 (1 / 0)

Waco

Ars Tribunus Militum
2,212
Subscriptor
LTO-10 drives aren't exactly cheap. If this hit the market at $50k for the drive and glass slabs for $50 each there would be a market. Even $100k and $200 there would be a small market.
The glass is indeed cheap from estimates I've seen (from other companies too).

The "drive" is a multi-rack scale thing with costs in the 7 to 8 figure range. It's promising tech but it has a long way to go before it becomes really usable.
 
Upvote
1 (2 / -1)

Martin Blank

Ars Tribunus Militum
2,591
Subscriptor++
The Library of Alexandria was irreplaceable because replicating the entire collection would have required paying thousands of scribes—who were among the best-educated people in classical times, and consequently extremely valuable to society—to spend decades to do nothing but copy old books. The cost in human resources would have been too much for even the Roman emperors to bear.
As I commented on the earlier page, the loss of the Library of Alexandria was not catastrophic. As best we can tell, Carl Sagan made up the effects of its destruction out of whole cloth, or perhaps passed on something he heard but never verified. (Learning this was disappointing to me, as Sagan was critical to public science education.) Professional historians do not believe much, if anything, was permanently lost because there were hundreds of other libraries scattered around the Mediterranean, and copying manuscripts was common, if expensive.

And the scribes were not the educated elite. They were very often literate slaves doing work for their masters, whether for private collections or building prestige by donating to libraries. There were thousands of them around the Mediterranean at any one time, and copying manuscripts was a major part of their time.
 
Upvote
1 (1 / 0)
Yes, I think a lot of folks miss the bit about being able to access the data after it's written. The "drive" needed to read the glass medium requires lots of expertise to setup and maintain, not to mention is prohibitively expensive.

Same thing with optical media for home use. This medium isn't prohibitively expensive. However, that disc might last your entire lifetime, but will the optical drive? Not to mention, will a computer exist that has the right interface for said optical drive?
Ah yes, the nightmare that is 4K Bluray has really driven that point home. There are processors right now that are incapable of playing back 4K movies, even with all the right software set up, purely because of Sony's insistence on setting up such a ridiculously layered DRM that they have effectively killed their own product. As it stands, if I want to play 4K movies, I use a console. If I want to RIP 4K, I need to hack custom firmware onto my drive, presuming someone's developed custom firmware for my specific model. With console makers eager to send optical drives to the dustbin of history, and streaming so easy to use and so popular, 4K is already a niche market. However, Sony's insistence on processor DRM which Intel themselves have decided to simply stop including, as well as insisting on locking down drives so people can't even get to their OWN movies through roundabout means without FURTHER roundabout means... THAT has flat out killed the product. It's not truly dead until companies simply stop producing 4K releases, but there's no way to sponge the writing off that stone at this point.
 
Upvote
4 (4 / 0)

Dzov

Ars Legatus Legionis
16,028
Subscriptor++
I recall that in Monkey Island this joke was patched out purely to save the tech support hotline the headache. It's a shame they didn't patch it back into the remake, long after anyone would have actually believed it.
Lol, I played that game and don't recall the joke. Oh well. Pretty sure that was before you could easily get game patches. Actually, how'd you even find out about that patch story? Like on FIDO net or IRC or one of the gaming magazines we used to have?

edit: I just googled "CRC Error at sector: 836864268543" and have no Monkey Island results. Are you sure you're remembering this correctly?
 
Last edited:
Upvote
1 (1 / 0)

McTurkey

Ars Tribunus Militum
2,209
Subscriptor
All of this is great, but has storing the data ever actually been the problem? I'm decidedly ignorant here, so I'm asking a genuine question. My read on this was always that it was the data itself that became obsolete, as file formats aged out, software changed, etc. If you don't have software that can read the data, what good is the data?
The mechanism by which aliens made contact in the novel Contact is the key.

You start from the absolute simplest mathematical premise (1 + 1 = 2), and build out a shared understanding of numeracy, math, and then languages and eventually the engineering to recreate the machines to decipher the data etched into the glass. The initial "Rosetta Stone" must be something that requires no technology to parse beyond vision or touch, so it's obviously going to be super low density as a storage medium, and also needs to last at least as long as the glass storage.

If you're serious about preserving data for thousands of years, this is absolutely necessary. The typical child born today will not be educated in how to read or write in cursive, and therefore will not be able to read most documents written in the last few hundred years or even many in the last few decades. Even the languages used today are wildly different than they were a few hundred years ago, let alone what humans were using ten thousand years ago.

I think there's real value in trying to preserve human knowledge in the event of systemic collapse. Even if it's just for alien archeologists to one day pick through--and if you're really looking at eon-level data retention, that is the lens you must use.
 
Upvote
5 (5 / 0)
Lol, I played that game and don't recall the joke. Oh well. Pretty sure that was before you could easily get game patches. Actually, how'd you even find out about that patch story? Like on FIDO net or IRC or one of the gaming magazines we used to have?

edit: I just googled "CRC Error at sector: 836864268543" and have no Monkey Island results. Are you sure you're remembering this correctly?
I was talking about the "floppy not found or corrupted" error rather than the CRC one, sorry! For more details, check this out!
https://tcrf.net/The_Secret_of_Monkey_Island_(DOS,_Mac_OS_Classic)/Revisional_Differences
As for how I found out, I didn't find out about the BACKGROUND of the change until several years later. I found out about the difference itself (without the background context) just by playing the CD version after the floppy version, since my floppy version happened to be the older unpatched one.
 
Upvote
2 (2 / 0)

ossuary

Seniorius Lurkius
31
Hmmm... Data stored on glass? That works for thousands of years? Crazy talk!

crystals.jpg
 
Upvote
3 (3 / 0)