Solid state revolution: in-depth on how SSDs really work

SSDs use a huge grab bag of techniques to make a computer feel "snappy."

Read the whole story

 

[pokrface]

Good article.

One thing that the article does not seem to cover is that many (but not yet that many) home PC users are starting to deploy Bitlocker or Truecrypt or other kind of full disk encryption. And even more users are forced to encrypt their laptops if they wish to work on them.

Using full disk encryption absolutely destroys SSD performance. There is zero use for a SSD if you do full disk encryption - it slows down to hard drives from 1996.

Another obvious problem with SSDs is that they can never be truly erased, which means all your passwords/pictures/etc. will be stored on them forever, and recoverable if you choose to sell your computer, discard it, or send it in for repairs.

Fear not--this article is the first in a series. I'll discuss full disk encryption in one of the upcoming parts.

 

[pokrface]

RAID 0 argues against both redundant and independent. It makes the disks entirely dependent on one another, and entirely non-redundant.

I contest that: Redundant Array of Independent Disks, where independent refers to the disks, not the array itself. By arguing against independent, you are arguing that the disk is no longer an independent disk when in an array, which simply doesn't make any sense.

RAID is an acronym, and it stands for "Redundant Array of Inexpensive Disks". Whether it is semantically correct or not in certain applications is immaterial, because RAID is an acronym and it stands for "Redundant Array of Inexpensive Disks". If you want to debate its appropriateness or etymology then that's fine, but the name of the thing is the name of the thing.

 

[superchkn]

Hmm, and there I would think the Lee would have problems with the word "redundant", not independent. If you put two disks in a RAID 0 array, they are still independent disks, but they are not redundant. (Perhaps that's why it was named 0) Strangely, Lee doesn't have any problem with the manufacturers using the word independent in their naming conventions - though, I don't think any of them are sticking them in a RAID 0 class configuration, so perhaps that is why.

RAID has always stood for "Redundant Array of Inexpensive Disks." The "I" has been bastardized of late and most folks say "Independent", but they're wrong. I can call a duck a cow, but that doesn't make it so.

RAID's opposite, which we don't hear much these days, is SLED--"Single Large Expensive Disk". When it first appeared in the late 80s, RAID was originally a cost-saving measure as much as anything else, since large hard disk drives at the time were extremely costly.

I'm not going to argue the history, because that is true that it began life as inexpensive. It would have been fine if you wanted to argue that, but instead you chose to say that it didn't make sense because of the independent part, which is simply untrue. Following that logic, it wouldn't make sense to use inexpensive for referring to a large array because the total cost of said large array would no longer fall into the inexpensive category.

The fact is that RAID is commonly defined both ways and personally, independent makes more sense in this day and age where a disks inclusion in an array doesn't preclude relatively expensive drives from finding a home there. The English language evolves and I see no reason why acronyms should not evolve with usage either.

 

[superchkn]

RAID 0 argues against both redundant and independent. It makes the disks entirely dependent on one another, and entirely non-redundant.

I contest that: Redundant Array of Independent Disks, where independent refers to the disks, not the array itself. By arguing against independent, you are arguing that the disk is no longer an independent disk when in an array, which simply doesn't make any sense.

RAID is an acronym, and it stands for "Redundant Array of Inexpensive Disks". Whether it is semantically correct or not in certain applications is immaterial, because RAID is an acronym and it stands for "Redundant Array of Inexpensive Disks". If you want to debate its appropriateness or etymology then that's fine, but the name of the thing is the name of the thing.

It must be material because Lee made that argument in the article. If it wasn't material and only the historical context mattered, then why make the argument in the first place?

 

[malor]

I've only gotten through page 1, so far, but I think the author probably uses a Mac.

Why? Because SSDs are terribly important on a Mac for good system performance. They matter so, so much, more than any other single component. If the author uses a Mac, then his description of the night-and-day difference is precisely accurate. It really is that big a deal.

Windows, however, especially 64-bit Windows 7 with a lot of RAM, does an excellent job of insulating you from a slow hard disk. It has very smart caching, and it's able to make a big-RAM machine feel much faster than it actually is, and way, WAY faster than OS X on the same hardware.

So, an SSD still matters on Win7, but it has much less of an impact, especially if you have 16 gigs. (which is super cheap, these days, under $100) The biggest thing an SSD helps with is boot speed, while the BIOS is still running the disk. Once the kernel takes over, and starts using all that copious RAM as a drive cache, the difference between SSD and rotating storage is reduced very sharply.

I presume that OS X is so absolutely dominated by drive seek time as a component of overall performance because of the ancient and creaky HFS+. With a replacement filesystem, or with performance tuning, that bottleneck could disappear. But at the moment, that is THE upgrade you want on a Mac, if you don't already have it. A Mac Mini with an SSD will feel faster, in routine use, than a monster Mac Pro on a rotating drive.

 

[DOOManiac]

Another fantastic article from Ars. Keep up the great work!

 

[superchkn]

I've only gotten through page 1, so far, but I think the author probably uses a Mac.

Why? Because SSDs are terribly important on a Mac for good system performance. They matter so, so much, more than any other single component. If the author uses a Mac, then his description of the night-and-day difference is precisely accurate. It really is that big a deal.

Windows, however, especially 64-bit Windows 7 with a lot of RAM, does an excellent job of insulating you from a slow hard disk. It has very smart caching, and it's able to make a big-RAM machine feel much faster than it actually is, and way, WAY faster than OS X on the same hardware.

So, an SSD still matters on Win7, but it is not the night-and-day difference that it is under OS X, especially if you have 16 gigs. (which is super cheap, these days, under $100) The biggest thing an SSD helps with is boot speed, while the BIOS is still running the disk. Once the kernel takes over, and starts using all that copious RAM as a drive cache, the difference between SSD and rotating storage is reduced very sharply.

I presume that OS X is so absolutely dominated by drive seek time as a component of overall performance because of the ancient and creaky HFS+. With a replacement filesystem, or with performance tuning, that bottleneck could disappear. But at the moment, that is THE upgrade you want on a Mac, if you don't already have it. A Mac Mini with an SSD will feel faster, in routine use, than a monster Mac Pro on a rotating drive.

Or, perhaps he uses a laptop? I'll happily spec all my laptops with SSDs over 16GB of RAM for the aforementioned subjective improvement in response along with the significantly reduced failure rate that comes along with a solid state drive. Not to mention the smaller hibernation file that would result from the perfectly adequate 4GB installed in your run-of-the-mill business machine.

 

[slogger]

It's also possible to emulate the beneficial effects of over-provisioning by simply using less than the stated capacity of an SSD—for example, by purchasing a 90GB SSD, creating a 30GB partition, and leaving the rest unallocated.

If this is the case, what's a good trade-off for spare space vs prolonging life?

Also, how do partitions live on SSDs? While the notion that storage at the block level is partition agnostic makes intuitive sense, it seems like you've then got to have somewhere for the abstraction to live and to map used pages to their current partition. In particular, what's constraining partition capacity if their storage space overlaps? It seems like 2 partitions would be greedy across shared space intuitively.

It seems like partitions themselves have to be some sort of constructed abstraction then an actual physical-level layout difference when it comes to SSDs.

Anyway really curious about this. (And know nothing.)

 

[pokrface]

I've only gotten through page 1, so far, but I think the author probably uses a Mac.

For what it's worth, I use a Mac desktop at home, a Windows 7 laptop at work, a Mac laptop on the road, and there are four Ubuntu servers in my closet. The only one without an SSD is the iMac, because cracking that puppy open is a little more involved than I care to get these days with PC repairs. Every system I have has benefited from SSDs.

Windows 7, in particular, picks up a ridiculous boost, because the version of Win7 that I'm using is one provided to me by my full-time job, and it's loaded down with all the things that a standard corporate image from a big company typically includes. It's a situation in which I'd imagine most corporate users find themselves, and an SSD is just about the only thing there is that can make a computer work well under those conditions.

 

[hairyfeet]

Great article but for me and my customers SSDs still have too high a failure rate. Read the article on coding horror called "The hot/crazy scale" to see what kind of failure rates we are talking, for that hot speed you are paying with crazy failure rates. At least with HDDs one gets SMART warning or noise before they die, I've had several gamer customer decide to ignore my advice and take the plunge and ALL had the drives die in less than two years and the failure was always the same, just flip the switch and nothing. No warnings, no heads up, just gone.

So if ALL you use the SSD for is the OS (or as a video editor friend uses it for a scratchpad for editing) and you ALWAYS keep a VERY current backup? Go for it. But for the average folks I'm lucky if I can get them to back up twice a month, they are so used to having things "just work" that the increased failure rate just isn't worth it. better to max out the RAM on the system and use a fast Readyboost cache to gain a little extra speed on small I/Os than to risk the all day PITB of restoring from backups and getting everything back the way it was.

 

[milan616]

I've been deeply missing Jon Stokes level of depth in hardware articles here at Ars, but I think Lee has picked up the torch in a fine way. Keep it up!

 

[pokrface]

It's also possible to emulate the beneficial effects of over-provisioning by simply using less than the stated capacity of an SSD—for example, by purchasing a 90GB SSD, creating a 30GB partition, and leaving the rest unallocated.

If this is the case, what's a good trade-off for spare space vs prolonging life?

Also, how do partitions live on SSDs? While the notion that storage at the block level is partition agnostic makes intuitive sense, it seems like you've then got to have somewhere for the abstraction to live and to map used pages to their current partition. In particular, what's constraining partition capacity if their storage space overlaps? It seems like 2 partitions would be greedy across shared space intuitively.

It seems like partitions themselves have to be some sort of constructed abstraction then an actual physical-level layout difference when it comes to SSDs.

Anyway really curious about this. (And know nothing.)

A good trade-off is to just use the drive's built-in overprovisioning, because otherwise you're further skewing the drive's cost per gigabyte However, if you know you're going to be buying a disk and then using it to write and delete a whole hell of a lot of tiny files, it might then be worth partitioning the disk down a bit. I don't know of any rules of thumb off the top of my head, but it's not something you'd need to worry about under most circumstances.

Regarding your question about partitions: the operating system talks to the SSD in terms of logical block addresses, and the SSD controller keeps a map called the Flash Translation Layer that maps LBAs to physical NAND pages. When pages are shuffled around as they're modified or as part of garbage collection, the controller updates the FTL map. From the operating system's perspective, it talks to the same LBAs no matter what physical addresses they actually correspond to.

Partitioning doesn't particularly matter--the OS knows the range of LBAs which make up its partitions.

 

[superchkn]

Great article but for me and my customers SSDs still have too high a failure rate. Read the article on coding horror called "The hot/crazy scale" to see what kind of failure rates we are talking, for that hot speed you are paying with crazy failure rates. At least with HDDs one gets SMART warning or noise before they die, I've had several gamer customer decide to ignore my advice and take the plunge and ALL had the drives die in less than two years and the failure was always the same, just flip the switch and nothing. No warnings, no heads up, just gone.

So if ALL you use the SSD for is the OS (or as a video editor friend uses it for a scratchpad for editing) and you ALWAYS keep a VERY current backup? Go for it. But for the average folks I'm lucky if I can get them to back up twice a month, they are so used to having things "just work" that the increased failure rate just isn't worth it. better to max out the RAM on the system and use a fast Readyboost cache to gain a little extra speed on small I/Os than to risk the all day PITB of restoring from backups and getting everything back the way it was.

SMART isn't all the reliable, take a look at Google's data. In desktops it may make less sense, but for laptop users that constantly move around (think a clinical environment), they are a clear win. The average mechanical drive lasted about 6 months for that use case. Even if I manage only two years, that would still be a significant improvement.

 

[whardy]

Loved the article! Incredibly informative and easy to read.

 

[mbmcavoy]

I have a 2010 Mac Mini that was painfully slow with an increasing workload. We have massive iTunes and photo libraries, moved from iPhoto to Aperture, and my wife is more frequently shooting RAW and editing in Pixelmator. She would spend most of her time looking at the beach ball...

A couple months ago, I replaced the internal drive with a 128GB SSD, and also bumped the RAM from 2GB to 8GB while I was in there. (Our media already was on an external FW800 drive.) This far exceedes expectations! Nothing slows down the user experience anymore; even running disk-intensive processes that peg the CPU to 100% for minutes on end, the machine remains as snappy as when idle.

Since then, the drive in our old MacBook failed. I really wanted to put in an SSD. Unfortunately, 128GB would be a squeeze, 256GB way too expensive to justify, and the old drive that came out of the Mini was free. (sigh...)

 

[RaveBomb]

Love this stuff.

As a side note, the Eve-Online cluster has been using Ramsan disk for a few years now in an effort to reduce latency.

 

[pokrface]

Since then, the drive in our old MacBook failed. I really wanted to put in an SSD. Unfortunately, 128GB would be a squeeze, 256GB way too expensive to justify, and the old drive that came out of the Mini was free. (sigh...)

There's a great perpetual SSD thread in the Ars OpenForum (link is to the current last page) which is pretty active with folks posting SSD buying advice and prices. There's a link on the last page, for example, for a 256 GB Crucial M4 at less than a buck per gig!

 

[TjOeNeR]

Lol, the cells in the images of the memory chips on the second page are spaced exactly the same amount as one mouse scroll.

Nice touch

 

[kharms]

It's articles like this one that make me always come back to Ars... fabulous work! Please post more stories like this!

 

[pokrface]

Lol, the cells in the images of the memory chips on the second page are spaced exactly the same amount as one mouse scroll.

Nice touch

All art in the article is done by Aurich, who is a rock star with the graphics and might also be an actual for-real rock star. Props to him

 

[HiWire]

I'm curious about why RAM disk adoption is so slow in the industry. The last page of the article mentions DRAM caching for enterprise systems, but I'm pretty sure that end users would also benefit from a huge speed bump.

The operating system already manages a RAM-based cache. It's called the file system cache.

That's why I found Microsoft's ReadyBoost so shockingly backward when it was first announced with Windows Vista. Flash memory via USB 2.0 is magnitudes slower than DRAM.

ReadyBoost isn't backward at all. It's exploiting flash's lower latency to mask spinning disk latency.

Why are operating system developers so slow to integrate native RAM disk support?

Because they've been doing something superior for 30 or more years.

Thanks, Peter, Lee, and Disconn3ct. As an average user, I didn't understand why large amounts of RAM remained unused by the system, but I'll trust that it is allocated intelligently by the system on an as-needed basis.

I've been doing a lot of storage research by reading StorageReview.com news in the last few months, but this question was never addressed directly there.

Also, I've been investigating the new Seagate Momentus XT drive (mentioned by a previous commenter) which has 2 levels of cache itself. It's hard to envision how all the caches work together in a system, especially if they are each working with their own, separate logic.

 

[zero cool]

The only one without an SSD is the iMac, because cracking that puppy open is a little more involved than I care to get these days with PC repairs.

I did this, and it was a bit scary, but ultimately rewarding. Used the iFixit guide and lots of crossed fingers. Replaced the DVD drive with the SSD, and replaced my old HD with a larger one (my previous one had failed). Been working well for about a year now. Very snappy!

 

[Geraden_]

I tip my internetz to you. Great article. As a current university student, I have to say this is higher quality material than anything I could hope to see from my professors. That may not be setting the bar very high however...

 

[rrohbeck]

>just thousandths of a millimeter above platters
You're a few orders of magnitude off. Flying heights are a few nm.
And as a "big iron" kind of guy I have to say that you left tape out, with access times of minutes to hours (or days if it's offsite.)
Other than that, great article on SSDs.

 

[dexvx]

Arstechnica needs more articles like this.

 

[sporkme]

One thing you might want to fact-check here is the bit about capacitor backup. It may be different now, but about 4-5 months ago when I was shopping I found the Intel 320 series was the only consumer drive that offered protection from power loss and the resulting scrambling that might ensue (especially with the one vendor who's name starts with an "O" that aggressively tunes for benchmark performance and leaves lots of data in the cache at all times).

Other people smarter than me have tested it: http://blog.2ndquadrant.com/intel_ssd_n ... _sherr_sh/

Supposedly there is a high-end OCZ with this feature, and then an assortment of "enterprise" drives with it, but the 320 was it when it came to something affordable.

I'd love to be proven wrong - my laptop has a 320 and my desktop is starting to feel laggy, so it's time to shop soon.

Another interesting update regarding 320 wear in heavy db testing use: http://blog.2ndquadrant.com/intel_ssds_ ... nd_the_32/

And a nice anandtech article about measuring write amplification under a given workload:

http://www.anandtech.com/show/5518/a-lo ... tel-ssds/6

 

[Fuzzmz]

Amazing article! Really glad to see this type of writing on the web and making a comeback on Ars!

 

[Wheels Of Confusion]

A very well-written article, and I especially started appreciating the in-depth discussion of how flash works when the later discussion of things like wear-leveling or SandForce's techniques were brought up.

 

[JGG]

I'm curious about why RAM disk adoption is so slow in the industry. The last page of the article mentions DRAM caching for enterprise systems, but I'm pretty sure that end users would also benefit from a huge speed bump.

The operating system already manages a RAM-based cache. It's called the file system cache.

That's why I found Microsoft's ReadyBoost so shockingly backward when it was first announced with Windows Vista. Flash memory via USB 2.0 is magnitudes slower than DRAM.

ReadyBoost isn't backward at all. It's exploiting flash's lower latency to mask spinning disk latency.

Why are operating system developers so slow to integrate native RAM disk support?

Because they've been doing something superior for 30 or more years.

Thanks, Peter, Lee, and Disconn3ct. As an average user, I didn't understand why large amounts of RAM remained unused by the system, but I'll trust that it is allocated intelligently by the system on an as-needed basis.

I've been doing a lot of storage research by reading StorageReview.com news in the last few months, but this question was never addressed directly there.

Also, I've been investigating the new Seagate Momentus XT drive (mentioned by a previous commenter) which has 2 levels of cache itself. It's hard to envision how all the caches work together in a system, especially if they are each working with their own, separate logic.

http://www.linuxatemyram.com/

Is somewhat relevant. All operating systems use "unused" RAM for disk cache. It's just the Linux nerds that report it as used.

 

[badfrog]

That may in fact be the most epic lead paragraph I've read in an Ars article ever. +1 internets to you sir.

 

[SarahSparkles]

I really appreciate the in-depth article. As a reader who's been dismayed by the seemingly downward trend in quality at Ars, this article is great to see.

 

[BoCaro]

Can SSD gurus explain me what's wrong in my case?

Brand new SSD, Crucial M4 128 GB, I updated Firmware 0309 to 000F. Using on netbook Acer Aspire One 722 (CPU = AMD C60, 4GB Ram). Replacing a 500 GB hard drive.

Then I installed Ubuntu 12.04 x64 bits. I modified fstab to tell ext4 to do trim. Not sure what the technical talk should be here. I just follow the guides found on various "SSD How to for Linux" published on the Internet.

I was expecting a spectacular improvement everybody said when trying SSD the first time. I got a barely noticeable improvement. The boot time is increased by 7 seconds (22s -> 15s). When using the machine, the operation seems to be a little bit faster. But nothing earth shattering. Is it because the CPU AMD C60 is too weak so that the Disk I/O improvement can not show its full potential?

Thanks for any advice.

 

[DCFusor]

I now have SSD's in all my key systems (intel). Since I run Ubuntu, I set things up thusly:
Spinning disk (usually seagate 1 tb) gets /home, SSD gets the rest. I tried to partition things so I could also move /var to the spinners, but due to ??? that one sometimes can't boot - something gets ready too quick (boot from SSD?) for this to work correctly and reliably.

The idea of course is to limit the number of writes to the SSD, but get the speed for loading software. No failures yet, but not that many zillions of hours on any of these yet either, so the jury will have to stay out awhile. But boy do they all boot quick, and load software fast. In a lot of cases, I can get around the /var problem by telling various programs to use space under /home for log files - I usually create a big pub directory there and sub directories of that serve well - and you get some raid-like advantages if you're reading from the SSD but writing logs to /home/pub/something.

There are some tricks to reduce the number of writes to the SSD, things you can do in fstab, but those change over time (and version of ubuntu - but I'm stuck at 10.04 as unity sucks so hard I can't bear it), and I've not implemented them all in all systems. Ideally, I'd do this (turn off access time updates, journaling - or move the journal to the spinner if I could) but life is short.

I used to do a lot of development with uP's and non volatile memory - Flash and eeprom. Let me tell you about nervous, back in the day ~1000 writes might be the limit - think how long it takes to fry one if you write a bad loop - so fast you can't hit the off switch! For one project, I developed a file system that doesn't keep the "system" on the flash at all - it's reconstructed in ram at boot, which makes that slow, but then runs like the wind (eg all the node info is discoverable by a brute force search at boot and kept in ram during the uptime - that product was an always-on thing, so this was only a one-time penalty).

What the author didn't mention is that quantum tunnelling is analogous to a tiny arc across the insulating oxide. This damages it - and that's why there's a limit to the write count. After awhile, it's not a very good insulator any more - some of the oxide is reduced to conductive metal each time.

 

[jamoau]

Thanks, great read!

Looking forward to your future article on this; encryption, security and forensics in regards to SSDs is fascinating. The other benefit I like from an SSD is instantaneous wake up from sleep.

 

[pokrface]

What the author didn't mention is that quantum tunnelling is analogous to a tiny arc across the insulating oxide. This damages it - and that's why there's a limit to the write count. After awhile, it's not a very good insulator any more - some of the oxide is reduced to conductive metal each time.

I do in fact talk about floating gate degradation -- there's an entire section on it I sourced the section from several different places, but primarily from this paper: Degradation of Rise Time in NAND Gates using 2.0 nm Gate Dielectrics. The short, short version is that electrons like to stay in the floating gate. Getting electrons into the floating gate requires current, and getting the electrons *out* of the floating gate requires more current; eventually, the amount of current it takes and the amount of time that current must be applied means that it takes way longer to make that cell change versus the other less-electron-y cells, and so it gets marked bad.

 

[hobgoblin]

I take it that memristors show a similar degradation over time?

 

[sanchonx]

Thanks to the authors for this insightful, in-depth, well-written and well-researched article! It answered a lot of questions for me.

 

[alxx]

Going from a mach 64 to quantum 3d obsidian 2 (still have it in a box on the shelf) was a massive step up

Nice article. Would be great to see a lot more like it.

Been a while since ars did a decent indepth cpu article.
Or maybe some coverage of the state of the art in networking.10/40/100G and articles on gpon etc.

 

[asdf25]

I just wanted to add that I think a good analogy for memory hierarchy is stock/parts in a factory floor. Registers are like the parts right next to the assembly line. The cache is a shelf right on the factory floor. The main memory is the onsite warehouse and the hard drive is the distribution center down the street. Or something like that. The trade-off between the need to have parts nearby and the cost of having storage space on the factory floor is analogous and intuitive.

 

[ismaris]

It's funny, I just made a similar comment on Facebook about my new SSD. To me it feels like the biggest leap in computing in the last decade, at least as far as user experience goes. I love my Windows on SSD setup.