ZFS hits the streets (sorta)

[skroo]

After half a decade of development, Sun's ZFS will be released in two weeks.

I'll admit to having a certain fascination with filesystems, and this one looks pretty killer. Apart from having a 128-bit filespace, the following (ut and paste from the article) look pretty damned tasty:

* Dynamic striping across all devices to maximize throughput
* Copy-on-write design makes most disk writes sequential
* Multiple block sizes, automatically chosen to match workload
* Explicit I/O priority with deadline scheduling
* Globally optimal I/O sorting and aggregation
* Multiple independent prefetch streams with automatic length and stride detection
* Unlimited, instantaneous read/write snapshots
* Parallel, constant-time directory operations

I want it, and I want enough storage to max it out.

[bascule]

Here's a story re: the upcoming release

http://www.theregister.co.uk/2005/11/02/sun_zfs_coming/

More than a year ago, Sun unleashed the idea of the Zettabyte File System (ZFS), and customers paid attention. A high-performance 128-bit file system that prevents data corruption and stretches across tons of systems? Sure. Why not.

Customers, however, have become concerned in recent months that ZFS was no more. Sun officials said relatively little about the product, and it didn't appear in Solaris 10 updates. So, a number of you wrote to us asking for an update on the software. Here it is.

Our phone taps at Sun have turned up some key ZFS information. First off, the software made it into the next version of Solaris's code base just this week. ZFS will stand as another feature in Solaris 11 or whatever Sun plans to call the next major version of its operating system.

In the meantime, customers will be able to grab ZFS from the Solaris Express program in two weeks time. It's reasonably safe to assume that an OpenSolaris launch will follow.

I think ZFS's biggest feature is that it integrates the filesystem with the volume manager. Thus you can manipulate the underlying drive configuration however you want, and the filesystem will automatically reorganize itself to compensate.

You could say, start with a RAID-1, then when you need more capacity add an additional drive which transforms it into a RAID-5. Tell ZFS about the new drive and the filesystem automatically doubles in capacity. "BFD, I can do that already!" you say? Well, then when you need more space down the road, you can add an additional drive. Parity data will be recalculated, and the filesystem reorganized to compensate for the additional space.

What if you aren't using the full capacity of a RAID array and you would like to use the drives elsewhere? Well, you can remove a drive from the array, and (provided there's enough free space remaining after the drive has been removed) parity data will be recalculated and the filesystem shrunk to compensate.

So basically your data is free to flow across whatever resources available in the system. No more twiddling with gimpy RAID BIOS that only allows a very select set of array transformations, and no more fiddling with seperate, non-integrated tools to resize the filesystem. ZFS puts it all together into one package to ensure a totally futureproof storage system.

[Voltage Spike]

(ut and paste

skroo using 1337-speak or a typo? We'll never know...

Dynamic striping across all devices to maximize throughput

What is the advantage of putting RAID in the file system? Better, doesn't RAID already detect data faults? (The article implies that it does not.)

Copy-on-write design makes most disk writes sequential

Is copy on write really a better design than log-structured file systems? It would seem that writing just the changed blocks would be more efficient (especially for large files).

I suppose they could mean copy-on-write on a per-block basis, though.

Unlimited, instantaneous read/write snapshots

Once again I am led to believe that this is a log-structured file system.

Parallel, constant-time directory operations

I was under the impression that this is a pretty standard feature these days.

I want it, and I want enough storage to max it out.

Sure enough. You weren't planning on using Jupiter for anything soon, were you?

[skroo]

skroo using 1337-speak or a typo? We'll never know...

"I'll take 'typo' for $100, Alex..." :)

Regarding everything else: I'd recommend reading the release that accompanied it. The bullet points I threw out there were probably the single worst selection I could've made from the text. Having said that, though, it sounds like ZFS could well be more than just the sum of its parts if done right.

[bascule]

What is the advantage of putting RAID in the file system?

The point is that the volume manager and filesystem are integrated to provide maximum flexibility.

Better, doesn't RAID already detect data faults? (The article implies that it does not.)

Not typically. RAID is designed to tolerate total drive failures, not provide data consistency checking to ensure it isn't corrupted by a malfunctioning disk. ZFS provides the latter.

I think there's huge potential for Solaris-based storage servers, especially when SATA controllers like this are supported:
http://www.shopbot.com.au/price/Adap...ort-PCI-X.html