A very important and geeky technology that Apple has been working on surfaced after the WWDC keynote, namely that Apple has been working on a new file system for the Mac and other Apple devices. It was probably too geeky to make the cut for a keynote broadcast live, but it's still incredibly important. Here's the background and what we know about APFS.
What is a File System?
A file system is method used by a computer to store and retrieve data. A simple way of thinking about it is an old-fashioned phone book. A phone book stores names, addresses and phone numbers alphabetically by name. When we look up a name, we know how to scan alphabetically, read across what we found, and retrieve the phone number on the same line.
Similarly, a computer files system is a set of protocols and code that specifies how data (and metadata) is stored, found, retrieved. In simple terms, things like multiple write contentions, buffering and file integrity, and file size and number limits are set.
Evolution
Computer file systems have evolved over the years to meet the need of ever advancing hardware. When hardware and processors were slow, it didn't make sense, nor was it possible, to design very advanced file systems. And, so, periodically we go through a technology convulsion as the vendor designs a more capable file system and we (carefully) migrate with it.
We've gone from floppy disks to hard disks to SSDs in the lifetime of the Mac, and the amount of user data has expanded by many orders of magnitude. File systems are revised, hopefully with backwards compatibility, so tha users can continue to access older devices while dealing with terabytes and perhaps petabytes of data.
The file system Apple has been using since 1998 for both the Classic Mac OS as well as the evolution of Mac OS X, OS X, and now macOS is called HFS+. It was suitable for the Classic Mac OS in the 1990s and perhaps barely suitable for a UNIX file system, BSD, that Apple called Mac OS X back in 2001. But then, it generally worked well, Apple had control of it, and it could be designed at will to meet the UI and UX needs of Apple's customers. Today, iOS, tvOS and watchOS have inherited HFS+.
Over the years, it became clear that HFS+ was not a file system that could take the Mac and inheritors of HFS+ into the future. Apple experimented briefly in 2009 with Sun's ZFS file system, but there were issues that prevented its deployment. That's beyond the scope of this discussion.
Other events that have driven Apple's quest for a new file system, especially on the Mac, have been the need for more robust security, the nagging problem of how Time Machine volumes are managed (and encrypted) and the widespread use of SSDs/Flash storage, especially as boot drives in Macs today.
Page 2: Criticism of HFS+, APFS to the Rescue, Release Plans
Page 2: Criticism of HFS+, APFS to the Rescue
Criticism of HFS+
Back in early 2015, the father of Linux, Linus Torvalds, had some sharp comments to make about the weaknesses of HFS+. "Linus Torvalds: Apple's HFS+ is probably the worst file system ever." Mr. Torvalds critiqued HFS+ in several important technical areas, and he was right. Thin-skinned Apple enthusiasts bristled at the comments, but Torvalds was right. And it stuck.
HFS+ lacks fine grained time stamps, checksumming, snapshotting, and sparse file support, to name a few. Also, it's also incapable of operating, by design, beyond early 2040.
What emerged in the past few years was a nagging feeling in the community, and likely Apple engineers, that HFS+ could endure not much longer, but for the technical reasons cited above, not the criticism of Mr. Torvalds. I pondered the dilemma back in April. "As Apple Moves From OS X to MacOS, What’s in Store for Users?" My guess is that a solution was already in the works when Torvalds made his comments in January, 2015.
WWDC 2016
After the WWDC keynote on June 13, the titles of some secret sessions were revealed. Notable was a session on Apple's development of an HFS+ replacement, APFS, for "Apple File System." Lee Hutchinson at ars technica wrote the best summary I've seen: "Digging into the dev documentation for APFS, Apple’s new file system." There, you'll find an excellent technical description of APFS if you want to dig deeper.
When Will APFS be Released?
File Systems are tricky to develop. One software bug can lead to the loss of gigabytes of data forever. Customers are particularly touchy about the integrity of their data and have a low tolerance for even the most idiosyncratic file system bugs. And so, Apple is rolling out a pre-beta of APFS to developers this week. It will require a long period of testing and certification before this file system is released to customers as part of macOS and its siblings, iOS, watchOS and tvOS.
It won't be in macOS Sierra released this fall, nor iOS 10. When will it be released? All we know is Apple's target date of 2017.
Even the most curious Mac developer may take pause at the current limits of APFS in its early stages. For example, APFS volumes cannot be used as startup disks or Time Machine backups. Or use FileVault. Data could be lost and likely will be at this early sage in development.
My guess is that APFS will be rolled out gradually on the Mac. First, it will be available to format external drives only. Then, new Macs will ship with APFS on the boot drive, perhaps in late 2017. (iPhones and other devices will follow.) Customers using El Capitan, Sierra or later will be able to reformat and do a clean install with APFS if desired. Every step of the way, starting in 2017, there will be gentle, careful migration and disciplined, cautious testing across all devices.
But when we finally arrive, there will be great rejoicing. Integral to APFS is file level security, and Apple's initiatives with the security and privacy of the iPhone in 2016 will carry over, at last, to the Mac in 2017-18. There will likely come a day when casual Mac users don't even think about not having their Mac and all external drives as highly encrypted as their iPhones and iPads. I surmise that the optional FileVault will be replaced with easy to implement, more fundamental encryption. (But users can still elect 'no encryption' if necessary for, say, thumb drives.)
There will be great joy in Appleville.
_____________________
Teaser image via Shutterstock.
