Inside Mac OS X 10.4.2

The true theme of Mac OS X 10.4.2 —released by Apple in mid-July—is simple: it fixes bugs. Customers found bugs that testers and developers did not, and this update fixes a bunch of them. Mac OS X 10.4.2 was not released just to support new hardware, or to add intriguing new capabilities to the operating system, or to pave the way for mutant iPods that work with iTunes to control your garage door while purifying the air. It fixes bugs, with some of the fixes more clearly documented than others.

The release of Mac OS X 10.4.2 marks the beginning, in earnest, of the Tiger update cycle. Because the development cycle for Mac OS X 10.4.1 started before 10.4 DVDs ever reached retail customers, 10.4.2 is the first Tiger update to address usability issues reported by non-simulated customers. The big themes seem to be networking and minor annoyances, with a secondary focus on iChat and Mail, which always seem to have problems in the first release of a major Mac OS X version.

It’s those changes—along with improvements to Tiger’s disk repair tools—that we’ll focus on during this inside look at Mac OS X 10.4.2. The update also includes fixes and changes to the Library, System Library, Core Services, and Unix folders, which are examined in greater depth in the July 31 issue of MWJ (as is the Mac OS X Server 10.4.2 update). Here are just a few of the highlights:

Applications

iChat AV The 10.4.2 update revises iChat AV, an application that seems to attract problems that survive both testing and seeding. Our testing of iChat in pre-release versions of both Panther and Tiger showed it as one of the last components to feel really solid during extensive daily use, and the videoconferencing support is still fragile for us. It is difficult to verify if the problems are in iChat, the network connection, the AIM software on the other end, the cameras, in one party’s configuration, or perhaps in the relative humidity inside a router somewhere.

According to Apple, Mac OS X 10.4.2’s updated iChat AV 3.0.1 (v392) and the associated public Instant Message framework (v356) fix an “issue” in the previous versions that could incorrectly end audio or video chats due to “insufficient bandwidth.” Although Apple has not confirmed such details, the third-party community believes the problem relates to Quality of Service (or QoS) guarantees that iChat AV tries to use.

The abbreviation QoS technically refers to the entire concept of guaranteeing throughput over a network; iChat AV appears to use DSCP (Differentiated Services Code Point) in TCP and UDP packets to implement QoS guarantees. The idea is that urgent packets—such as those for real-time person-to-person video and audio—should be marked as such so that routers on the Internet can give them the priority they need.

The problem, apparently, is that some routers are so confused by the DSCP information embedded in packet headers that they route these packets slower than other packets. When packets don’t show up as fast as they should, iChat AV thinks it’s because you don’t have enough bandwidth. It tells you so and ends the conference.

Since Apple has not confirmed the problem, there’s very little information available to suggest how Mac OS X 10.4.2 fixes it. We’d guess it has something to do not only with the iChat application, but also with a revised private Video Conference framework. The simplest solution we could muster is trying again: if a conference with QoS information appears to have insufficient bandwidth, iChat AV could omit the QoS information and try again, to see if that confuses a router a bit less. It’s just a theory.

Safari Mac OS X 10.4.2 includes a new version of Safari, but the two documented changes in Apple’s browser are almost certainly implemented in the public Web Kit framework: better scrolling with scroll-wheel-capable mice, and the ability to log into Cingular’s Web-based account system. According to other documentation, Mac OS X 10.4.2 also fixes problems that could prevent Safari from downloading “certain” widgets, and that might have auto-installed some widgets when you did not wish it.

Mac OS X 10.4.2 also includes new versions of the private Syndication and SyndicationUI frameworks that handle Tiger’s RSS download and display capabilities. It might be related to a bug developers knew only as “issue with JavaScriptCore and RSS Visualizer.” Although these frameworks are private, Apple seems to understand that syndication grows more important each day. The company has created a new syndication developers’ mailing list so developers can talk about Safari RSS (and how these frameworks handle RSS feeds), iTunes podcast feeds, the Apple Chapter Tool for making “enhanced” podcasts, and more—just not user-level concerns.

Mail and Others Apple documents several changes to Mail in Mac OS X 10.4.2 that come not only in the revised Mail application in the /Applications folder, but also in the public Message framework (in /System/Library/Frameworks) that implements much of Mail’s back end.

The release notes also mention bugs fixed in Stickies and iCal, as well as a bug in Preview that’s fixed in some framework because 10.4.2 does not contain a new version of Preview. The update also provides undocumented changes to Address Book, Automator (not just to some of Apple’s bundled Automator actions), the Installer (perhaps related to the documented bug installing to write-protected drives), and Printer Setup Utility.

Disk Repairs

Mac OS X 10.4.2 updates Disk Utility (in the /Applications/Utilities/ folder), as well as the /usr/sbin/diskutil command line utility. Both are front-ends to the private Disk Management framework that contains the code for verifying and repairing HFS Plus volume structures and privileges. The framework—and therefore both Disk Utility and command-line utilities like the fsck disk-repair tool—see a major upgrade in Mac OS X 10.4.2 in how they handle files with overlapping extents.

An extent is merely a sequence of consecutive blocks on an HFS or HFS Plus disk. The Mac OS always tries to allocate files in consecutive chunks, so it tracks file segments by extents, identifying each one by its starting block number and the count of the number of blocks in the extent. An unfragmented file, no matter how large, has only one extent because it resides on disk in consecutive blocks. A 12KB file fragmented into three 4KB allocation blocks on disk has three extents, each one block long. Because both HFS and HFS Plus were designed to prevent fragmentation, tracking files by extents is more efficient than tracking each block allocated to a file.

How can extents overlap? Disk corruption often starts with errors in block allocation. Whether by programming error or power outage, corruption begins when the file system uses some blocks on the disk but fails to mark those blocks as used in the allocation file (or, under the older HFS format, the volume bitmap). Sometime later, when the OS needs space to write something, it finds that these blocks are not marked as used in the allocation file, so it uses them, unaware they’re already part of some other extent. Just like that, two files believe they’re using the same blocks on disk.

Suppose that your Safari bookmarks file overlaps a picture in your iPhoto library. If the picture was written last, Safari might crash on launch, unable to parse what it thinks should be an HTML file full of bookmarks that instead has some chunk of binary JPEG data in it. If the bookmarks were written last, then if you’re lucky, iPhoto only complains that it can’t display the picture. If you’re unlucky, iPhoto crashes on launch trying to display a JPEG image with a chunk of HTML where binary data should be, and you get to track down the trouble yourself.

Overlapping extents don’t trigger File Manager error codes that applications can catch and display. A file with an overlapped extent is neither too short nor too long, and the OS doesn’t know or care that the binary data inside the file is incorrect for one of the owners. As long as the HFS Plus code can find, read, and write the blocks, it won’t return errors—it won’t even know that multiple extents claim the same block. Fortunately, any decent disk utility rebuilds the allocation file from scratch as part of examining a volume, looking at every extent and making sure that every block is used once and only once. That’s the only way you’ll ever know if two or more extents overlap.

Finding the problem is easier than fixing it. In fact, it can’t be “fixed,” because no more than the last file to use the overlapped extent could have correct data in it. By definition, if an overlapping extent exists, a program wrote data for a file into blocks that were already used by an earlier-created file. The data from the earlier file in those overlapping blocks is gone forever. Utilities can recognize the problem and untangle the extents by copying all files to new parts of the disk, but no more than one of these rescued files contains correct data. If multiple extents overlap in tangled ways, it’s possible that none of the files has all of its own data anymore.

Mac OS X 10.4.2’s revised Disk Management framework significantly boosts Tiger’s ability to repair disks with overlapping extents. In 10.4.1 and earlier, fsck (and Disk Utility’s log) would merely report that overlapping extents exist, displaying the affected file’s ID numbers (catalog node ID numbers, technically, or CNIDs) as part of the error message. You then had to use the find command’s “-inum” parameter to find the real pathnames for yourself. In 10.4.2, the system displays the pathname of a damaged file as well as its CNID.

That’s just a start. Mac OS X 10.4.2’s HFS Plus repair code attempts to allocate new space for every shared extent. If two files share the same three blocks, the code now allocates two new three-block extents on disk, copies the contents of the shared extent to both of the new extents, and updates the catalog so that the two original files now refer to their new, individual extents instead of to the older, illegally shared extent. Apple warns that this may fail or succeed silently, but unless your disk is either so full or so fragmented that the code can’t allocate the new extents, it should succeed.

Next, the new Disk Management repair code creates a folder named DamagedFiles at the root level of the disk in question, and creates a real Unix symbolic link (not a Mac OS alias of any kind) for each of the affected files, named with the original CNID and the file name. In our example, if your Safari Bookmarks file had the file ID 45892 and the picture that overlapped an extent with it had file ID 81834, the DamagedFiles folder would contain two symbolic links, one named “45892 Bookmarks.plist” and one named something like “81834 DSCN0001.JPG”, depending on the image’s original filename. As noted, only one of these files can even possibly contain correct data, and they may both be corrupted. Thanks to the symbolic links, you can easily find them, and determine which newly untangled files to keep and which to replace.

Utilities like DiskWarrior have long repaired overlapping extents with similar procedures, but it’s great to have this built into the operating system, if for no other reason than that Mac OS X uses this code to verify and repair your file system during startup when the computer wasn’t properly shut down. If you see a DamagedFiles directory show up on the root level of your hard drive, stop what you’re doing and look at the files within it. See which of the files are damaged, and either replace them from backup or recreate them. Then run DiskWarrior if you have it, or boot into single-user mode and run fsck so it can verify there are no more problems with your disk. (You can boot from your Tiger installation disk and run Disk Utility, but since it doesn’t contain the Mac OS X 10.4.2 version of this code, it won’t do as well with overlapping extents, so we recommend single-user mode first.)

The Last Word

Although Mac OS X 10.4.2 takes a big swipe at user-reported problems with the first two Tiger releases, it’s not perfect. A bug that wipes out URL information in Address Book when using synchronizing and using .Mac data as the master source is not fixed in 10.4.2, for example, nor are several other bugs.

The update did not address an iPhoto 5 distraction that was embedding the wrong color profile in an image every time you edited it under Tiger, but the iPhoto 5.0.3 update released shortly after Mac OS X 10.4.2 fixes that. We’re not sure why it requires Mac OS X 10.4.2 as well, because nothing related to ColorSync appears to have changed in the OS update, but there you go. Unfortunately, iPhoto 5.0.3 crashes when rotating some images, prompting the quick release of iPhoto 5.0.4.

Mac OS X 10.4.2 has been in the wild for a couple of weeks, and no one has reported any massively debilitating bugs for it. There will be more bug fixes in Mac OS X 10.4.3, and 10.4.4, and 10.4.5 beyond that. Jaguar got eight significant revisions; Panther got nine (and that might not have been enough). The 10.x.2 update usually arrives about two months after the major release, and 10.4.2 is in line with that schedule.

In other words, there’s nothing really unusual going on here. It’s a big update, but so is every 10.x.2 update, because that’s when the customer-reported bugs start getting fixed.

If you’re running Tiger, install Mac OS X 10.4.2 if you haven’t already done so. If you’ve been waiting to install Tiger until they get “the first bugs” worked out, it’s time to put that reasoning aside and update. It’s not an earth-shaking update. It’s just one you want.