Technical Analysis: The iTunes Broken Promise: Broken Links

| Editorial

One of the reasons we are fans of Apple is that an implicit proposition of being a customer is that our computing life will be simpler, less vexing, and more productive. With iTunes, however, that promise is broken in part.

We know that iTunes is a money machine for Apple. Accordingly, development resources are going to be applied first to the maintenance and improvement of its ability to secure the revenue stream. Unfortunately, when it comes to the hard work of making iTunes a blissful and reliable application, efforts have been lacking. The area of focus here is the iTunes broken link.

iTunes glamor shot

iTunes

It's not the cause of the issue, but a broken link is related to the fact that iTunes keeps a database, in XML format, of your iTunes library. (Actually, it also creates a binary version for faster access, but will revert to the .xml file if necessary.) It contains all kinds of useful metadata, such as song rating, lyrics, cover art and play count -- plus a pointer to the actual song. You'll see this file in your iTunes directory: "iTunes Music Library.xml."

Separate from the database itself is the actual library of songs. That's in a directory called iTunes Music, and that directory can be moved -- so long as you tell iTunes where you put it. Say, an external drive. You do this in iTunes -> Preferences -> Advanced -> iTunes Media Folder Location. See the link above for other housekeeping details.

This schema, in principle, should work fine, but the opportunities for both unexpected user actions, difficult Time Machine file recoveries, and lack of iTunes software robustness can lead to a situation where the iTunes database no longer knows where the actual song is -- even though it's easy enough for the user to find it with a simple Finder search. The broken link is indicated with an exclamation mark, like this:

iTunes broken links

Broken Links

Apple's first line of defense against this is an almost DOS-like, pathetic avoiding of responsibility by iTunes engineers. When you double click a song that has a broken link, iTunes invites you to go look for it, manually, with a File dialog box. That's not the user experience Apple is famous for.

iTunes Find song

The cop-out leads to File dialog...

If you have a few of these, they're easy to fix. If you have a hundred or more, the process can become tedious. Right away, you'll find yourself looking for an iTunes menu item that can do the tedious work for you. In principle it would look like this:

File-> Fix Broken Links
   -? Please specify the the location of the iTunes Library
      -Code -> Finds all the songs w/ broken links and mends the link.
      -? Asks for confirmation if in doubt.

 

If there's any doubt, the function can throw up both the metadata and the path information to confirm the association. Such a function is missing in iTunes and remains an arrogant testament to the notion that Nothing Can Go Wrong. (Or Apple engineers didn't have the time to write the code. Or the product manager doesn't want to admit that there's a problem that requires attention.)

The next thing the user may do is search around on the Internet for a utility that does this dirty job for them. Alas, neither Jeff Gamet nor I have been able to find one that specifically addresses this limited problem. A search will come up with TuneUp from Tuneupmedia.com and an AppleScript written by Jennifer Croft. While encouraging at first, it doesn't appear that either of these solutions addresses the specific task described in the hypothetical menu function above. Instead, they drag the user into a blind alley of false hope, other functions, then frustration.

The third option is to manually fix the broken links as they arise or all at once if one has the patience. I did this by having a Finder search window under iTunes, searching for the song name, then using the path info at the bottom of the search window to guide me to the song when iTunes asks for the song location. In the screen shot below, I've overlapped the two windows to make the image bigger and easier to read.

iTunes + search

iTunes & Search windows overlay


iTunes search magnified

Magnified view of above

One of the impediments to an easy solution is that, to fix the link, one has to carefully edit the .xml database file. One can do that with a command line editor, like pico, nano, vi, etc. But requires a bit of knowledge about the structure of the XML and is a technical hurdle that would keep most users from easily writing their own utility. Also, the technology of iTunes changes rapidly over time. Accordingly, the heavy lifting here should be done by Apple, not the user.

A search of the Apple and MacOSHints user forums hasn't revealed any insanely simple solutions. However, there's no guarantee that I haven't overlooked something, and reader feedback is welcome.

There are many complexities and opportunities for problems with the schema Apple has chosen. For some users, the only sane solution is to blow out the iTunes support files and start all over with iTunes -> File -> Add to Library. This will rebuild everything, but will also reset all the metadata you've built up, such as play count, rating and lyrics. For those who don't care about that, it's another painful but workable course of action.

iPhone Sync

Another way in which the user can become frustrated by broken links is when syncing playlists to an iPhone or iPod touch. When there are broken links in a playlist to be synced, iTunes will throw up a dialog box like this:

iTunes sync error

iTunes sync warning

The issue here is that there's absolutely no way to print, select, or copy this list -- which can be long. The best the user can do is scroll through the list. Those who would like to attend to broken links, first, for their iPhone will not be able to use that dialog box as a helpful basis. Instead, one must identify all playlists to be synced, then clean up those playlists, and finally re-sync the iPhone.

Summary

iTunes is a fabulous product, but somehow, along the way, Apple has seemingly failed to roll into the product a first class schema for robustness, error correction, and a user-friendly menu item to do the work only a computer should be doing. Instead, in a cop-out, iTunes asks the user to do all the grunt work of finding and selecting songs when the link is broken. To fulfill Apple's implicit promise to make our life better, something more capable than that is called for.

As I said above, reader feedback is most welcome here, especially in the area of automated tools to fix an abundance of broken links or better workaround procedures. If our community comes up with an elegant solution, I'll post a follow-up article.

Sign Up for the Newsletter

Join the TMO Express Daily Newsletter to get the latest Mac headlines in your e-mail every weekday.

13 Comments Leave Your Own

Rick

A couple other things I don’t like about iTunes (please correct me if I’m wrong about these):
1. There’s no library-wide search (since about iTunes 7?). Occasionally, podcast files or audiobook files are filed as music, which means that a search in podcasts that doesn’t throw up any results and has to be repeated in any other sub-libraries where they might be.
2. You can’t do anything with lyrics, such as having them auto-scroll as a song plays or come up full-screen. And the window is modal, so you have to close the lyrics window if you want to do anything else in iTunes.

CCardona

Thank you! FInally a wake-up call to Apple! This problem can especially hit you hard when you move your database to another drive.

The only protection is to export your Library, and then each individual playlist as an XML file!! For those of us with dozens of playlists, it becomes a real hassle, but at least then you can “put things back the way they were” before iTunes lost it’s way.

jorisx

Yes, this is a real frustrating issue,
After a few years of building a big library, data got messed up
folders inside my iTunes library got moved and the whole thing broke down

Last and only solution, I had to re-create a new library from scratch
shame.

the should fix this pronto smile

geoduck

Very good!!
They’ve been tarting up the front end while forgetting the hidden parts that are very much showing their age. It’s the digital version of Joan Rivers.

While we’re in b***h mode here’s mine: the iTunes store. It’s great if you know the name of what you’re looking for but if I don’t happen to know the name of the song or artist, or maybe I just want to search for something generic (let’s see what Anime they stock now let’s see what Adult Swim or Nickelodeon is stocking), it’v very hard to do. A couple of revs ago they hid many of the options I used every visit. Going to TV used to include a list of the networks. I have a devil of a time finding this now. Do a search for Anime brings up Podcasts, and songs, and such but no TV shows. But I know that particular networks have them However doing a search for Adult Swim for example brings up Podcasts and Apps but none of the shows.

It’s a mess. It’s bad enough I don’t buy as much from iTuns Store as I likely would.

geoduck

While we?re in b***h mode here?s mine: the iTunes store./quote]
Never mind. I just went back and though it’s down the page a bit I found what I was lookingfor: network listings and such.
(I gotta remember to stop posting in the afternoon when I start work at 5;00am. I’m just too fuzzy.)

fudfud

This has been a long time annoyance, thanks for writing about it.

Nothing like scrolling through your whole iTunes library looking for all the “!” after an almost worthless “sync warning” dialog.

I always wanted to sort the left iTunes column containing the “!”, by clicking the heading, so they would appear together at the top. Seemed like a simple fix to at least find the songs with broken links.

With every new version of iTunes I would try to sort that left column…no such luck, and now there is the triangle for album view.

Maybe this will help motivate Apple to put some effort into broken links.

Tiger

Just curious, but do any of you use the Apple feedback options? I’m not being snarky, I’m really interested to see if you have and if you’ve ever seen something change…eventually.

I have used it for years to provide valuable information, but rarely do I see any result. I have had better luck submitting ideas through customer service reps and System Engineers working for Apple than through their feedback system (which is here http://www.apple.com/feedback/)

Drew

Correct me if I’m wrong. But this really should only affect someone whose iTunes library is bigger than a single hard drive. Hmm, Limewire much?

I have used iTunes since Day 1 on at least 7 different computers and have never run into broken links.

I simply move the whole iTunes folder around as a unit and never mess with it internally in the Finder.

Josh

Congratulations Drew!! You’ve been lucky enough to avoid this problem.  However, that doesn’t mean it isn’t a problem for others.  Furthermore, just because your collection is small doesn’t mean that my large collection is illegal. 

My wife is a musician and vocal, piano & violin instructor.  She has literally THOUSANDS of legally purchased CD’s.  We ripped them all to a computer I’m using as a server for the sake of convinience, and she streams music to the laptop in her studio when ever she needs something.  (the iTunes Home feature from the last update was a god send)

We’ve run into this problem hundreds of times.  It usually happens when I’ve had to move the library to a larger hard drive as it grew (it currently spans 3 fairly large HD’s when you include the movies and TV shows I’ve ripped from the DVD’s that we own.  No illegally downloaded material at all).  Even when I had a small collection I ran into this problem, more frequently with the CD’s that were in worse shape to begin with or were originally ripped on my 400 Mhz iMac DV SE back in college.  There was something wrong with the optical drive and tracks from the ends of CD ended up being easily corruptible for some reason.

Rick

While we?re in b***h mode here?s mine: the iTunes store./quote]
Never mind. I just went back and though it?s down the page a bit I found what I was lookingfor: network listings and such.
(I gotta remember to stop posting in the afternoon when I start work at 5;00am. I?m just too fuzzy.)

You’re not alone, geoduck. Someone on MacBreak Weekly the other week was complaining about the same thing. Even though the options you were complaining about are there, I still think they should offer more flexible ways to browse. Sometimes the breadcrumb trails they give you don’t make a great deal of sense and don’t help much when you want to “zoom out” from the thing you’re looking at now.

GlennG

Has anyone tried this one?
Super Remove Dead Tracks v2.1
Surveys iTunes for tracks whose files are no longer available (as indicated by a “!” next to their names) and removes them. Any playlists that subsequently contain no tracks will then be deleted. Progress is displayed every 500 tracks.

http://dougscripts.com/itunes/scripts/ss.php?sp=removedeadsuper

John Martellaro

Surveys iTunes for tracks whose files are no longer available (as indicated by a ?!? next to their names) and removes them.

Yikes.  Not what we want.  That would presumably delete the pointer to the song, leaving it orphaned. We want to get the two connected.

playfans

I totally second that!

We’re close to the 10th version and iTunes still lacks such a huge feature! I was using musicmatch before iTunes and I can tell you that I discovered what broken links were with iTunes! Musicmatch had a tremendous “watchfolders” functionnality. And it is the only feature iTunes really need to implement in the next release.

Log-in to comment