Inert Detritus The Internet's dust bunnies

Hacking iScrobbler for multiple iPods

I’ve been on Last.fm since 2004. I love music, and I love sta­tis­tics and data analy­sis, so when the site launched and I saw what they had planned, I imme­di­ate­ly set up an account.

I’ve long used iScrob­bler for a Mac client to sub­mit plays to Last.fm: it’s light­weight, sup­ports run­ning in a menu bar-only mode, and has giv­en me zero grief over the years. iScrob­bler watch­es iTunes to detect and scrob­ble¹ songs played as they hap­pen. This obvi­ous­ly isn’t pos­si­ble with iPod plays, so iScrob­bler asks you to use a Recent­ly Played playlist that it watch­es: after you sync an iPod, iTunes updates that playlist with all the tracks played while you were out, and iScrob­bler diffs the playlists to know what to submit.

Until this sum­mer, I used both my 2nd gen nano and my iPod pho­to to play music: the pho­to fit near­ly every­thing I had, while the nano was great for the gym. I now use the nano, my iPhone 3G, and an iPod clas­sic for play­ing music on the go, so my #First­World­Prob­lem of mak­ing sure all iPod² song plays get scrob­bled to Last.fm hit a bit of a speed bump. I start­ed inter­leav­ing plays on the devices: I’d lis­ten to my iPhone on the bus ride in, use the nano at the gym, then lis­ten some more on the iPhone before sync­ing them back to iTunes.

To sup­port the iPod, iScrob­bler takes snap­shots of the iTunes Library, and com­pares the Last Played and Play Counts in a playlist you con­fig­ure³ to fig­ure out what songs you lis­tened to.

After an iPod update, iScrob­bler will take the last played time of the last played song, sets this as the new “iTunes Last Played Time”, and scrob­ble every­thing played since the pre­vi­ous iTunes Last Played Time.⁴ With inter­leaved plays, this quick­ly becomes a prob­lem: when you sync iPod A, iScrob­bler bumps the iTunes LPT; any plays on iPod B that hap­pened before that time are now “lost”, since that iPod has­n’t been synced, and they fall before that time­stamp. Those plays are very dif­fi­cult to get back lat­er, so this is a prob­lem worth avoid­ing⁵

Playlist snap­shots let iScrob­bler work around an iTunes/Last.fm prob­lem: iTunes only stores the Last Played time, but Last.fm wants a time­stamp for every song play. If iScrob­bler sees a song which incre­ment­ed Play Count by more than 1, it has to “syn­the­size” the play. The song must have a time­stamp for Last.fm sub­mis­sion, so iScrob­bler finds gaps in the play times where no songs were play­ing, and invents rea­son­able time­stamps for any songs need­ing a time.

So, for the past few months, I’ve been clos­ing iScrob­bler before sync­ing my iPods. This let me sync both iPods with iTunes, which updat­ed all the songs’ meta­da­ta, then launch iScrob­bler and tell it to man­u­al­ly update from the iPod playlist.

A few weeks ago, instead of just clos­ing iScrob­bler before an iPod sync, I start­ed run­ning iTunes with iScrob­bler com­plete­ly closed. All my plays on the desk­top go into the same playlist⁶ as any tracks my iPods play, so they would get picked up like the rest of my iPod plays.

iPods don’t cross-fade play­back, so the Start Time of one song⁷ per­fect­ly adjoins the Last Played time of the track before it. Last.fm is very par­tic­u­lar about track lengths and time­stamps, and won’t let you sub­mit plays that over­lap, so if that hap­pens, iScrob­bler drops the sec­ond track to avoid API issues.

The writ­ing on the wall is clear: if you have cross-fade play­back enabled in iTunes and play songs with iScrob­bler closed, 50% of your desk­top plays will be dropped by iScrob­bler as over­lap­ping. I haven’t filed a bug against the API on this yet⁸. You can work around it by turn­ing off cross-fade play­back in iTunes; that makes iTunes behave iden­ti­cal­ly to iPods with play­back, and you can keep iScrob­bler closed for days at a time with no adverse effects.

Post­script: the iPhone 2.0 bug where Play Counts and Last Played times weren’t updat­ed just about killed me. I had to exhib­it extra­or­di­nary self-con­trol to only play music on the nano, since it was the only device that prop­er­ly updat­ed the song metadata.

Yes, I think a play count will fall through the cracks if you play it once on both devices; iTunes will crush of the device’s play count with the oth­er’s. I haven’t test­ed this to con­firm or deny it, however.

1. Sub­mit, in iScrobber/AudioScrobbler/Last.fm speak [↩]
2. For the sake of brevi­ty, “iPod” includes iPhones. [↩]
3. This is best set to a “Recent­ly Played” smart playlist [↩]
4. It does this so takes less time and work to avoid dupli­cat­ing sub­mis­sions. Dupli­cate sub­mis­sions are bounced by the Last.fm API with great prej­u­dice. [↩]
5. I typ­i­cal­ly resort to leav­ing a playlist run­ning overnight to fake them. [↩]
6. “Recent­ly Played” cap­tures any­thing played in the last few weeks [↩]
7. iScrob­bler always has it, so iTunes must be book­keep­ing it some­where. As far as I can tell, this is stored behind-the-scenes and not exposed in the iTunes UI [↩]
8. There needs to be a few sec­ond “grace peri­od” for device clock skew and cross-fad­ing play­back [↩]

Thanks C‑note. I will be try­ing this very soon.

I use an iPhone and iPod nano with iScrob­bler. Here’s what I do. From menu bar, select iScrob­bler, then “Pause Sub­mis­sions.” Sync the iPhone. Dis­con­nect. Go back to menu bar, and “Resume Sub­mis­sions”. Sync the iPod nano. When that fin­ish­es, iScrob­bler uploads the union of iPhone and nano’s com­bined Recent­ly Played.

I nev­er knew whether I could trust “Pause Sub­mis­sions”, so I’d nev­er tried it. That’s very good to know, thanks, I’ll give that a try.

A post by the iScrob­bler does­n’t seem to back that up: it appears that but­ton can be used to silent­ly drop plays.

Thanks for shar­ing this. I won­der how many last.fm scrob­blers are as fanat­ic. I’ve spent a lot of time obsess­ing over mak­ing “every play count,” although I’ve mel­lowed enough to not grieve over lis­ten­ing to CDs in the car anymore ;)

This is very infor­ma­tive post. I love this and I will check your blog again.