Inert Detritus The Internet's dust bunnies

21 March 2010 @ 11am

Comments Off on On Ice

On Ice

I’m shelving this blog in favor of The Confusatory over on Tumblr. The blog has had a good run, but it’s too much to manage: I have to log in every few days to check for WP updates; I have to manage backups of the MySQL tables and the blog itself; etc etc. Managed software beats home-grown maintenance for convenience. There’s also something about Tumblr that’s much more conducive to short-form posts, links, and the occasional longer post.

In any case, point your feed readers that-a-way, and I’ll see you around the intertubes.

14 February 2010 @ 12am

Comments Off on Briefly: Network Topologies

Briefly: Network Topologies

Things I learned today: network link-layer topology makes a huge difference for wireless.

My file transfers via 802.11n from one laptop to another were pretty slow: max sustained throughput was 3–4 MB/sec. As an experiment in solving unrelated AirPort base station association issues, I plugged one laptop (that’s already wired up to the TV as a media center of sorts) into the base station via GigE. Because the AP no longer has to negotiate with two clients on the same slice of spectrum for network traffic, my inter-machine transfer rate jumped to 15–18 MB/sec.

Note that this only applies when the laptop is associated to the base station that the other client is wired to. Since I have both a Time Capsule and an AirPort Extreme, when the wireless laptop roams to the Time Capsule, its link to the Time Capsule is contending with the Time Capsule <> AirPort Extreme wireless link, and the speed falls off again.

The network layout looks something like this: 

3 September 2009 @ 12am


SSDs and you: my dual-drive setup

I’ve been running a dual-drive setup on my new MacBook Pro for the past two months. Here’s a bit of info about how I’ve got it set up, and how it’s working for me

A quick introduction to SSDs

The vast majority of a HD’s time is spent traveling from bit A to bit B: reading or writing small amounts of data is fast, but getting the read/write heads to where it needs to be takes forever, in computing terms.1 Sets of bits that are farther apart on the platter take longer to reach: for example, duplicating a file on one disk takes much longer than copying it to another disk, because the heads have to move to where the file is, read some bits, move to where there’s free space, write some bits, and repeat hundreds of times.

SSDs have a constant access time, regardless of what bits you ask it to read or write: reading two physically adjacent blocks of a song is just as fast as reading a block of data for an application followed by a block of data for a download.

It doesn’t matter if an SSD beats a spinning platter HD by inches or miles for sequential read and writes: the real magic is the throughput and speed of random reads and writes.

Only big media files, like pictures, music, and videos, can typically be sequentially read and written.2 The rest of the system, from OS libraries to application bundles and preferences, are scattered across the disk. When you launch an app, OS X tries in a very short amount of time to read the app’s binary, open any libraries the app needs to run, load any additional app resources it needs, and read its preferences and other application data. None of those bits sit near each other on the drive, so it takes a lot of back-and-forth hard drive seeking to get everything read in and ready to go. SSDs don’t have to pay any time penalty for randomly choosing two bits to read, so app launches are lightning fast.

The setup nitty gritty

I did this to my mid-2009 15″ MacBook Pro. I upgraded the built-in HD to a 500 GB, 7200 RPM Seagate drive. I bought an Optibay3 and an 80 GB Intel X25‑M and installed them myself. The install was easy: take 10 screws out of the bottom case to expose the guts of the laptop, and remove three more inside to swap out the optical drive for the SSD carrier. Close it back up, format the drive, and you’re ready to go.

Why two drives?

I wanted an SSD for the ludicrous speed increases, but I have a few hundred gigabytes of pictures, videos, and music, and no one manufactures (affordable) 500 GB SSDs just yet. Instead of moving lots of that media off the machine and onto an external drive, I’ve split my filesystem setup between the two drives. I moved as many of the more-randomly-read bits onto the SSD as possible (the base OS, /Applications, and especially ~/Library), and kept my user’s Music, Movies, Pictures, Desktop, and Downloads on the spinning platter drive, where bit storage is cheap and longer seek times don’t matter. By keeping the entire OS and the vast majority of my user’s home directory on the SSD, boot times, login times, and app launch times are all largely the same as an SSD-only system, but I still have plenty of capacity for large downloads of videos, pictures, or music, and can still work and use all my media while on the go. It’s the best of both worlds.

Dragons ahead

Getting an already-installed OS properly copied to the SSD was tricky: I hacked a SuperDuper script to copy over my 500 GB boot volume, minus the above paths and a few others, and symlink over what I didn’t copy. It was tricky, and this is a good time to make sure you’ve got an up-to-date SuperDuper backup of your boot volume somewhere: I had to restore the 500 GB drive from backup once when I typo’d a path and it thought I meant /Music and not ~/Music.

A special note for anyone wondering about how OS X installs handle a dual-drive setup: I haven’t tried a Leopard Archive and Install on this machine, but I can’t think of any reason it wouldn’t work, as long as you check the box to preserve Users. On Snow Leopard, I’ve done several OS upgrade installs since setting this up, and have had zero problems.

Results! I demand results!

I initially didn’t copy my user over at all, but the system wasn’t much more responsive than a spinning-platter setup: though apps and the base OS were on the SSD, there was still lots of drive thrashing when loading preferences and application data. The biggest speedup in login and app launch time came from putting ~/Library on the SSD: ~/Library/Application Support and ~/Library/Preferences are hit heavily by apps on launch. I also moved over all of ~/Music except for the actual iTunes Music folder: having the iTunes Library file on the SSD greatly improved iTunes launch times and song metadata updates.4

Right now, only a few specific applications access the 500 GB drive: iTunes (and only when playing songs), Aperture, and any copy operations to/from ~/Desktop or ~/Downloads.


Since most of the files used by my system are on the SSD, sudo pmset -b disksleep 1 is my new best friend. This Terminal command sets the disk sleep timeout while on battery to 1 minute: most of my time on battery is spent with the SSD powered and the Seagate drive spun down. I’ve seen a big improvement in battery life the past weeks as a result.5

Was it good for you too?

This configuration was a bit tricky to get set up, but it was very much worth it. OS boot, login, app launches…almost every daily operation on the machine is lightning fast,6 and I didn’t have to make any compromises with what music or photos I keep on the machine. From now on, I’ll be making sure that any machine I use day-to-day as my primary system is running on an SSD.

  1. HD seek and rotational latency times are usually measured in milliseconds: a ten millisecond seek time burns 20 million CPU cycles on a 2 GHz processor. Entire empires can rise and fall in the time it takes those platters to spin and the read/write heads to move []
  2. assuming no file fragmentation on the drive []
  3. Fraser Speirs got a MaxConnect kit for his setup []
  4. the computer seems to write changes to the library file at the same time as the actual media files. Having them on separate drives eliminated the typical disk thrashing this causes []
  5. If you install the Developer Tools, check out Spindown HD in /Developer/Applications/Performance Tools/CHUD/Hardware Tools/ for an easy way to know if the drive is asleep or spinning. []
  6. even OS reinstalls! []

1 September 2009 @ 11pm

Comments Off on Switching databases with Things

Switching databases with Things

[September 1st: updated for Things 1.2. See bottom section for details.]

I use Things for managing tasks at work, and for tracking my personal to-do list. I didn’t want to mix the two up, however, so I use two Things libraries.

To swap libraries, you need to hold down option at program launch, and choose your other library. But, since this is OS X, defaults read and defaults write can help us automate this process a bit: Cultured Code is simply saving the path to the file in their preferences.

I wrote a small perl script to toggle between my two libraries, and it’s available here. Save that to, chmod u+x it, and save it somewhere in your PATH (I personally prefer ~/bin). You’ll need to change "/PathTwo" to your other library location, and my $USERNAME = ''; to something useful.

I was unable to open -a from within the script, so my shell alias to quit Things, swap libraries, and relaunch the app is alias && open -a

As of Things 1.2, I was getting an “Upgrading Things library” dialog on each swap-and-launch cycle. Cultured Code added another property list key, “SpotlightIndexedLibraryPathKey”, which needs to be updated once both your libraries have been converted for Spotlight indexing. Adding another defaults write inside the if() blocks in the perl script to write that key as well took care of it for me.

27 August 2009 @ 9am

1 Comment

CouchDB, and achieving consistency

What happens when you change the same document in two different databases and want to synchronize these with each other? CouchDBs replication system comes with automatic conflict detection and resolution. When CouchDB detects that a document has been changed in both databases, it flags this document as being in conflict, much like they would be in a regular version control system.

This isnt as troublesome as it might first sound. When two versions of a documents conflict during replication, the winning version is saved as the most recent version in the documents history. Instead of throwing the losing version away, as you might expect, CouchDB saves this as a previous version in the documents history, so that you can access it if you need to. This happens automatically and consistently, so both databases will make exactly the same choice.

CouchDB: Eventual Consistency. [emphasis added]

Every application that does syncing should support some notion of “revisions”.

It takes only one sync algorithm mistake or a bad conflict resolution approach to suffer data loss. As more apps become “revision-aware”, like Dropbox does so well, there’s many more novel things to be achieved with it, and this is one of them.

8 August 2009 @ 11pm


Obsessive Completionism

I feel compelled to consume every last bit produced in certain digital domains. Twitter users and RSS feeds are my latest vices; before that, it was Twitter and podcasts, and before that, it was podcasts and issues of The Economist. And every yeah, the SxSW showcasing artists torrents interrupts my music listening as I slowly go through it.

I’m not sure where this drive comes from. I suspect it lies in an irrational extrapolation, a strange assumption about quality of content: all past and future content will be as valuable, funny, or interesting as what I’ve thus far consumed.

Over the past three years, I’ve slowly advanced through different stages of dealing with this. Initially, I simply threw time at the problem, dedicating more and more time to reading in order to stay “caught up”. Next, I reluctantly pared back sources, as a lack of time or energy forced me to choose between selective permanent bankruptcy (unsubscribing) or wide-spread temporary bankruptcy (mark all as read). For whatever reason, I was more comfortable making an unsubscribe decision than a mark-all-as-read decision, possibly because it was easier to make a permanent judgment on a single site, relegating them to the dustbin, rather than tossing out perfectly good news items simply because their date of publishing fell on the wrong week. RSS examples are easiest: the first sites to go were the high noise, high volume, low signal feeds like Digg, Boing Boing, and Slashdot.

As I got my RSS problems under control, Twitter picked up steam, and I began following more and more people. I’ve now gone through two Twitter purges, each of them only cutting back from around 300 following to 250. I find it difficult to drop people one at a time, but like spring cleaning, the sheer momentum of the first few unfollows puts me into a much more objective mood. Twitter’s new following list UI makes this much, much easier, showing you the person’s last tweet: this often reminds you of just why you’d been meaning to unfollow them.

Lately, RSS has been easier to make these decisions with, and, ironically, I’ve found that stepping away for a few days and taking a look at your aggregator makes this exercise easier. Don’t read anything for a week; sort your feeds by most to least unread items, and ask, “Do I really need to read all 300 items? How many good links or articles will be in there? 3? 30? 150?”

The most important thing I’ve learned about this “obsessive completionism” is to know that it’s going on: it forces tough decisions about feed sources much sooner, since you know that the volume will eventually be overwhelming.

27 July 2009 @ 12pm

1 Comment

A few words about Fever

Shaun Inman’s Fever has been out for a little over a month, and I’ve been using it full-time since its release.

I love it. By its very nature, a web-hosted app means never having to say, “I’m syncing”. When I close my laptop at home, walk to the shuttle, and open Fever on my iPhone, it’s up to date instantly, with no read status sync problems to worry about. This is nothing against NetNewsWire: syncing is hard, and I never got a two machine setup to consistently work for me.

I use Fever’s aggregation feature1 only when I’m a few days behind and want a quick summary of what’s being written or linked to.

The aggregation filter works best with a lot of feeds: give it a wide base of feeds to start looking for signal, and it’ll have no problem pulling together a good list of that day or week’s most heavily linked items. I’m not sure how well it would work for someone with a less insane number of feeds, but that can be countered by adding more feeds to Sparks to help things along. I’m subscribed to about 300 feeds, and have another 20 in Sparks, to attach some numbers to what I’m talking about.

I do miss some features of real desktop applications: a local cache of article text would be nice. NetNewsWire stored article text and links locally, but Fever, being a pure web app, doesn’t do this. It’s hard to use the site on the shuttle (a high latency, low bandwidth connection), and it’s obviously impossible to read completely offline.2

The iPhone-compatible version of the site is well-designed and easy to use, and is lightweight enough to use on EDGE. Going to a site to read an article and getting back to the reader is dumb easy: you’re already in Safari, so there’s no app switching to mess with.

A lot of people are reluctant to try Fever, either because they don’t have hosting set up, or there’s no way to try it for a week or two before buying. I already had hosting and a couple of domains with Dreamhost, so setting up the application was easy. If you don’t have hosting, I don’t know if it’s worth noodling with a locally hosted setup: it’s probably possible, but I can’t imagine it’s easy to configure all the pieces you need present: PHP, Apache, and MySQL, at the very least.

Long story short, I love the lack of cross-machine sync, and I miss completely offline reading. If you’re a happy NetNewsWire user that uses more than one client in a week, you should check it out. If you’re a single machine user, and don’t have domain hosting already configured, I don’t see much of a reason to try it.

  1. “Here, let me aggregate that for you” []
  2. A single suggestion for future features: I want a Gears/HTML 5‑compatible version, where it pre-downloads article text and links. I’d be ok not supporting a fully-offline mode, since this puts things back into a, “how do I sync read status?” situation like most desktop apps, but if you pull text and links from local storage, and only transmit read status changes over the wire, the site will work well even in the most bandwidth-constrained situations. []

1 July 2009 @ 7am

Comments Off on Formatting digital text for ebooks

Formatting digital text for ebooks

Protip: don’t put anything in between me and the beginning of your ebook. Credits, info, dedications: all that junk should be at the end, not standing in between me and your first real words.

31 May 2009 @ 11pm

Comments Off on Quickly, with feeling

Quickly, with feeling

just so we’re all on the same page:

This place,, is for tech and social commentary (see posts on Twitter, Russia, et cetera).

Posting frequency: maybe once a month; things bake as a draft for a long, long time before they pop out.

chyrp is my personal blog. Thoughts, self-analysis, vague hints at angst, et cetera. Be careful subscribing: I’m holding very little back, and you may find yourself referred to in a roundabout fashion one day.

Posting frequency: updates whenever the hell I get around to writing something.

The Confusatory is my Tumblr playground. Very little original content there, mostly just links to interesting photos/writing/music.

Posting frequency: updates 3–4 times a week, when I’m on top of things. Might go a week or two without anything if I’m snowed under on newsfeeds and tabs in Firefox.

13 May 2009 @ 8am

1 Comment

Excuses for @replies

Addressing this briefly before anyone else starts echoing nonsense that @chockenberry said earlier.

Twitter can’t possibly have dropped the @reply choice (show all @replies, or show only @replies to users I follow) in the interest of load management:

It takes more work to take my list of followees, take each @reply post that one of my followees writes, and compare it to the entire list to decide whether or not to drop it from my stream. It takes more work, not less.

(Edit, 11:30 AM):

Twitter just posted a response on their blog, part of which said:

We’re getting a ton of extremely useful feedback about yesterday’s update to Settings. The engineering team reminded me that there were serious technical reasons why that setting had to go or be entirely rebuiltit wouldn’t have lasted long even if we thought it was the best thing ever.

Now I’m curious: I don’t think they’re lying, but I’m wondering what issues they’re facing.

 ← Before