Inert Detritus The Internet's dust bunnies

On Ice

I’m shelv­ing this blog in favor of The Con­fusato­ry over on Tum­blr. The blog has had a good run, but it’s too much to man­age: I have to log in every few days to check for WP updates; I have to man­age back­ups of the MySQL tables and the blog itself; etc etc. Man­aged soft­ware beats home-grown main­te­nance for con­ve­nience. There’s also some­thing about Tum­blr that’s much more con­ducive to short-form posts, links, and the occa­sion­al longer post.

In any case, point your feed read­ers that-a-way, and I’ll see you around the intertubes.

Briefly: Network Topologies

Things I learned today: net­work link-lay­er topol­o­gy makes a huge dif­fer­ence for wireless.

My file trans­fers via 802.11n from one lap­top to anoth­er were pret­ty slow: max sus­tained through­put was 3–4 MB/sec. As an exper­i­ment in solv­ing unre­lat­ed Air­Port base sta­tion asso­ci­a­tion issues, I plugged one lap­top (that’s already wired up to the TV as a media cen­ter of sorts) into the base sta­tion via GigE. Because the AP no longer has to nego­ti­ate with two clients on the same slice of spec­trum for net­work traf­fic, my inter-machine trans­fer rate jumped to 15–18 MB/sec.

Note that this only applies when the lap­top is asso­ci­at­ed to the base sta­tion that the oth­er client is wired to. Since I have both a Time Cap­sule and an Air­Port Extreme, when the wire­less lap­top roams to the Time Cap­sule, its link to the Time Cap­sule is con­tend­ing with the Time Cap­sule <—> Air­Port Extreme wire­less link, and the speed falls off again.

The net­work lay­out looks some­thing like this: 

SSDs and you: my dual-drive setup

I’ve been run­ning a dual-dri­ve set­up on my new Mac­Book Pro for the past two months. Here’s a bit of info about how I’ve got it set up, and how it’s work­ing for me

A quick introduction to SSDs

The vast major­i­ty of a HD’s time is spent trav­el­ing from bit A to bit B: read­ing or writ­ing small amounts of data is fast, but get­ting the read/write heads to where it needs to be takes for­ev­er, in com­put­ing terms.¹ Sets of bits that are far­ther apart on the plat­ter take longer to reach: for exam­ple, dupli­cat­ing a file on one disk takes much longer than copy­ing it to anoth­er disk, because the heads have to move to where the file is, read some bits, move to where there’s free space, write some bits, and repeat hun­dreds of times.

SSDs have a con­stant access time, regard­less of what bits you ask it to read or write: read­ing two phys­i­cal­ly adja­cent blocks of a song is just as fast as read­ing a block of data for an appli­ca­tion fol­lowed by a block of data for a download.

It does­n’t mat­ter if an SSD beats a spin­ning plat­ter HD by inch­es or miles for sequen­tial read and writes: the real mag­ic is the through­put and speed of ran­dom reads and writes.

Only big media files, like pic­tures, music, and videos, can typ­i­cal­ly be sequen­tial­ly read and writ­ten.² The rest of the sys­tem, from OS libraries to appli­ca­tion bun­dles and pref­er­ences, are scat­tered across the disk. When you launch an app, OS X tries in a very short amount of time to read the app’s bina­ry, open any libraries the app needs to run, load any addi­tion­al app resources it needs, and read its pref­er­ences and oth­er appli­ca­tion data. None of those bits sit near each oth­er on the dri­ve, so it takes a lot of back-and-forth hard dri­ve seek­ing to get every­thing read in and ready to go. SSDs don’t have to pay any time penal­ty for ran­dom­ly choos­ing two bits to read, so app launch­es are light­ning fast.

The setup nitty gritty

I did this to my mid-2009 15″ Mac­Book Pro. I upgrad­ed the built-in HD to a 500 GB, 7200 RPM Sea­gate dri­ve. I bought an Optibay³ and an 80 GB Intel X25‑M and installed them myself. The install was easy: take 10 screws out of the bot­tom case to expose the guts of the lap­top, and remove three more inside to swap out the opti­cal dri­ve for the SSD car­ri­er. Close it back up, for­mat the dri­ve, and you’re ready to go.

Why two drives?

I want­ed an SSD for the ludi­crous speed increas­es, but I have a few hun­dred giga­bytes of pic­tures, videos, and music, and no one man­u­fac­tures (afford­able) 500 GB SSDs just yet. Instead of mov­ing lots of that media off the machine and onto an exter­nal dri­ve, I’ve split my filesys­tem set­up between the two dri­ves. I moved as many of the more-ran­dom­ly-read bits onto the SSD as pos­si­ble (the base OS, /Applications, and espe­cial­ly ~/Library), and kept my user’s Music, Movies, Pic­tures, Desk­top, and Down­loads on the spin­ning plat­ter dri­ve, where bit stor­age is cheap and longer seek times don’t mat­ter. By keep­ing the entire OS and the vast major­i­ty of my user’s home direc­to­ry on the SSD, boot times, login times, and app launch times are all large­ly the same as an SSD-only sys­tem, but I still have plen­ty of capac­i­ty for large down­loads of videos, pic­tures, or music, and can still work and use all my media while on the go. It’s the best of both worlds.

Dragons ahead

Get­ting an already-installed OS prop­er­ly copied to the SSD was tricky: I hacked a SuperDuper script to copy over my 500 GB boot vol­ume, minus the above paths and a few oth­ers, and sym­link over what I did­n’t copy. It was tricky, and this is a good time to make sure you’ve got an up-to-date SuperDuper back­up of your boot vol­ume some­where: I had to restore the 500 GB dri­ve from back­up once when I typo’d a path and it thought I meant /Music and not ~/Music.

A spe­cial note for any­one won­der­ing about how OS X installs han­dle a dual-dri­ve set­up: I haven’t tried a Leop­ard Archive and Install on this machine, but I can’t think of any rea­son it would­n’t work, as long as you check the box to pre­serve Users. On Snow Leop­ard, I’ve done sev­er­al OS upgrade installs since set­ting this up, and have had zero problems.

Results! I demand results!

I ini­tial­ly did­n’t copy my user over at all, but the sys­tem was­n’t much more respon­sive than a spin­ning-plat­ter set­up: though apps and the base OS were on the SSD, there was still lots of dri­ve thrash­ing when load­ing pref­er­ences and appli­ca­tion data. The biggest speedup in login and app launch time came from putting ~/Library on the SSD: ~/Library/Application Support and ~/Library/Preferences are hit heav­i­ly by apps on launch. I also moved over all of ~/Music except for the actu­al iTunes Music fold­er: hav­ing the iTunes Library file on the SSD great­ly improved iTunes launch times and song meta­da­ta updates.⁴

Right now, only a few spe­cif­ic appli­ca­tions access the 500 GB dri­ve: iTunes (and only when play­ing songs), Aper­ture, and any copy oper­a­tions to/from ~/Desktop or ~/Downloads.

Hacks

Since most of the files used by my sys­tem are on the SSD, sudo pmset -b disksleep 1 is my new best friend. This Ter­mi­nal com­mand sets the disk sleep time­out while on bat­tery to 1 minute: most of my time on bat­tery is spent with the SSD pow­ered and the Sea­gate dri­ve spun down. I’ve seen a big improve­ment in bat­tery life the past weeks as a result.⁵

Was it good for you too?

This con­fig­u­ra­tion was a bit tricky to get set up, but it was very much worth it. OS boot, login, app launches…almost every dai­ly oper­a­tion on the machine is light­ning fast,⁶ and I did­n’t have to make any com­pro­mis­es with what music or pho­tos I keep on the machine. From now on, I’ll be mak­ing sure that any machine I use day-to-day as my pri­ma­ry sys­tem is run­ning on an SSD.

1. HD seek and rota­tion­al laten­cy times are usu­al­ly mea­sured in mil­lisec­onds: a ten mil­lisec­ond seek time burns 20 mil­lion CPU cycles on a 2 GHz proces­sor. Entire empires can rise and fall in the time it takes those plat­ters to spin and the read/write heads to move [↩]
2. assum­ing no file frag­men­ta­tion on the dri­ve [↩]
3. Fras­er Speirs got a Max­Con­nect kit for his set­up [↩]
4. the com­put­er seems to write changes to the library file at the same time as the actu­al media files. Hav­ing them on sep­a­rate dri­ves elim­i­nat­ed the typ­i­cal disk thrash­ing this caus­es [↩]
5. If you install the Devel­op­er Tools, check out Spin­down HD in /Developer/Applications/Performance Tools/CHUD/Hardware Tools/ for an easy way to know if the dri­ve is asleep or spin­ning. [↩]
6. even OS rein­stalls! [↩]

Switching databases with Things

[Sep­tem­ber 1st: updat­ed for Things 1.2. See bot­tom sec­tion for details.]

I use Things for man­ag­ing tasks at work, and for track­ing my per­son­al to-do list. I did­n’t want to mix the two up, how­ev­er, so I use two Things libraries.

To swap libraries, you need to hold down option at pro­gram launch, and choose your oth­er library. But, since this is OS X, defaults read and defaults write can help us auto­mate this process a bit: Cul­tured Code is sim­ply sav­ing the path to the file in their preferences.

I wrote a small perl script to tog­gle between my two libraries, and it’s avail­able here. Save that to ThingsSwap.pl, chmod u+x it, and save it some­where in your PATH (I per­son­al­ly pre­fer ~/bin). You’ll need to change "/PathTwo" to your oth­er library loca­tion, and my $USERNAME = ''; to some­thing useful.

I was unable to open -a Things.app from with­in the script, so my shell alias to quit Things, swap libraries, and relaunch the app is alias swapthings=ThingsSwap.pl && open -a Things.app.

As of Things 1.2, I was get­ting an “Upgrad­ing Things library” dia­log on each swap-and-launch cycle. Cul­tured Code added anoth­er prop­er­ty list key, “Spot­lightIn­dexedLi­brary­PathKey”, which needs to be updat­ed once both your libraries have been con­vert­ed for Spot­light index­ing. Adding anoth­er defaults write inside the if() blocks in the perl script to write that key as well took care of it for me.

CouchDB, and achieving consistency

What hap­pens when you change the same doc­u­ment in two dif­fer­ent data­bas­es and want to syn­chro­nize these with each oth­er? CouchDB’s repli­ca­tion sys­tem comes with auto­mat­ic con­flict detec­tion and res­o­lu­tion. When CouchDB detects that a doc­u­ment has been changed in both data­bas­es, it flags this doc­u­ment as being in con­flict, much like they would be in a reg­u­lar ver­sion con­trol system.

This isn’t as trou­ble­some as it might first sound. When two ver­sions of a doc­u­ments con­flict dur­ing repli­ca­tion, the win­ning ver­sion is saved as the most recent ver­sion in the document’s his­to­ry. Instead of throw­ing the los­ing ver­sion away, as you might expect, CouchDB saves this as a pre­vi­ous ver­sion in the document’s his­to­ry, so that you can access it if you need to. This hap­pens auto­mat­i­cal­ly and con­sis­tent­ly, so both data­bas­es will make exact­ly the same choice.

—CouchDB: Even­tu­al Con­sis­ten­cy. [empha­sis added]

Every appli­ca­tion that does sync­ing should sup­port some notion of “revi­sions”.

It takes only one sync algo­rithm mis­take or a bad con­flict res­o­lu­tion approach to suf­fer data loss. As more apps become “revi­sion-aware”, like Drop­box does so well, there’s many more nov­el things to be achieved with it, and this is one of them.

Obsessive Completionism

I feel com­pelled to con­sume every last bit pro­duced in cer­tain dig­i­tal domains. Twit­ter users and RSS feeds are my lat­est vices; before that, it was Twit­ter and pod­casts, and before that, it was pod­casts and issues of The Econ­o­mist. And every yeah, the SxSW show­cas­ing artists tor­rents inter­rupts my music lis­ten­ing as I slow­ly go through it.

I’m not sure where this dri­ve comes from. I sus­pect it lies in an irra­tional extrap­o­la­tion, a strange assump­tion about qual­i­ty of con­tent: all past and future con­tent will be as valu­able, fun­ny, or inter­est­ing as what I’ve thus far consumed.

Over the past three years, I’ve slow­ly advanced through dif­fer­ent stages of deal­ing with this. Ini­tial­ly, I sim­ply threw time at the prob­lem, ded­i­cat­ing more and more time to read­ing in order to stay “caught up”. Next, I reluc­tant­ly pared back sources, as a lack of time or ener­gy forced me to choose between selec­tive per­ma­nent bank­rupt­cy (unsub­scrib­ing) or wide-spread tem­po­rary bank­rupt­cy (mark all as read). For what­ev­er rea­son, I was more com­fort­able mak­ing an unsub­scribe deci­sion than a mark-all-as-read deci­sion, pos­si­bly because it was eas­i­er to make a per­ma­nent judg­ment on a sin­gle site, rel­e­gat­ing them to the dust­bin, rather than toss­ing out per­fect­ly good news items sim­ply because their date of pub­lish­ing fell on the wrong week. RSS exam­ples are eas­i­est: the first sites to go were the high noise, high vol­ume, low sig­nal feeds like Digg, Boing Boing, and Slashdot.

As I got my RSS prob­lems under con­trol, Twit­ter picked up steam, and I began fol­low­ing more and more peo­ple. I’ve now gone through two Twit­ter purges, each of them only cut­ting back from around 300 fol­low­ing to 250. I find it dif­fi­cult to drop peo­ple one at a time, but like spring clean­ing, the sheer momen­tum of the first few unfol­lows puts me into a much more objec­tive mood. Twit­ter’s new fol­low­ing list UI makes this much, much eas­i­er, show­ing you the per­son­’s last tweet: this often reminds you of just why you’d been mean­ing to unfol­low them.

Late­ly, RSS has been eas­i­er to make these deci­sions with, and, iron­i­cal­ly, I’ve found that step­ping away for a few days and tak­ing a look at your aggre­ga­tor makes this exer­cise eas­i­er. Don’t read any­thing for a week; sort your feeds by most to least unread items, and ask, “Do I real­ly need to read all 300 items? How many good links or arti­cles will be in there? 3? 30? 150?”

The most impor­tant thing I’ve learned about this “obses­sive com­ple­tion­ism” is to know that it’s going on: it forces tough deci­sions about feed sources much soon­er, since you know that the vol­ume will even­tu­al­ly be overwhelming.

A few words about Fever

Shaun Inman’s Fever has been out for a lit­tle over a month, and I’ve been using it full-time since its release.

I love it. By its very nature, a web-host­ed app means nev­er hav­ing to say, “I’m sync­ing”. When I close my lap­top at home, walk to the shut­tle, and open Fever on my iPhone, it’s up to date instant­ly, with no read sta­tus sync prob­lems to wor­ry about. This is noth­ing against Net­NewsWire: sync­ing is hard, and I nev­er got a two machine set­up to con­sis­tent­ly work for me.

I use Fever’s aggre­ga­tion fea­ture¹ only when I’m a few days behind and want a quick sum­ma­ry of what’s being writ­ten or linked to.

The aggre­ga­tion fil­ter works best with a lot of feeds: give it a wide base of feeds to start look­ing for sig­nal, and it’ll have no prob­lem pulling togeth­er a good list of that day or week’s most heav­i­ly linked items. I’m not sure how well it would work for some­one with a less insane num­ber of feeds, but that can be coun­tered by adding more feeds to Sparks to help things along. I’m sub­scribed to about 300 feeds, and have anoth­er 20 in Sparks, to attach some num­bers to what I’m talk­ing about.

I do miss some fea­tures of real desk­top appli­ca­tions: a local cache of arti­cle text would be nice. Net­NewsWire stored arti­cle text and links local­ly, but Fever, being a pure web app, does­n’t do this. It’s hard to use the site on the shut­tle (a high laten­cy, low band­width con­nec­tion), and it’s obvi­ous­ly impos­si­ble to read com­plete­ly offline.²

The iPhone-com­pat­i­ble ver­sion of the site is well-designed and easy to use, and is light­weight enough to use on EDGE. Going to a site to read an arti­cle and get­ting back to the read­er is dumb easy: you’re already in Safari, so there’s no app switch­ing to mess with.

A lot of peo­ple are reluc­tant to try Fever, either because they don’t have host­ing set up, or there’s no way to try it for a week or two before buy­ing. I already had host­ing and a cou­ple of domains with Dreamhost, so set­ting up the appli­ca­tion was easy. If you don’t have host­ing, I don’t know if it’s worth noodling with a local­ly host­ed set­up: it’s prob­a­bly pos­si­ble, but I can’t imag­ine it’s easy to con­fig­ure all the pieces you need present: PHP, Apache, and MySQL, at the very least.

Long sto­ry short, I love the lack of cross-machine sync, and I miss com­plete­ly offline read­ing. If you’re a hap­py Net­NewsWire user that uses more than one client in a week, you should check it out. If you’re a sin­gle machine user, and don’t have domain host­ing already con­fig­ured, I don’t see much of a rea­son to try it.

1. “Here, let me aggre­gate that for you” [↩]
2. A sin­gle sug­ges­tion for future fea­tures: I want a Gears/HTML 5‑compatible ver­sion, where it pre-down­loads arti­cle text and links. I’d be ok not sup­port­ing a ful­ly-offline mode, since this puts things back into a, “how do I sync read sta­tus?” sit­u­a­tion like most desk­top apps, but if you pull text and links from local stor­age, and only trans­mit read sta­tus changes over the wire, the site will work well even in the most band­width-con­strained sit­u­a­tions. [↩]

Formatting digital text for ebooks

Pro­tip: don’t put any­thing in between me and the begin­ning of your ebook. Cred­its, info, ded­i­ca­tions: all that junk should be at the end, not stand­ing in between me and your first real words.

Quickly, with feeling

just so we’re all on the same page:

This place, blog.cbowns.com, is for tech and social com­men­tary (see posts on Twit­ter, Rus­sia, et cetera).

Post­ing fre­quen­cy: maybe once a month; things bake as a draft for a long, long time before they pop out.

chyrp is my per­son­al blog. Thoughts, self-analy­sis, vague hints at angst, et cetera. Be care­ful sub­scrib­ing: I’m hold­ing very lit­tle back, and you may find your­self referred to in a round­about fash­ion one day.

Post­ing fre­quen­cy: updates when­ev­er the hell I get around to writ­ing something.

The Con­fusato­ry is my Tum­blr play­ground. Very lit­tle orig­i­nal con­tent there, most­ly just links to inter­est­ing photos/writing/music.

Post­ing fre­quen­cy: updates 3–4 times a week, when I’m on top of things. Might go a week or two with­out any­thing if I’m snowed under on news­feeds and tabs in Firefox.

Excuses for @replies

Address­ing this briefly before any­one else starts echo­ing non­sense that @chockenberry said earlier.

Twit­ter can’t pos­si­bly have dropped the @reply choice (show all @replies, or show only @replies to users I fol­low) in the inter­est of load management:

It takes more work to take my list of fol­lowees, take each @reply post that one of my fol­lowees writes, and com­pare it to the entire list to decide whether or not to drop it from my stream. It takes more work, not less.

(Edit, 11:30 AM):

Twit­ter just post­ed a response on their blog, part of which said:

We’re get­ting a ton of extreme­ly use­ful feed­back about yes­ter­day’s update to Set­tings. The engi­neer­ing team remind­ed me that there were seri­ous tech­ni­cal rea­sons why that set­ting had to go or be entire­ly rebuilt—it would­n’t have last­ed long even if we thought it was the best thing ever.

Now I’m curi­ous: I don’t think they’re lying, but I’m won­der­ing what issues they’re facing.