An abacus is a state machine. It executes no instructions, and maintains no history, but it does store a single state, semi-persistently and nearly-infinitely rewritably, and it stores it in a representation that facilitates operator-initiated state-changes of certain types. An electric typewriter with a single-character backspace function is approximately equivalent in computational terms. Both of these are very useful devices.  

A typewriter with a multiple-character backspace function has both state and memory. The simplest electric calculator has both state and automated instruction execution. A semi-modern calculator has state, instruction and memory, and at this point we can call it a basic computer. The subsequent history of human-computer interaction design has been a slow process of iteratively transcending decreasingly unimaginative understandings of the implications of state, instruction and memory.  

The conceptual breakthroughs of the earliest text-processing programs were 1) that semantically non-numeric information could be represented in numeric memory, 2) that semantically non-mathematical operations could be modeled in mathematical instructions, and 3) that quantitative increases in memory capacity could enable qualitatively different uses of that memory. Further thought about representation led to storing formatting in addition to text itself. Further thought about instructions led to the automation of layout operations, and the addition of text-processing operations like search-and-replace. This makes for a more interesting state-machine than an abacus, but still effectively a state machine in most user-apparent aspects.  

The conceptual shift from state machine to information appliance can be reduced, symbolically, to model-altering insights embodied in three perhaps seemingly incremental features. From the critical realization that the computer's representation of information could include more than the ostensible current state came the radical notion of Undo, and later its extrapolation to Undelete. From the realization that a significant body of pre-existing external human knowledge could be represented and usefully applied to user-generated information came the extraordinary new idea of machine proofreading. File transmission applied signal-bearing wires to the space between people, rather than just between devices. Combine data application and wires and you get the net as gigantic reference library and perpetual market. Combine wires and internal state and you get distributed applications and the net as communication infrastructure. Combine data application and internal state and you get data mining and machine translation. Combine all three and you get more or less everything in modern computing up to the night before IM, online dating, eBay, Mapquest, Napster, Google, SETI@Home, "people who bought this also bought", phonecams and the blogosphere.  

But it's the next morning, now, and I don't really want an information appliance. I want a virtual personal assistant. I want my writing software to think of its job not just as formatting documents, but as remembering everything I do when I write, including things I don't realize I'm doing, and things I do while writing that aren't themselves writing. I don't just want document-level Undo, I want a coherent journal view of everything I typed, including all the dead-end phrases I tried and deleted and might now want to revisit. Actually, I don't want fundamentally document-level anything, I want a dynamic evolving history of my entire interaction with my computer and the network beyond it, navigable by chronology or association. I want to jump from an email to the web page on which I found the stat I cited four replies ago in the note that started the conversation. I want to go from the song I'm playing to the birthday of the person who told me about it, to a cross-referenced list of the other music I associate with the song and the other music all my friends have mentioned in emails and IMs and forum notes and shared playlists and now-playing monitors. I want to see rhythms of correspondences and patterns of discovery and contours of neglect. I want the things I've forgotten to know when to remind me of themselves, and the things I think I know to have the humility to volunteer for their retirements.  

The primary challenges for the design of virtual personal assistants are of a different nature (naturally) than the challenges for the design of information appliances or state machines. What the state machine worries about representing, the assistant thinks about communicating and transforming and connecting. What the information appliance struggles to remember, the assistant has to decide how to share and correlate, and when if ever to forget. The state machine works to its capacity. The information appliance works to its parameters. The assistant, however, must be self-governing and evolvingly aware of its own limits, able to differentiate between automating and advising. The assistant will be evaluated not only on what it accomplishes, but on what it knows to ask and when. The state machine's applications were solipsists, however creative. The information appliance's applications were autocrats, however occasionally beneficent or enlightened. The assistant's applications are inventors and ambassadors and advocates and court jesters, and sometimes mercenaries and cannon fodder, and every once in a while oblivious innocent bystanders willing to go home without complaint when you promise them there's nothing to see here.  

And in a connected and definingly social world, the virtual personal assistant is a distributed and intimately negotiated function, and the rules that maintain the productive tension between isolation and aggregation are even more complex. What is the currency of the economy of privacy and trust? On what grounds do you delegate a privilege or retain it? What of yourself are you willing to reveal in return for what collective wisdom, from what collectives, and for that matter which and how much "wisdom" are you prepared to consume, and in what forms? When is it information we seek, and when is information-exchange merely a proxy for personal contact? When does a system become more humane by modeling its users more precisely, and when does it serve them better by leaving them to their own improvisation and compromise?  

The new world will be many things, some of which are already emerging and some of which are yet deeply hidden, but here are a few of what may be its truths:  

- Millions now stored will never be erased. In the last era, everything not saved was instantly lost. In the new era, everything not meticulously preconstructed for disintegration will be indexed and archived forever.  

- Data belongs to people, not processes. There are no silos in the new architecture. Persistence doesn't mean writing something so that it can be reconstructed by its originating code, it means writing it so that it can be reconstructed without its originating code.  

- You are in a maze of twisty passages, each explicably unique and enticingly beckoning. The new systems must not only know when to ask you questions, they must know how to categorize the properties of the possible answers. They must know how to empower your responses with nuance rather than luring you into literalist traps.  

- Everything good is relative. The old era was about identification and instantiation and encapsulation. The new era is about connection and abstraction and subcomposition and change. The old tools had files and records and pages. The new ones have links and self-description and self-direction. The old world was measured in assignments and addresses, the new one in associations and relationships. The old tools took knowledge apart, the new ones must put it back together again.  

- There are three classes of the acted-upon and the acting: objects, creatures and artists. Objects have no value except as they benefit creatures or express the work of artists, and perform no act except in response. Creatures are to be respected and defended and delighted, and acknowledged in their free will, but not burdened with responsibility or solicited for decisions. Artists are the source of all authority and the ultimate ends of all means. Humans are sometimes artists but always at least creatures. Machines and systems and programs (and policies and corporations and governments and precepts, including these) are never more than objects. The first obligation of any designed system is to be obsessively devoted to the intricate cognizance of these boundaries.  

The simplest worthy tools exist to protect or sustain something alive. The best ones express something that makes living more beautiful. What numbers do your machines safeguard that an abacus wasn't sufficient to protect? What do your machines make beautiful, that was ugly when all we had were wood and beads and hands?

There ought to be a qualitative difference between a computer and an abacus.

 

The results and answers are up from last week's 20 Covers audiotest.

I have now posted the disappointingly mundane iTunes script I really use. I am still trying to decide whether the joke is on 43 Folders and Boing Boing for linking to last week's customization parody/fantasy without noticing that the second half of it is patently absurd, or on me for describing imaginary features that ought to actually exist.

Before Tiger renders some large amount of it obsolete, I want to make some notes about my customized iTunes configuration. The actual Applescript code is too obscure and specific to be very enlightening, but possibly a brief description of the overall flow would be of interest to others.  

First some notes about use. At this point iTunes controls virtually all of my music listening. The bulk of my music is still acquired by purchasing physical CDs, but these are ripped into iTunes immediately (AAC, 128kbps). At home our main stereo is hooked to an Airport Express so both B and I can beam music to it from our Powerbooks. We both also have small speakers on our desks, and there's another set by the bed that we can plug iPods into when we want. I still do use the CD changer in my car, but both of us also use our iPods in our cars, and I use either the Powerbook or the iPod at work.  

My music collection is far larger than my Powerbook or iPod can accommodate, so my encoded selection rotates as new and revived interests push other things out of the active 20GB. Much of the time I use iTunes in Browse mode, listening to individual whole albums in the same way I would have pre-shuffle-era. But increasingly, and especially during periods when my listening isn't so dominated by new releases, I also use iTunes' Party Shuffle mode, fed by a Smart Playlist that filters out non-music genres, cuts out tracks that are too short (<1:30) or too long (>5:22) for my shuffle attention-span, and via another playlist reference excludes anything that has been played recently (i.e., in the last two weeks, or the last 10 hours of music, whichever list is shorter). If I'm in an especially random mood, I have an Applescript that goes through the upcoming Party Shuffle selections and eliminates repetition of artists.  

Organizationally, I normalize all incoming music to one of four genre tags: J-Pop, Metal, Noise and Rock, where "Rock" means simply "everything else". For sorting purposes I flip all artist names to "Last, First" and delete leading "The"s. Sound Check, Sound Enhancer and Crossfade Playback are all off. I show the little arrow links, but have tweaked the preference file so that they search within my library first, instead of jumping to the iTMS.  

I use Synergy heavily. The Play/Pause button sits alone in my menu bar, but I have keyboard shortcuts for Previous, Play/Pause, Next, Volume Up and Volume Down, and I like Synergy's own Floater better than the version it produces through Growl. The rest of my customization is run by an Applescript triggered by Synergy as a Track Change Item.  

The script in turn does several things:  

1. Updates my IM status message to include the track info for whatever is playing.  

2. Uploads the track info and artwork for the current track to this website, with the proper HTML encoding for Unicode characters.  

3. Maintains a single-track playlist called "last started" which contains the most recent track that iTunes began playing.  

4. Checks, any time iTunes switches to the "stopped" state, to see if the track in "last started" is the same as the last track in "Recently Played" (which iTunes adds tracks to when they finish), and if so, if that track falls in the middle of an album. If all these things are true, the script restarts playback on the next track of that album. This is a complicated but fairly effective way of compensating for iTunes' predilection to lose track of what it was playing if you're poking around in your Library or the iTMS while you're listening to something else. The began/finished double-check is necessary to prevent this part of the script from effectively disabling the Stop button entirely, which my first iteration of it did.  

5. Automatically updates the rating for the track. Although on very rare occassions I do rate tracks manually, for the most part I find that it is more effective to treat the rating as a temporary variable representing my actual behavior towards the track, instead of an attempt to measure my subjective assessment directly. In my case, the rules are approximately these (I've left out some of the more logistical obscurities):  

a. New songs enter the system rated 0, which means simply "unplayed".  

b. If a song plays all the way through, and it is currently rated 0 or 1, its rating is increased to 2.  

c. If a song is played again while it is still on the Recently Played playlist (which with my Party Shuffle configuration would only happen through my deliberate action), its rating is increased. It takes only one repetition to promote a track from 2 to 3, but two to go from 3 to 4, and three to go from 4 to 5. So 5, my highest rating, means that I've played the song all the way through at least seven times in relatively close succession.  

d. If a song plays for at least :03, but less than half its length, its rating is decreased by 1. It takes only one skip to demote a track from 5 to 4, 4 to 3 or 3 to 2, but two to go from 2 to 1. Since promotion goes from 0 directly to 2, the rating 1 is reached only by demotion, which allows it to serve as an unambiguous indicator of disfavor.  

e. If a set of songs from an album plays all the way through in order, and then the subsequent track on the album starts but doesn't finish, and nothing else from that album is played, I am considered to have skipped all the unplayed songs on the album (where "unplayed" means they don't occur in the previous or next 10 entries in the track history, to catch the case where I interrupt an album but return to it). This rule is screwed up if I single out the last song on the album, since there's no subsequent track to check for, so I try not to do that.  

f. At the end of a calendar month, any song with a rating of 3 or lower that hasn't been played since the previous calendar month has its rating decreased by 1, and any track whose rating would be reduced to 0 by this is left at 1 but added to a "to be deleted" playlist, which at the moment I still review manually. The missing detail here, obviously, is that I need to keep track of both a track's current rating and its historical maximum. A track that never got above 3 can be deleted when it gets back to 0, but a track that was ever rated 4 or higher should be down-converted to a lower bitrate encoding instead of being deleted entirely. Hopefully the new Automator framework in Tiger will expose programmatic control of encoder parameters, which the Applescript interface to iTunes 4.7.1 does not.  

6. Feeds the track's first two artworks (on the assumption that these are the front and back cover) via ImageEngine to Manatics' Handler plug-in, set at 15%, which applies a mostly transparent mask of my scanned-fingerprints to simulate package wear. The script skips this step if the track or its album are repeated without any non-album tracks intervening, figuring that in physical use this would have meant I didn't need to touch the LP sleeve or CD case.  

7. Checks the Comment section for the track, which I maintain in a semi-structured format, for my private rights coding. Any track marked "Bought" came from a legally purchased CD or download, and so requires no further compensation to be issued. A track marked "Preview" is one I have acquired without the artist being compensated, but which only exists temporarily in my library while I decide whether to purchase it. By default no compensation is issued for these tracks during the first week after they are added to the library. Each time one is played more than a week after download, the script asks whether to convert their status to "Honor". Tracks marked "Honor" are ones for which I owe the artist separate compensation from any involved in acquiring the data itself.  

For handling the bookkeeping of this separate compensation, the script (actually a combination of Applescript and perl/MySQL) maintains an external database of compensatable artists. By default each full album is worth $4, and individual non-album tracks are worth $0.20 (the system provides support for different values per-album and per-track, and for different system defaults based on acquisition date, but at the moment I'm not actually using those features). I prorate these amounts over the first four times I play the material, so if I play a whole album once, but only once, my accounting considers the artist to be owed only 1/4 of the value, currently $1. The database keeps track of the total amount I owe each artist, the amount they have been paid already, and the amount that has been transferred into a compensation escrow account.  

For artists who accept electronic payment via Paypal or credit cards, the database records the relevant payment info. A separate nightly perl script issues electronic payments (via CapitolOne's excellent web-services "micropayment" (sic) interface) where possible (batched until the amount exceeds $1.75), and for artists without electronic payment info, totals the corresponding amounts and transfers the overall total to the money-market escrow account I have for this purpose. The escrow account isn't automated yet, but I usually kludge around this, when I get new payment info for an artist, by simply moving the money back out of escrow, resetting the amount paid/escrowed to $0, and rerunning the original script. At the end of each month I mail physical checks to artists for whom I have physical addresses but not electronic, and rebalance the escrow account accordingly. I don't currently have a way to split this compensation across the performer and the publisher for material where the two are different. The best I can do is include track info with all my payments and rely on the artist in turn to pay for material they acquired from elsewhere.  

8. Finally, and I admit that this last bit is a little geeky, the script files some extra metadata that I use periodically to generate some reports of personal interest. It notes a) where I was when I played a track, b) if it's a running day, whether the track was played before or after my run, c) whether the previous track's info got any hits on my website before being replaced by the current one, d) whether the track starts or ends exactly on a minute boundary according to the NIST atomic clock, e) how many years old the track is (that is, current year minus release year), f) whether the track is an alternate version of an original album track, and g) if the track comes from a release that consisted on CD of multiple discs. For portability reasons this metadata is written into a semicolon-delimited self-addressed email, and re-extracted by a different Applescript run by a Mail rule and appended to an Excel file that I then output in PDF form for archiving.  

Clearly there's a lot more that could be done to make my iTunes usage actually conform to my personal listening habits, rather than me just letting the technology control my life the way it usually does, but eventually you've got to strike some kind of balance.

20 short clips of unidentified cover songs you might enjoy recognizing or not.

There are two songs I never finished that periodically come back to mind. One was an awkward story about ecocatastrophe, extraterrestrial emigration and constructed aesthetics called "The Death Rose", which I was working on interminably in Cakewalk at the point a decade ago when I decided that my MIDI array was keeping me from ever finishing songs. Switching to analog instruments for a while did, in fact, radically improve my productivity, but it also orphaned that song, which sounded wrong to me whenever I tried to reinvent it outside of its programmed loops. Every once in a while, though, I still find myself arbitrarily humming "...merging our hyyyyyydrofarms into a terraformed commmmmmmpromiii-zeh-...", which probably in itself tells you enough to guess that nobody is really missing much. But we'll never know for sure. Only the me that still took the song seriously could have finished it.  

The other song was even more stilted, a weird poetic-justice revenge fantasy crossed with an insurance procedural, told as the suicide monologue of a protest-casualty activist's widow driving an explosives-laden Toyota Celica into Manhattan from New Jersey. I wrote the lyrics to this one in 1986 or 1987, and obviously it would come off very differently now. Writing-wise, its biggest problem was that I used more specific place-detail than I was actually familiar with, and never took the time to go find out whether that particular block of the Avenue of the Americas, or that particular inbound Holland Tunnel lane, had any of the properties the events of the song required. Also, it's not clear what I had against Celicas. Possibly these problems could be fixed.  

But I've never been able to think of another name for the character in the song. The widow never identifies herself, but she repeats her husband's name several times, quoting news stories and police reports and insurance forms. The name I invented, at random, entirely for the way the consonants and vowels of it fit the melody, was Randall Thierry. I am nearly certain that I had never heard the name of Operation Rescue founder Randall Terry before that, but I certainly have since, and Terry is not at all the kind of activist I meant. It seems like it should be incredibly trivial to switch out "Randall Thierry" for a name with a similar shape, like "Matthew Delaney" or "Carlo Fiori" or something, but I've never thought of one that doesn't sound wrong to me when I sing it. In my head, the character's name is Randall Thierry, and that means the small story of his accidental fictional martyrdom and has been pre-empted by a real man's sadly non-fictional idiocies.  

So next time you feel like self-censoring some pretentious, awkward, overwrought, underinformed thing you half-invented, just remember that you grow too quickly out of every precious delusion, and whatever the realities that supplant them offer us, it's almost never deeper comfort or higher inspiration. We should write down, every day, everything we can imagine we believe. Otherwise, how many of them will by tomorrow we already have lost?