I got mine. Get yours.

Thanks to the collapse of the imaginary-asset industry in the US in the months just before the 2008 presidential election, we have had an unusual opportunity to observe both major candidates actually dealing with a perceived national crisis as presumptive leaders. And thus we have learned, clearly and unequivocally, this: when faced with a complex problem with economic, philosophical and public-policy ramifications, neither one of them had any real clue what happened, why and whether it matters, or what to do about it.  

But they differ dramatically in the way they dealt with their ignorance. Obama participated in the process, made some fairly noncommittal comments about the logistics, and patiently reiterated his socio-philosophical agenda. McCain lurched around spastically, spewing hurriedly half-baked ideas and trying to find somebody to yell at.  

There is your 2008 presidential decision in a single indicative reduction: do you want the guy who wants the world to be a better place, or the one who would just as soon headbutt you if he thought he could blame you for anything.  

Will Barack Obama really bring Change and Hope, with those grand Pooh-like capital letters? Can he really, as president, alter the emotional tone of American social discourse? Can he, by sheer persistence of talking about principles as if they actually inform the operation and effects of government and culture, actually bring an American social discourse (back?) into meaningful existence?  

I don't know. Maybe. I'm not exactly counting on it. My distrust of the two-party political protection-racket system far outweighs any personal warmth I feel for this particular guy. He wasn't even my candidate of choice, and I wasn't all that enthusiastic about that guy, either. Government is a very big machine optimized for nothing I need or care about.  

But there's no way anybody who is bothering to vote tomorrow, and I will be one of those bothering, can justify not voting for Barack Obama. As presidential qualifications go, calm principles are no Nobel Peace Prize, but at least they're something. Scamming cut-rate health-insurance from Utah and refusing to stand up for even the mildest forms of social equity are not anything. McCain and Palin are the nihilist ticket: they stand for nothing. Or, more precisely, they stand for not standing for things, for politics as cynical process, and for methodically undermining the idea that we could believe in each other. They stand, as George Bush stood before them, for small-mindedness. They stand only when we're watching, and only because they're afraid we'll see them wince when they sit down.  

A vote for Barack Obama is a vote for ourselves as an idea, for a novel idea that is transformative precisely because it ought to be so mundane. It is a vote for consideration over greed, for resolve over fear, and for the stubborn belief in the possibility of progress over an eager resignation to invisible curses. Barack Obama will not single-handedly solve our problems, and may not even get to preside over the solutions in the time he has, but he will stand with us while we face them, and he will help keep us going while we collectively try to think of things to try. He will stand with us, and as we stand with him we will believe an us into existence. A vote for Barack Obama is not just a vote for this country, and not even just a vote for a country, it is a vote for society itself as a virtue, and for the idea that it can be better.  

A vote for Barack Obama is not a vote for a great man, it is a vote for a decent man who has what ought to be the rudimentary sense to believe that there is a greatness bigger than himself that he himself can, and thus must, participate in creating. It is a vote for the bigger greatnesses to which we too have the capacity to aspire, and in which we thus have the responsibility to play a part. This vote, then, is part of our task. It is, if we have not begun already, where we start. It is the beginning of a long and necessary collective project of reclamation and redesign. It is a vote forward.  

I vote forward. I believe in us, and I take my part of our responsibility for our future. I stand for us and with us, and I accept the project of our society and its potential as my own. I am voting for Barack Obama.

I note, for the record, that a new Powerset feature announced today is effectively a server-specific implementation of a browser feature I suggested last year. Although probably Ted Nelson thought of it some time before I was born.

I gave another version of my Whole Data blog post as part of a panel called "Death of the Relational Database?" at the Web 3.0 conference today.  

My answer to the question in the panel-title is that the relational database was always mostly a coping mechanism for a time of relative scarcity, when our data volume and needs exceeded our storage and processing capacities. This era is not entirely past us, but with more memory, faster data-loading, faster processing and better programming tools, there are increasingly many datasets of larger and larger size for which we can now support qualitatively richer abstractions. Where we used to have to break data apart, and thus were constrained to interact with it in fragmented and broken ways, we can now increasingly afford to keep it whole.  

And here, then, to try to explain why that matters, is my current four-point reduction of the Whole Data manifesto:  

1. What logic allows, storage may not constrain.  

The forms of the logical data-model and its inquiries are by definition independent of any extra-logical storage mechanics. That is, a proper data system exists for the express purpose of sustaining the illusion that the logical data-model is real. This must also be true over time: logically-sensible changes in the logical data model cannot be constrained by storage mechanics.  

2. All relationships are lists.  

Not only must multiple-value relationships and no-value relationships be exactly as easily expressed, manipulated and inquired about as single-value relationships, but every fundamental data operation should thrive on multiplicity and sequence.  

3. There is no up or down, only onwards from here.  

Humans impose hierarchy and directionality on data, but different humans recognize different hierarchies in the same data at the same time. The system itself must be agnostic. No fact is inherently the property of another, they are all independent and structurally equal, and although pairs of relationships can be inverses of each other, there is no absolute “up” or “down”, or “forwards” or “backwards”. A database of albums, artists and years, for example, is exactly as much a database of years as it is of artists as it is of albums, not by the generous will of its schema-creator, but by the intrinsic nature of data. Most significantly, you must be able to start at any node and see everything else in this dataset to which, and from which, it is connected. In fact, turn this around and you get the essential practical definition of a “dataset”: it is a collection of data in which all internal connections are expressed.  

4. Human inquiry follows paths; machine inquiry follows all the paths at once.  

The reason to put data into computers is so that computers can answer questions that are meaningful to humans, but faster for computers. A data model and query language exist to communicate human needs to computers, not to communicate machine preferences to humans, and the responsiveness of a data system is its qualitative ability to answer questions with human motivations, not its quantitative throughput.  
 

At the end of this talk I also did the first (very brief) public demo of Thread, the path-based query language I've written as part of the data-modeling (among other things) system I've been working on at ITA Software. I'm looking forward (to put it mildly) to being able to talk much more about this, but for now I just whirled through a fast series of mostly-unexplicated queries that at least gestured in the direction of the points in my manifesto:  

Album:Year=2000

Album:Artist=Nightwish

Artist:Album~Dark

Label:Album~Dark

Album:~Dark.Label

Artist:(.Album.Label:=Spinefarm)

Label:=Spinefarm.Album.Artist

Artist:(.Album:#5.Year:>2000)

Artist:!(.Album:#5)
 

I also mentioned in the talk, but didn't actually show, the hypothetical query "Who are all the living movie directors who ever directed Cary Grant?":  

Actor:=Cary Grant.Film.Director:!(.Date of Death)  

Go ahead and try to answer that in IMDB without a query language. For that matter, try to answer it in Freebase with a query language. (Or, even worse, try to answer it in Freebase yesterday, before I fixed a bunch of dates of death by hand...)  
 

There's also a short article up on semanticweb.com at the moment, which if nothing else gives a transcribed sense of how I explain these ideas differently in a phone interview than I do in writing.  
 

And there's space for discussion here.

When she thinks she's alone, my daughter practices air guitar.

1. I have $1,000, you have nothing, a latte costs $4.
2. I "leverage" my $1,000 to "lend" you "$100,000".
3. You "leverage" "your" "$100,000" to "lend" Enrique "$10,000,000".
4. Enrique "leverages" "his" "$10,000,000" to "lend" me "$1,000,000,000".
5. The three of us go to the coffeeshop and stand in line talking about our "money" until they panic about "inflation" and raise the "price" of a latte to "$4,000,000". I order one, Enrique orders two.
6. When we go to pay, for some reason Visa rejects both of our cards, despite the two of us having a combined net "worth" well in excess of "$2,000,000,000".
7. Enrique calls in his "loan" to me, "forcing" me to call in my "loan" to you.
8. You have nothing to pay me, and are angry about the whole thing because you took out a "$100,000" "loan" and still couldn't buy a single lousy latte. So you declare "bankruptcy". And then so do Enrique and I, and the coffeeshop for good measure.  

Question for Study:  

In the subsequent $2,032,200,000 government bailout, after I get my $1,000,100,000, Enrique gets his $1,010,000,000, you get your measly $10,100,000, and the coffeeshop gets their $12,000,000, how much does the coffeeshop charge us to reheat $12 of cold lattes?  
 

Or the real question, as always:  

Who profits from your fears?

I note, for the record, that I gave an adapted version of my recent Whole Data post as a talk at the Cambridge Semantic Web Gathering tonight.  

Sadly, usual host Tim Berners-Lee was away on vacation, so he didn't get to hear me reveal his secret superhero identity as Graphius.  
 

At the beginning of the evening we went around the room doing introductions, and each person was supposed to say their name, their affiliation, and then three words. Evidence suggests that a lot of Semantic Web people cannot count.  

My three words were Norwegian Death Metal. The relevance of this was not explicated in the talk, but if you're curious, there is actually a connection.  
 

Some additional notes, and space for discussion, here.

I like CSV. It addresses a common data-exchange need, and does it with admirably little conceptual or syntactic overhead. As formats go, it's totally unglamorous, but glamor is very definitely not an end in itself. If you need to exchange a simple table of simple data, one datum per row/column, CSV is a fine tool.  

I also like JSON. Actually, I really like JSON. It has the data-modeling virtues of my old Elemental XML proposal, with a cleaner semantic distinction between hashes and arrays. I would like to see the spec amended to allow keys to be strings or numbers, rather than only strings, but other than that I think the whole thing is a solid, self-consistent, useful, appropriately scaled solution to an impressively large set of data-exchange and serialization problems. Certainly anything you could model with CSV, you could put into JSON instead, and not at all vice versa.  

But the Law of Conservation of Complexity applies here, as in most things. Although JSON syntax is pretty simple, any particular JSON data model can be arbitrarily complex, and the modeling issues in data-exchange are always far more involved than the syntactic ones. The important simplicity of CSV is in constraining the data model to be a table.  

Flexibility, however, can be employed in moderation. It is possible, for example, to render a CSV model in JSON syntax. That is, to turn this:  

Artist,Albums
Cradle of Filth,7
Nightwish,8
To/Die/For,4  

into this:  

[
["Artist","Albums"],
["Cradle of Filth","7"],
["Nightwish","8"],
["To\/Die\/For","4"]
]  

The advantage of doing so will, I hope, become quickly obvious once you realize that JSON then makes it trivially easy to have cell values be arrays. If we want a list of artists, each with their list of albums, CSV forces us to refactor:  

Album,Artist
Cruelty and the Beast,Cradle of Filth
Damnation and a Day,Cradle of Filth
Dusk...and Her Embrace,Cradle of Filth
Midian,Cradle of Filth
Nymphetamine,Cradle of Filth
The Principle of Evil Made Flesh,Cradle of Filth
Thornography,Cradle of Filth
Angels Fall First,Nightwish
Bless the Child,Nightwish
Century Child,Nightwish
Dark Passion Play,Nightwish
Oceanborn,Nightwish
Once,Nightwish
Over the Hills and Far Away,Nightwish
Wishmaster,Nightwish
All Eternity,To/Die/For
Epilogue,To/Die/For
IV,To/Die/For
Jaded,To/Die/For  

This is annoying at minimum, and unworkable if we also wanted to see other properties of artists. But in JSON, or this modeling reduction of JSON that we might as well call "JSV", we can simply say:  

[
["Artist","Album"],
["Cradle of Filth",["The Principle of Evil Made Flesh","Dusk...and Her Embrace","Cruelty and the Beast","Midian","Damnation and a Day","Nymphetamine","Thornography"]],
["Nightwish",["Angels Fall First","Oceanborn","Wishmaster","Over the Hills and Far Away","Century Child","Bless the Child","Once","Dark Passion Play"]],
["To\/Die\/For",["All Eternity","Epilogue","Jaded","IV"]]
]  

JSV retains the modeling simplicity of CSV (a single row of "fields" defined at the top), but allows a cell to be either a single value or a list of them. This is still unglamorous, but to me it's a big gain in expressivity for essentially no technical cost. I suggest that if combining lists and tables isn't a big deal to you (and you're the sort of person to whom a data-serialization format could be a big deal), maybe you've allowed tabular single-mindedness to beat all the natural human list-thinking out of you. Time to reclaim your right to multiplicity. Time to reclaim lots of your rights, really. Yet another argument for more lists, everywhere around us.

1. Grand Magus: "Like the Oar Strikes the Water" (from Iron Will)  

Music pitched to carry across churning seas and urge on flagging longboats, and for suggesting that maybe I could have loved those thankless non-Ozzy/Dio/Gillan Sabbath voyages, too, if I could somehow separate them from where they weren't ever leaving or landing.  

2. Týr: "Fipan Fagra" (from Land)  

Maybe this should be beyond obvious, but one of the simplest ways to be bigger than the sum of your parts is to keep the parts from wasting so much of their energy differing. You can make good work out of one great idea and a few boring ones, but you can also sometimes make a great work out of one good idea and a relentless disregard for distractions.  

3. Gyöngyvér: "Halhatatlan ámok" (from Világok virága)  

And for eliminating distractions, don't underrate ignorance. You don't have to have the slightest idea what something says to have a beckoning vision of what it means. Something about the limits of anticipation, or how sometimes you honor what you allow to recede, or how chaos snarls itself around everything but starts unraveling from any end you have the patience to pull.  

4. Dark Tranquillity: "Below the Radiance" (from Fiction expanded edition)  

So stand on the nearest prow, steer into your fears, and call for speed.  

5. Everon: "South of London" (from North)  

There's always some direction for progress to take you.  

6. DragonForce: "A Flame for Freedom" (from Ultra Beatdown)  

And always some other side to come through to, out into enough sunshine to cheer and bask in for a while even if you're basically more lost than you were before.  

7. Katy Perry: "Waking Up in Vegas" (from One of the Boys)  

Because the question isn't, ultimately, if you're more or less lost. If you ask it right, it isn't even where you are. It's where next?  
 

[Download all 7.]