David Bisbal: Silencio (1.6M mp3)  

I know very little about Spanish-language pop, but enough intriguing things bounced past my awareness while we were in Puerto Rico that I've been doing some investigating, and finding some stuff I really like. Bisbal is actually Spanish from Spain, and I came across him just poking around in iTunes, but his new album Premonición has some production work by Puerto Rican reggaeton duo Wisin & Yandel, so there's the connection for you.

If the conditions are exactly right, at least, you can still disappear into the sky. You float on your back in the Bahía Mosquito on a moonless and cloudless night, trailing your arms beside you in the warm Caribbean water. You lift them up and point from Vieques towards Orion, and suddenly the stars are falling along your arms and past you and you are diving up into the sky and away.  

But the simple magics for escape are waning. All our magics for escape are waning. You used to only have to evade the moon and the clouds, but now people are building along the shores of the Bahía, and the oblivious lights of their carports slash between the earnest dinoflagelates and the billion far sad stars. Who builds on the shore of the world's brightest bioluminescent bay? And who, having picked this of all the places in the world to stack yet more buildings, puts lights on the outside of them? Who drags sacks of cement and disregard out to a twenty-mile island off the east coast of Puerto Rico where the army finally stopped dropping bombs, and curses some of the best darkness we have left?  

Buried in a karst sinkhole in Arecibo, less than an hour's flight from Vieques, is the reflector dish of the world's largest radio telescope. This is a different darkness, and a different way of trying to reach into the sky, but signs on the viewing platform have to beg visitors to turn off their cell phones, and are probably one bad funding decision away from having to beg them not to perch houses and Subways and ferret stores on the ringing hills. And maybe darkness doesn't sound like it's worth that much, or that the kind of amazement we can see in tiny light-up sea monsters and SETI@Home are petulant objections against seeing where you're parking your Durango, but these scattered oases of left-alone darkness are part too of how we see who we are.  

We're in Puerto Rico, Belle and I, to float in darknesses and sit on still-near-perfect beaches for a few days before we go back to gray New England winter and the last few months in which we're not yet parents. So we're part of the problem ourselves, in the strictest sense. We don't leave any trash on the beaches, but we drive rented cars to them. We paddle around the biobay on kayaks, with little cyalume sticks, but I can't pretend I really know that this is harmless at the current scale, never mind the ones implied by the flow of people like us. Las Cavernas del Rio Camuy, just a little south of Arecibo, used to be a private magic kingdom for bats and brave Camuy kids with flashlights, and now they're a trolley-fed public park where tourists from Boston and New York line-up to point digital cameras at, mostly and necessarily, the new buried lighting system hacked into million-year-dark stone.  

We always change what we observe, of course. Tourism and colonialism are differences of degree. Arecibo beeps like the Earth is backing up, and we kick millions of dinos into panicked glow, so even in the last few darknesses we the just are not just listening. Curiosity is inherently presumption. Our worst egotism and our most powerfully humane hopes are both entwined in science, and everything we touch we touch. Our greatest revelations and our most inexcusable massacres share the gaudy heraldry of discovery. There were people here when Columbus came, and none of them are left to make forests again out of his statues and plazas.  

And no, I don't know how to follow him into promising unknowns without following him into collapse. I don't know how to stop asking questions, or how to stop answering them with more and faster. I don't know how to stop wanting to be the one for whom we make the exceptions, the one person allowed to make the near-perfect darkness one more person less perfect. I don't know how to improve the world in any better way than by trying to make what we discover be worth what we can't keep it from costing. I don't know what better to do with our vanishing darknesses than promise to remember them with our fondest curses. I don't know how to promise that there will always be a sky into which to disappear. For a while, probably, but before too long we're going to need what we used to only want. Selfishness and charity are differences of timing. We will fill in all these holes. We will build highways from everywhere to everywhere else, until all that is left of place is name. We will build casinos on the moon, and come to them to throw away whatever we saved from home below.  

But every once in a while, if we run a little quieter and faster, just outside the lights, we get to see a little of what it used to be like. And what we can't help displacing, we will find tiny ways to become. Everything we touch at least has touched us back. I believe in darkness, but not as much as I believe in us. I believe we help more than we hurt, and that we understand more than we know, and that we know enough. I believe we're ready, and that this child and this world will be grateful for each other.  

I will miss Orion, when he finally turns and leaves us here, but not a billionth as much as I'd miss these blaring lights and crushing statues and people if we too gave up and disappeared into the sky.

(I'm trying to forget about another score from today...)

I signed up for the NetFlix Prize without, originally, any intention to ever actually compete. I haven't studied any math or statistics since AP Calculus over 20 years ago, and my usual analytical tools aren't remotely adequate for analyzing data on the scale required by the contest. I have a couple laptops at my disposal, but no distributed number-crunching farms. My current programming tool of choice is Ruby, which is optimized for coding efficiency rather than processing speed or memory compactness.  

But my current job involves thinking about useful ways to explore and analyze arbitrary bodies of data, and it's not often that anybody gives away a dataset as large or interesting as NetFlix's, so I wanted to at least play with it a little.  

And after playing with it a little I realized that submitting a set of bad guesses, at least, wasn't that difficult. So I did, just for the satisfaction of feeling involved. One of the strangest and maybe most addictive draws of the NetFlix Prize is that bad guesses, if there's even the slightest bit of non-badness embedded in them, are for most practical purposes only trivially worse than fairly good guesses.  

The way the contest works is that you're given the data for several million rating actions taken by real people on the real NetFlix system. Each rating has a user ID, a movie ID, a date, and a score of 1, 2, 3, 4 or 5. You get no other information about the user IDs, of which there about half a million, and only titles and years for the movies, of which there are 17,770.  

In addition to all this, you also get a file of movie IDs, user IDs and dates, but no scores. These represent other rating actions actually taken by some of those same users on those same movies. NetFlix knows what scores go in these blanks. You don't. To play, you just have to submit a set of guesses for all the blanks. You're scored on how close you get, with a little squaring and square-rooting just to make it sound more complicated.  

Meanwhile, back at headquarters, NetFlix has run their own actual software for analyzing movie ratings, using this same exact data. It scores 0.9514, which means it can usually guess ratings within about 1. My first simplistic set of guesses, doing no similarity analysis of any kind, scored 1.0451. This means that my dumb method can usually guess ratings within about 1. Obviously 0.9514 is better than 1.0451, and it's even a little better than it looks due to the exact math, but the human-scale truth is still this: when you're talking about person-assigned integer ratings on a scale of 1 to 5, nobody but us number-geeks is ever going to get excited about the difference between one "about 1" and another "about 1" based on hundredths of a point. In once sense this makes the whole contest pretty idiotic, because who the hell cares?  

But if you can get your score down to the NetFlix-chosen target of 0.8563, you stand to win a million dollars. And if 0.8563 is also pretty much "about 1", the difference between $1,000,000 and $0 is much more readily apparent. And submitting some guesses that were at least a little smarter than my first ones wasn't really any harder.  

So I still have absolutely no illusions of winning. As of now, the leaders are fighting for 0.0001s down in the neighborhood of 0.9042. I got down to 0.9829 without any similarity-analysis, just to see how much variability I could squeeze out of the problem before starting to work on actual logic. My first wild-ass approximation of "similarity" scored worse than that, but after a small amount of de-assing it got me to 0.9508. As of today, you need 0.9402 to get onto the official leaderboard, but 0.9508 is better than NetFlix's own software, which means that in a few hours of my spare time, using no fancy methods or hardware (albeit with a MacBook Pro running overnight a couple times), I have produced better software for this specific definition of movie recommendation than however many people NetFlix pays to work on this problem full-time. If I were one of those people, I'd be embarrassed.  

But actually, if I were one of them, I'd probably also be feverishly working on my own entry in the contest. I'm betting their real engine was built by committee, and thus represents nobody's best ideas. If you held a sufficiently well-constructed contest to see if people could do your team's team-job better than your team was doing it before you knew there was going to be a contest about it, you'd probably lose it about as quickly as NetFlix has lost this one. No matter how complicated you think your work is, once you figure out how to simplify it to the point where you can challenge other people to do it better, somebody will be able to.  

Which suggests, of course, a one-step self-improvement program for just about any existing endeavor: treat your contribution as if you are entering a contest to do your job better than you. Assume some stranger could, with hardly any real insight or work, so that's how easy it ought to be.  

Or, if that all sounds more tiring than inspiring, there's the other possible moral of the story: The most intense effort in any project usually ends up being spent on the 0.0001s, tiny fractional improvements of no genuine consequence, given apparent significance only by the act of measuring them. It's not clear whether it's actually possible to get to 0.8563 using this set of data. Nor whether doing so would meaningfully improve any NetFlix user's real experience. If they really wanted to qualitatively improve their system, NetFlix would have to hold a design contest, not a data-analysis contest, and that would be a lot harder to score. Maybe you're doing your job, as it's currently defined, more or less as well as it can meaningfully be done, too. Maybe, instead of a contest to dicker over your 0.0001s, you should figure out how to do your current job just as well with a lot less effort, and then spend the rest of your time entering the contest to find a totally different and better way.

The logistical irony of substantive blogging, meaning something closer to essay-writing than link-jockeying, is that it enthusiastically both rewards and consumes time. For quantitatively ideal productivity you need to minimize the amount of time you spend doing anything else in your life, which obviously includes anything orthogonal to blogging like shopping for diaper-containment systems or eating brunch with your friends (unless, obviously, you are running a diaper-containment- or brunch-blog), but also includes whatever it takes to actually generate the material to write about whatever it is you're writing about.  

I only ever came up with two good solutions to this, where by "good" I mean effective in producing output, not necessarily in improving my own life any.  

One was writing about music during a period when most of my spare time was spent listening to and thinking about music anyway. It's a trivial psychological insight to observe that the magnitude of this preoccupation betrays it as an attempt to distract attention, mostly mine, from the absences of other things in my life.  

The other solution was writing about technology, or around technology in the same sense that I wrote around music, during a period when my day-job had devolved into pretty much exclusively sitting around thinking about and around technology. This is a pretty interesting experience to get to have, and as I repeated countless times in the job interviews that followed its inevitable eventual termination, I was never even remotely as well informed about the state of technology when I was actually contributing to it.  

I'm contributing to it again. My new job, still settling past its newness after only a few months, is by far the best job I've ever had. There's one formulation of job ideal where you say "I'd be doing this even if they weren't paying me", and this one is beyond that to "I'd be sitting at home frustrated at being unable to participate without a team around me."  

Several things are different and cooler about this job, at least in my own experience, the top few being that I'm explicitly managing it in addition to leading by design, that we're trying to find the shape of the solution we're building (and of the problem we're defining to solve) instead of being handed a VC precis or business plan to fulfill, and that the development work is not primarily segmented by function.  

The combination of these things is very challenging and extremely engrossing. I discovered a quantum jump in responsibility when I went from being a member of a large design team at Interchange to being the sole designer at eRoom, and it's a similar one from being the designer to being the designer and manager, and maybe another level again to be the person with the most navigational accountability for an exploration undertaken knowingly without a map. The internal pressures intensify radically in both directions: to dismiss inanely crazy ideas to avoid causing magnified distractions, and to take superficially inexplicable intuitions even more seriously for their transformational potential. If you don't know where you're going, exactly, you have to operate as if patience and diligence are doomed.  

Somehow this has resulted, over the past few weeks, in my spending the vast majority of my working hours actually programming. I won't reveal much by saying that we're building a big software system that will, when it's working, take in a large amount of previously scattered and unanalyzed data, and attempt to reduce its disorder. Most of the team has been working, since before my arrival, on the data-acquisition aspects of the project. So without anybody else to assign to the languishing data-analysis portion of the system, I assigned it to myself, and have been maniacally trying to figure out what I mean it to do by writing it.  

I am not, in case I haven't been clear about this, really a professional programmer. Over the course of my career I've written a lot of code in a lot of languages, but almost always in carefully circumscribed senses. The UI architecture of eRoom was devised in no small part with my individual capabilities in mind, so that I could write a large amount of the actual UI code without having to participate in non-goal-oriented conversations about lazy pointer deallocation or rogue mutexes. And even the code I'm writing now is, probably, only a prototype, and I've written lots of prototype code.  

My previous prototyping, though, has always been about faking a system in order to play with how it looks and acts. The easiest way to fake a large system is usually to actually build parts of it more or less for real, of course, but every project I seem to fake a little less. The content-management and discussion-forum systems underlying this website are constrained and minimal, but not faked at all. And although the prototype I'm writing at work will probably be migrated out of my eccentric Ruby improvisation into somebody's tediously meticulous Java for dull scalability reasons, my Ruby version really does do what it says it does.  

This is tremendous fun, but it also pushes me into a part of my brain where my most obsessive nature is most dramatically exacerbated. I sit down at my desk in the morning, and eight or nine hours slide by in an un-self-interrupted continuum. I forget to eat lunch, I forget to check my email, I forget the five-minute phone-call-returning task I'm supposed to do while eating the lunch I forgot to eat, I look up at the clock and realize I was supposed to be home an hour ago, and when you live a block from your office there's no possible excuse. Making information fall magically into order is not the only thing I want to do with my life, but it's one of the ways I think I can make the world better, and in the grip of a particularly involved spell it feels like a calling. Something shadowily like a pattern starts to emerge, and I frown at it, and twitch something I've built to be twitched, and suddenly answers are fountaining out of the screen from their own joy at existing.  
 

And that's not even the interesting news.  

The interesting news is that Bethany is pregnant. If the tests and milestones keep going as well as they've been going so far, about six months from now we'll have a child. Software isn't easy, exactly, but it's difficult in easy ways. If you know what to twitch, you can know what kind of magic you'll be able to make.  

People are, obviously, far less tractable. I don't know what kind of child we're going to have. I don't know what kind of parents we're going to be. I don't know what's going to be ecstatic and what's going hurt and what's going to be unbearable. I know nothing will ever be this simple again, and I was already barely coping with simplicity's complexity. Beth and I could probably have spent another ten or twenty years just figuring out how to merge two lives most inspiringly, and now we're going to try to make sense of three. My own explanation of wanting to do this is that raising children is the biggest project available to individual humans, and Bethany is the person with whom I feel most capable of the most humanity. I want us to take on the greatest, hardest, most human challenge for which we are eligible.  

As crazy intuitions go, this makes my daftest futurist semantic-web predictions feel like placing wrist-flick roulette bets with free (and mostly unredeemable) chips. I really do believe that the semantic-web project I'm working on has the potential to be the seed of something that changes the dynamics of human interaction with information sufficiently to advance human nature, but I know that that potential is prorated by its likelihood. I have a small team, and there are many teams and bigger, and many factors and convolutions. Most attempts at deliberate innovation fail, and although opportunities for accidental innovation fail at a far greater rate, they arise at an even greater rate than they fail, so the innovations arrived at by luck vastly outnumber and outfascinate the few we make knowingly.  

We will have made this child knowingly, and although accidents will probably outnumber clevernesses even more lopsidedly thereafter, in this project we will not have the luxury or salvation of irrelevance. There are no swarming competitors feverishly plotting to steal away our parent-share or undermine B and my parenting brand. There are no breathless marketers trying to find the most flattering spin on whatever turn out to be our signature family dysfunctions. Nothing will release us from this. There's just us, and a new person nobody on Earth has met yet. Somebody has to care for everybody, and this one is ours.  
 

So I'm obviously not ready, and can't possibly be, but I'm ready. There are a thousand things to do, and stacks of books and friends and doctors and counselors lining up to add more things to our lists, and I can't imagine how we'll ever get them all done, so I assume and stipulate that we must not have to. I will do whatever it takes to be a good parent, and that will usually be almost enough, and hopefully every once in a while will jubilantly suffice. I will try to be better at this than anything else I have ever done. I don't intend for it to become the only thing I do, but I understand that, like marriage, it is a private commitment that takes precedence over my public aspirations, and that I will make different choices now, and dream of different things. I will have far more important things to write about than b-sides, and rarely even enough time to list them. I will do what I can, and allow myself not to catalogue, even mentally, all the things I would have catalogued when I had time instead of purpose.  

And if this is the most ordinary, and thus in a sense the most mundane, way of raising the stakes for the ways in which I hope to improve the world, and to avoid being even a passive vector of thoughtlessness or evil, then so be it. These swiveling electrical-outlet covers I need to install so a baby that doesn't even exist yet can't stick (I guess) a fork in them, and whatever diaper-containment system we ultimately settle on, and the semantic web and the way we love each other and ourselves, will now all also be part of the inescapably consequential context for a new life. None of this will be faked, or maybe all of it will be, or maybe this is where it stops being meaningful to talk about the difference between what you fake and what you know. This is where suddenly the craziest answers are fountaining out of nowhere into a flood of what it is up to us to make be joy.  
 

(Discussion on vF...)

The theme was Greek. I wore this.  

Belinda: "Bella Traición" (1.7M mp3) (from Utopía)  

Maybe the year's best demonstration of the universality of J-Pop.

I'm not mechanically inept, but my range of experience in home-maintenance tasks is not large, and mostly clustered around either calling in experts to do projects that most people would not undertake themselves, or things involving adding or removing a relatively small number of nails or screws. I recently repaired, rehinged and rehung two shed doors without difficulty, that's about my usual scale.  

So replacing our water-hemorrhaging "food disposer" (didn't these used to be called "disposals"?), which involved both plumbing and electricity, seemed like the kind of task that could descend into total chaos about halfway through, when some minor piece of the replacement apparatus proved incompatible with some intractable facet of the existing environment.  

Somewhat to my disappointment, the replacement went really easily. The old corroded hulk gave way without much complaint, the new one slid in with equanimity.  

The only interesting part of the task was tracking down the source of a strange trickle of water that appeared to be coming out of a wall where there was, as best I could tell, no plumbing at all. It turned out that the old unit had been leaking water into the flex-pipe for its own electrical cables! Not only is this obviously bad, but the constrained space made it an interesting topological puzzle to get the several ounces of fetid water back out of the pipe.  

But even if the tasks were ultimately simple and only trivially grimy, and probably a Real Man would have sucked the water out of the electrical pipe without even turning off the breaker first, I note for the record that in this upgrade I have gone from a third of a horsepower to three whole fourths, and I think we can all agree that that basically reeks of machismo.

My sister asked me, as part of her wedding ceremony, to Say Something About Marriage. I've been married for over two years, so obviously I have way too much insight on this issue than could reasonably be fit into the middle of an otherwise fairly compact service, but with some effort, and some help from my wife Bethany, I got the list down to twelve potentially useful pieces of advice. These probably aren't the most useful twelve pieces of advice, but I think they're better than nothing. As delivered at the wedding last weekend:  

1. Your world is getting bigger today, not smaller! More history, more friends, more possibilities. Marriage is not the end of the search, it's the beginning of all the searches that are more fun to do together.  

2. Be the guardians of each other's solitudes. Not only do you need to give each other space, you need to make each other space.  

3. No difficult conversations after 10pm. Not only is it harder to solve problems when you're tired, but at least half the time being tired is the problem.  

4. The Dutch principle of Total Soccer means that any player can attack when there is an opportunity, and any player can defend when there is a need. In Total Marriage you only have two players, so this is even more important. Both of you should be able to do everything your team needs. You'll have your preferences and strengths and habits, but if one of you goes down, the other one has to be able to cover.  

5. Wedding rings don't really come with magic powers. You will learn how to take care of each other one insight at a time. And even when you're not sure how, show up and you'll think of something.  

6. Headphones; separate closets.  

7. If you aren't already the world's leading experts on each other, you will be soon. It is thus your responsibility to be not only the world's biggest fans of each other's best qualities, but also the world's staunchest fans of each other's weaknesses and flaws.  

8. Get pets. By far the easiest way to remember that you have to feed your shared life together is if part of it comes and stomps on you every morning.  

9. No ultimatums. Ever.  

10. Travel. Surprise and challenge yourselves. It's easier to have a world together if you have a world to compare it to, and part of the fun of getting to know each other is putting yourselves, together, in positions where neither of you know what you're going to do yourself.  

11. Committing yourselves to one another is one of the most mature, responsible, focused decisions you can make. Balance it out by being immature, irresponsible and playful together as often as possible.  

12. When people, especially your relatives, offer you long lists of marriage advice, just smile politely and nod until they finally shut up.