¶ Algorithms and Humility (and All the Days the Music Doesn't Die) · 3 February 2024 listen/tech
When you go to an artist's page on Spotify, there's a big Play button at the top. This seems reasonable enough. Playing their music isn't necessarily what you want to do, but it's one of the most likely things. What does it mean, exactly, to "Play an artist", as opposed to playing a particular release? Hit the button and pay attention to the track-sequence you get, and you can quickly figure out what Spotify has chosen to make it do, which is that it plays the artist's 10 Popular tracks in descending popularity order. After that it gets a tiny bit trickier to follow, because it goes through the artist's releases, and those releases are listed right there on the artist page, but the playback order usually doesn't match the display order. But poke around and you'll find that the playback order matches the Discography order (what you get to via the "Show all" link next to the list of "Popular releases"), which is reverse-chronological in principle, although release-dates are a contentious data-field so good luck with that.
This is reasonable behavior, not least because it's explainable, but it's not always the greatest listening experience. What you probably want, I think, if you just hit Play without picking your own starting point, is a sampler of the artist's songs. Their 10 most popular songs are a subset, but not always a great sample. They might all come from the same album, they might include multiple versions of the same song, they might include intros or interstitial tracks that don't actually make sense on their own. And a reverse-chron trudge through literally all the artist's releases, after those first 10 popular tracks, is not a "sample" at all.
This bothered me, so at one point pretty early in my long time at Spotify I spent a little while seeing if I could devise an algorithm to create a better sample-order. It wasn't especially complicated, but it tried to diversify the selection by album, and to group song-versions in order to understand singles as part of their album's eras, and not play the same real-world song over and over due to minor variations. It rarely produced the same summary of an artist's career that a knowledgeable human fan would have, because it didn't have any real cultural insight to work with, but it did a decently non-idiotic job for most artists. I felt pretty good about claiming that it was a better default introduction to an artist than playing the 10 most popular tracks and then every single release.
That wasn't what we ended up doing with the idea, though. Invisible improvements are unglamorous. Instead, it became A Product. That product was the "This Is" artist-playlist series. And because Products make Claims, this new playlist series got an ambitious tagline: "This is [artist name]. The essential tracks, all in one playlist."
Here, apropos of today's anniversary of The Big Bopper's untimely death, is the contents of This Is The Big Bopper:
You can see, I think, that the execution has not quite lived up to the premise. The algorithm has done its best to vary the order of nominal source albums, but The Big Bopper didn't make any albums while he was alive, so all of these are actually posthumous compilations. He didn't record very much, period, so in an attempt to make a playlist that isn't just his two hits, the rules have picked a bunch of tracks that aren't even available for streaming, including a couple of sub-1:00 news clips that we are probably happy to be forced to skip, and a very dubiously misspelled "It's the Thruth, Ruth" that probably shouldn't have been released in the first place. But even without those, it makes little musical sense to describe this set as "all the essential tracks". Most of these are no more "essential" than the others, and the official a-side of his first single, "Purple People Eater Meets the Witch Doctor" ("Chantilly Lace" was the nominal b-side of this), is missing.
As an unseen track-order for a sampler, though, this isn't terrible. It improves on the default play-button behavior by not playing the same songs 3 or 4 times each, at least. I'm pretty sure my original version of this algorithm had a duration-filter that would have eliminated the news clips, and an availability filter that would have blocked the Thruth. The algorithm, itself, was a small useful thing that improved the world a little bit. That's all, as its author, I ever claimed about it.
The claims we make, about our algorithms, are a different thing from what they are. I was not in charge of the claims Spotify ended up attaching to this one. I believe that algorithmic intermediation of culture should be done with relentless humility and care. This is not the attitude generally adopted by tech-product marketing. "All the essential tracks" is a more compelling premise than "a slightly better sample-order", for sure. I wouldn't have used it, because the algorithm doesn't deliver it. Marketing doesn't care.
Does it matter? In this case, maybe it doesn't matter a lot. In truth there's probably only one "essential" Big Bopper song, and it's "American Pie". You've achieved a minimally acceptable cultural literacy if you know what Don McLean's memorial is about, and extra credit if you can hum "La Bamba" and any Buddy Holly song that isn't actually by Weezer. The Big Bopper is, sadly, a lot more famous for dying in a plane crash than he is for anything he sang. If you hit his Play button and get "Chantilly Lace", that's already more than most people know.
The This Is series has gone on to be pretty popular. It's exciting to get a This Is playlist, as an artist, because it suggests that you have "essential" tracks. But that, too, is a marketing claim with no inherent grounding. The criteria for generating them are logistic, not cultural, and the thresholds have been adjusted downwards over time. I have one, and my music is as non-essential as you can get without employing AI. Illusory validation caters to vanity, and subtly devalues actual validation.
Taken in collective aggregate, these tech-marketing tendencies to oversell the significance of algorithms, and in particular the hubris in making cultural claims about the results of mostly-uncultural computation, are a sort of pervasive reverse-gaslighting, substituting brightly confident light where it should be modestly dim. And every little cognitive dissonance like this that we accept erodes either our actual awareness of misrepresented reality, or our trust in systems, or both.
But here's the thing. At the end, there's still music. The algorithms have no soul for music to save. We do. Our machines can only gaslight us if we grant them authority. So don't. They serve at our pleasure, but sometimes they work. You don't have to trust them to cherish them when they help. The past doesn't always organize itself, and math and patterns of our listening can tell us things we only almost already knew. Here's another of my attempts to put songs in algorithmic order:
This one tries to re-center the universe of music on any individual artist of your choosing, and then follow a vague spiralish pattern outwards in every direction at once. If we start with The Big Bopper, does it reconstruct the Music that Died that day? I don't know, I wasn't even born yet. But this math started from the The Big Bopper and rediscovered Buddy Holly and Ritchie Valens without knowing it should, so that's an interesting start. Is it "canonical"? No, of course not, the title is my rueful joke, and there's a note at the bottom that explains what I'm attempting. If you think algorithms themselves are the problem, I'm definitely part of it. I believe in attempts. If I had written the blurb for This Is, it would probably have said "An earnest algorithmic attempt at finding the maybe-essential tracks." Marketing doesn't talk that way. It isn't earnest, and it certainly isn't self-aware of how earnest it isn't.
But where self-awareness is systemically missing, we can sometimes reintroduce it ourselves. Not always, but sometimes. We don't have to let overselling trick us into thinking every oversold thing underperforms. We don't have to let premature marketing hubris scare us away from experimentation and helpful progress. Defuse their claim of essentiality with a now-knowing smirk, and those This Is playlists can be interesting. This may not be a canonical path, but it might still take us somewhere. That's something. We can let it be enough. Let algorithms work when they work for us, and fail cheerfully when they don't, and this will be yet another of all the days that music doesn't die.
This is reasonable behavior, not least because it's explainable, but it's not always the greatest listening experience. What you probably want, I think, if you just hit Play without picking your own starting point, is a sampler of the artist's songs. Their 10 most popular songs are a subset, but not always a great sample. They might all come from the same album, they might include multiple versions of the same song, they might include intros or interstitial tracks that don't actually make sense on their own. And a reverse-chron trudge through literally all the artist's releases, after those first 10 popular tracks, is not a "sample" at all.
This bothered me, so at one point pretty early in my long time at Spotify I spent a little while seeing if I could devise an algorithm to create a better sample-order. It wasn't especially complicated, but it tried to diversify the selection by album, and to group song-versions in order to understand singles as part of their album's eras, and not play the same real-world song over and over due to minor variations. It rarely produced the same summary of an artist's career that a knowledgeable human fan would have, because it didn't have any real cultural insight to work with, but it did a decently non-idiotic job for most artists. I felt pretty good about claiming that it was a better default introduction to an artist than playing the 10 most popular tracks and then every single release.
That wasn't what we ended up doing with the idea, though. Invisible improvements are unglamorous. Instead, it became A Product. That product was the "This Is" artist-playlist series. And because Products make Claims, this new playlist series got an ambitious tagline: "This is [artist name]. The essential tracks, all in one playlist."
Here, apropos of today's anniversary of The Big Bopper's untimely death, is the contents of This Is The Big Bopper:
You can see, I think, that the execution has not quite lived up to the premise. The algorithm has done its best to vary the order of nominal source albums, but The Big Bopper didn't make any albums while he was alive, so all of these are actually posthumous compilations. He didn't record very much, period, so in an attempt to make a playlist that isn't just his two hits, the rules have picked a bunch of tracks that aren't even available for streaming, including a couple of sub-1:00 news clips that we are probably happy to be forced to skip, and a very dubiously misspelled "It's the Thruth, Ruth" that probably shouldn't have been released in the first place. But even without those, it makes little musical sense to describe this set as "all the essential tracks". Most of these are no more "essential" than the others, and the official a-side of his first single, "Purple People Eater Meets the Witch Doctor" ("Chantilly Lace" was the nominal b-side of this), is missing.
As an unseen track-order for a sampler, though, this isn't terrible. It improves on the default play-button behavior by not playing the same songs 3 or 4 times each, at least. I'm pretty sure my original version of this algorithm had a duration-filter that would have eliminated the news clips, and an availability filter that would have blocked the Thruth. The algorithm, itself, was a small useful thing that improved the world a little bit. That's all, as its author, I ever claimed about it.
The claims we make, about our algorithms, are a different thing from what they are. I was not in charge of the claims Spotify ended up attaching to this one. I believe that algorithmic intermediation of culture should be done with relentless humility and care. This is not the attitude generally adopted by tech-product marketing. "All the essential tracks" is a more compelling premise than "a slightly better sample-order", for sure. I wouldn't have used it, because the algorithm doesn't deliver it. Marketing doesn't care.
Does it matter? In this case, maybe it doesn't matter a lot. In truth there's probably only one "essential" Big Bopper song, and it's "American Pie". You've achieved a minimally acceptable cultural literacy if you know what Don McLean's memorial is about, and extra credit if you can hum "La Bamba" and any Buddy Holly song that isn't actually by Weezer. The Big Bopper is, sadly, a lot more famous for dying in a plane crash than he is for anything he sang. If you hit his Play button and get "Chantilly Lace", that's already more than most people know.
The This Is series has gone on to be pretty popular. It's exciting to get a This Is playlist, as an artist, because it suggests that you have "essential" tracks. But that, too, is a marketing claim with no inherent grounding. The criteria for generating them are logistic, not cultural, and the thresholds have been adjusted downwards over time. I have one, and my music is as non-essential as you can get without employing AI. Illusory validation caters to vanity, and subtly devalues actual validation.
Taken in collective aggregate, these tech-marketing tendencies to oversell the significance of algorithms, and in particular the hubris in making cultural claims about the results of mostly-uncultural computation, are a sort of pervasive reverse-gaslighting, substituting brightly confident light where it should be modestly dim. And every little cognitive dissonance like this that we accept erodes either our actual awareness of misrepresented reality, or our trust in systems, or both.
But here's the thing. At the end, there's still music. The algorithms have no soul for music to save. We do. Our machines can only gaslight us if we grant them authority. So don't. They serve at our pleasure, but sometimes they work. You don't have to trust them to cherish them when they help. The past doesn't always organize itself, and math and patterns of our listening can tell us things we only almost already knew. Here's another of my attempts to put songs in algorithmic order:
This one tries to re-center the universe of music on any individual artist of your choosing, and then follow a vague spiralish pattern outwards in every direction at once. If we start with The Big Bopper, does it reconstruct the Music that Died that day? I don't know, I wasn't even born yet. But this math started from the The Big Bopper and rediscovered Buddy Holly and Ritchie Valens without knowing it should, so that's an interesting start. Is it "canonical"? No, of course not, the title is my rueful joke, and there's a note at the bottom that explains what I'm attempting. If you think algorithms themselves are the problem, I'm definitely part of it. I believe in attempts. If I had written the blurb for This Is, it would probably have said "An earnest algorithmic attempt at finding the maybe-essential tracks." Marketing doesn't talk that way. It isn't earnest, and it certainly isn't self-aware of how earnest it isn't.
But where self-awareness is systemically missing, we can sometimes reintroduce it ourselves. Not always, but sometimes. We don't have to let overselling trick us into thinking every oversold thing underperforms. We don't have to let premature marketing hubris scare us away from experimentation and helpful progress. Defuse their claim of essentiality with a now-knowing smirk, and those This Is playlists can be interesting. This may not be a canonical path, but it might still take us somewhere. That's something. We can let it be enough. Let algorithms work when they work for us, and fail cheerfully when they don't, and this will be yet another of all the days that music doesn't die.