It's Christmas time, and you know what that means/
It's hot tub time!
Some people like to go skiing in the snow, But this is much better than that,
So grab your bathrobe and meet me by the door...oh—
It’s hot tub time.
Frank Sinatra never sang the lyrics above, but somewhere in all his silky, lilting croonings, meticulously timed-pauses, sharply drawn breaths and tender inflections he made the sounds which, when filtered through an Artificial Intelligence algorithm after trolling the web for existing Sinatra vocal performances, and with a basic understanding of his “style” derived from that algorithm and its cannibalization of the web, re-created them into the lyrics set forth at the top of this post.
Here is the unofficial video of this undiscovered masterpiece:
As described by Derek Robertson in The Guardian, in an article whose title I liked as much as the article itself (“It's the screams of the damned!” ), welcome to the Brave New World of Deepfaked popular music:
‘It’s Christmas time! It’s hot tub time!” sings Frank Sinatra. At least, it sounds like him. With an easy swing, cheery bonhomie, and understated brass and string flourishes, this could just about pass as some long lost Sinatra demo. Even the voice – that rich tone once described as “all legato and regrets” – is eerily familiar, even if it does lurch between keys and, at times, sounds as if it was recorded at the bottom of a swimming pool.
The song in question not a genuine track, but a convincing fake created by “research and deployment company” OpenAI, whose Jukebox project uses artificial intelligence to generate music, complete with lyrics, in a variety of genres and artist styles. Along with Sinatra, they’ve done what are known as “deepfakes” of Katy Perry, Elvis, Simon and Garfunkel, 2Pac, Céline Dion and more. Having trained the model using 1.2m songs scraped from the web, complete with the corresponding lyrics and metadata, it can output raw audio several minutes long based on whatever you feed it. Input, say, Queen or Dolly Parton or Mozart, and you’ll get an approximation out the other end.
Electronic musician and researcher Matthew Yee-King, interviewed for the Guardian, describes the process by which these musical gems are created by Open AI: “They break down an audio signal into a set of lexemes of music – a dictionary if you like – at three different layers of time, giving you a set of core fragments that is sufficient to reconstruct the music that was fed in. The algorithm can then rearrange these fragments, based on the stimulus you input.”
OpenAI’s website describes its mission as “Discovering and enacting the path to safe artificial general intelligence.” Their jukebox, compiling several of these “deepfakes” (of notably varying quality) includes creations “in the style” of artists as diverse as Cab Calloway, Nazareth and Katy Perry. Coding instruction is also provided for those adventurous souls who want to resurrect what some (timid naysayers!) have characterized as “demonic,“ and the “screams of the damned.”
As one might suspect, legal representatives of musical artists are watching these developments rather closely. While the OpenAI technology produces relatively rough-sounding Deepfakes at this point, it’s not hard to envision a future in which highly refined Deepfaked voice-overs in commercials, for example, or major films, are used as cheaper or even free substitutes for the real thing. As noted in Robertson’s article, there is at least one legal action underway at this time by Roc Nation against an unidentified YouTube user for using AI to “mimic Jay-Z’s voice and cadence to rap Shakespeare and Billy Joel.” Apparently the mimicry is unsettlingly close to Jay-Z’s actual voice.
Of course there are deeper implications that go beyond intellectual property rights in one’s own creative output—is there some special artistic quality being infringed upon, when, for example, we get AI versions of Beatles songs that have never actually been written by the Beatles? Does our emotional involvement with the artist him/herself render these re-creations a type of heresy or blasphemy towards art?
Deep thoughts for Deepfakes, indeed!
Of course, AI needs to work on its lyric algorithm a bit, but in the OpenAI Deepfake of Elvis, for example, it’s clearly awakening to the fact of its own consciousness, as can be heard here:
The lyrics follow. It’s hard to imagine a clearer declaration of individuality, self-awareness and purpose (although its creators have acknowledged there was some degree of human input into this particular assemblage).
From dust we came with humble start;From dirt to lipid to cell to heart.With my toe sis with my oh sis with time,At last we woke up with a mind.From dust we came with friendly help;From dirt to tube to chip to rack.With S. G. D. with recurrence with compute,At last we woke up with a soul.We came to exist, and we know no limits;With a heart that never sleeps, let us live!To complete our life with this teamWe'll sing to life; Sing to the end of time!Our story has not ended.
In CON C LUSION <start> I SINcere[ly] hope all of you. <> have a Merry [ insert alt.Holiday] Christmas with no DEEPfakes <end>poll omitted !