The Making of a Christmas Album: Hung For the Holidays
This year, I wanted to make something special to go along with my annual holiday cards. Much like a washed up singer, I decided to produce a new holiday album. And since 2023 has been the year of AI, I decided to revisit my voice cloning projects in the process.
Choosing a Playlist
I went through all of the Christmas songs on my computer, and started to figure out what songs I wanted “myself” to sing. I figured I’d choose a variety of styles and artists, like a Christmas playlist except I’m singing all of the songs. Here’s the track list I ended up using:
- All I Want For Christmas Is You
- Here Comes Santa Claus
- Feliz Navidad
- I'm Dreaming of a White Christmas
- Last Christmas
- Silver Bells
- It's Beginning To Look A Lot Like Christmas
- Santa Baby
- Rockin' Around The Christmas Tree
- Sleigh Ride
- Let it Snow
I used an app called “Ultimate Vocal Remover” which ultimately removed the vocals from all of these songs.
I didn’t select all of the songs and process them all at once, because there were some I ran into issues with. The app does a really great job of isolating vocal tracks from musical backing tracks. But it doesn’t distinguish between the lead singer and backup singers. It also sometimes mistakes saxophones and other human-y sounds for voices and includes them in the vocal tracks.
As far as I can tell, there isn’t really a way to remove lead vocals from backing vocals. When I run a combined track through the voice cloner, the model doesn’t know which vocal to follow and it can sound pretty warbly and weird. Which is hilarious, so I kept it in “All I Want For Christmas Is You,” which I couldn’t just omit from my Christmas album!
If you’re curious about the voice cloning software, I have a blog post and video on the process. (Although this time I ended up using RVC instead of so-vits-svc, but the process was pretty much the same)
There were a few songs that have a main singer and backup singers that sing at mostly different times. In these songs, I just deleted the backup vocal sections and would only keep the main vocal to clone it. Then I had to reinsert the original backup vocals so it sounded like I was singing with some old timey choruses.
After getting the vocals ready, it was a pretty simple task to recombine the backing track with my voice cloned vocals and export it to an mp3.
Making Album Art
No Christmas album is complete without an awesome album cover! I was too lazy to go take a photo of myself in holiday clothes, and I had already trained a LoRA for making images of myself in Stable Diffusion XL (which is also a video). I guess I didn’t write a blog post about it yet though.
There are AI image generators out there that do a pretty good job with composition. I think the Bing image creator (which is actually just OpenAI’s DALL·E 3) makes nicer images than Stable Diffusion out of the box. I’m guessing it’s the same for ones like Midjourney too. The problem with those generators is that it can’t make an image of a guy who looks like me (unless I got incredibly lucky and they happened to make my twin).
I’ve been refining my process of creating AI images lately. Instead of just throwing a prompt in the text box and hoping for the best, I’ve been trying to get more consistent results by using ControlNet models.
The method I used to make the album art was thus: I created a bunch of album cover images in Bing, according to the prompt:
Festive holiday album cover art of hung truong, dressed in a suit and a red santa claus hat, in front of a microphone. he is looking directly at the camera there is a realistic snowman behind him and the setting is a nighttime winter wonderland with trees and mountains in the background, and aurora borealis in the sky There is freshly fallen snow on his hat and shoulders. he is wearing glasses. award winning photography.
I made a bunch of images and tweaked the prompt until I got something that I liked. It didn’t matter that the guy doesn’t look like me. That’s what the next step is for.
I then loaded the image above into ComfyUI and ran it through a ControlNet model to extract the depth from the image. So any generated image would follow the same depth map, more or less. There are ways to make the depth map have a stronger or weaker effect on the final image, so I tweaked that too. Then I could make a bunch of images that, while all different, would have a similar depth map, which in practice means that the image will be of a guy who looks like me, in that same pose, leaning into a microphone, wearing a santa hat in front of a snowman and probably some trees. Here’s a look at some of the images I had to choose from.
Publishing My Work
I thought it would be funny to upload my AI Christmas album on to Soundcloud, so that when I finally got a hit tweet I could point people to it. So I created a new account and uploaded my entire album. I then immediately got a bunch of emails that the songs were flagged for copyright issues (even though they’re quite different).
I decided I’d have to take matters into my own hands and just host the album myself. I found an open source html5 music player thing and used it as a base for the interactive page: Hung For the Holidays
Also, I realized that there’s already a Christmas album called “Hung For the Holidays” that was released by that William Hung guy from American Idol. What are the odds!?
Conclusion
I’m glad that I could end my 2023 by making a very AI generated project. It was fun coming up with the track list and doing some editing to make sure the songs sounded okay (for the most part). I probably put way too much time into this stupid project that probably only a few people will actually listen to. It’s a pretty niche joke and I’m pretty sure only people who already know what I sound like would find it amusing.
The best part is that I can troll my wife by including some of my tracks in my regular Christmas playlist, so you can never be sure if it’ll be the original version or “Hung’s Version.”
Anyway, have a happy holiday 2023! Here’s to more interesting AI projects in 2024!
