Making faces: How to train an AI on your face to create silly portraits

By now, you've read a lot about generative AI technologies such as Midjourney and Stable Diffusion, which translate text input into images in seconds. If you're anything like me, you immediately wondered how you could use that technology to slap your face onto the Mona Lisa or Captain America. After all, who doesn't want to be America's ass?

I have a long history of putting my face on things. Previously, doing so was a painstaking process of finding or taking a picture with the right angle and expression and then using Photoshop to graft my face onto the original. While I considered the results demented yet worthwhile, the process required a lot of time. But with Stable Diffusion and Dreambooth, I'm now able to train a model on my face and then paste it onto anything my strange heart desires.

In this walkthrough, I'll show you how to install Stable Diffusion locally on your computer, train Dreambooth on your face, and generate so many pictures of yourself that your friends and family will eventually block you to stop the deluge of silly photos. The entire process will take about two hours from start to finish, with the bulk of the time spent babysitting a Google Colab notebook while it trains on your images.

Before we begin, a couple of notes:

System specs

For this walkthrough, I'm working on a Windows computer with an Nvidia 3080Ti that has 12GB VRAM. To run Stable Diffusion, you should have an Nvidia graphics card with a minimum of 4GB of video RAM. Stable Diffusion can run on Linux systems, Macs that have an M1 or M2 chip, and AMD GPUs, and you can generate images using only the CPU. Those methods require some tinkering, though, so for the purposes of this walkthrough, a Windows machine with an Nvidia GPU is preferred.

Ethical concerns

Further Reading

Lastly, tech like Stable Diffusion is simultaneously exciting and terrifying. It's exciting because it gives people like me, who peaked artistically with fingerpaints in kindergarten, the ability to create the images I imagine. But it's terrifying because it can be used to create frighteningly realistic propaganda and deepfakes with the potential to ruin people's lives. So you should only train Stable Diffusion on photos of yourself or someone who has given you consent. Period.

Now, who's ready to do this?