Training an anime colorizer (Part 1)
For the past few months, I’ve been building and training an anime colorizer on and off, and I wanted to document my personal learnings and journey so far. I originally decided to build this after feeling visually exhausted while reading the Jujutsu Kaisen manga. At a certain point, I couldn’t follow what was going on due to a wall of black lines. I’ve also tested out how other AI manga/anime colorizers out there and they have always been disappointing and they always cherry picked the best images to show on the marketing page. I thought, if we can generate fully colorized images, it must be possible to build an anime colorizer. I also figured it would be a good learning experience to try to fine-tune my own model with the latest research.
Since I’m doing this as a hobby project and limited amount of resources I had to constrain my goals to be something reasonable. I have a few thousand dollars aws credits that I want to utilize but don’t want to go over. Since my compute was going to be my main limitation I need to make sure the model would be small enough to train and to break down the problem in to simpler pieces. My first goal is to create a simple model that can colorize a 512x512 image. If I am having trouble I want to explore making the problem easier like adding color hints and reference images.
