A team of researchers from Meta AI, Tencent, MIT, and Carnegie Mellon University have developed a new AI system that can automatically animate children’s drawings of human figures. The research, published in April 2023, aims to make it fast and easy for kids to see their hand-drawn characters come to life through computer animation.
The system takes a photo of a child’s drawing as input and then detects the human figure, separates it from the background using segmentation, estimates the pose with a skeletal model, and animates the character using motion capture data. To handle the wide variability in children’s drawings, the researchers had to fine-tune existing computer vision models on new datasets of amateur drawings.
In tests, the AI system was able to generate appealing animations for 89% of hand-drawn figures after training on just 2,500 drawings. The researchers also conducted a perceptual study showing that users preferred animations that mixed perspectives between the upper and lower body, matching the “twisted perspective” often seen in kids’ art.
The team publicly released the system in December 2021 as the Animated Drawings Demo website, which went viral online. In the first 9 months, over 3.2 million people from around the world animated 6.7 million drawings, spending over 5 minutes on average per drawing.
To further research on amateur drawings, the team collected a dataset of 178,166 drawings from the demo website, called the Amateur Drawings Dataset. The large collection of annotated sketches is aimed at helping to develop and evaluate new algorithms on abstract, unconventional images.
The ability to easily animate children’s drawings could let kids bring their characters to life and tell dynamic stories. The researchers suggest future work on inferring details about the type of character, creating models to represent 3D aspects, and supporting interactive animation. Ultimately, the team hopes that AI animation tools like this will unlock new dimensions of creativity and expression for children and amateurs through their simple sketches.