AI-powered motion capture: Deepmotion’s Animate 3D
Recently I got a chance to learn about Markerless motion capture. It’s one of many mocap systems used, but currently in rapid development and gaining popularity with free services such as Rokoko Vision. My project used Deepmotion’s Animate 3D, which has not only body tracking, but also hand- and face tracking. First, let’s take a look what motion capture is in general.
Motion-capture is not a genre. Motion-capture is a tool and technique and what we tried to do was to really use both motion-capture and traditional animation to build a system.
—Peter Jackson
Markerless technology
Motion capture relies on tracking joints. In order to track them, we need a suit with inertial sensors, physical optical markers or a markerless system that doesn’t require subjects to wear equipment. First two options are out of reach for many students and small studios due to high cost, but the latter is more accessible.
In order to try out Markerless mocap, all one needs is a camera to film in a room with enough space for desired movements.
To make the mocap project stand out, one might consider modelling and rigging a custom character.
In the background is filmed video material used as a source for tracking the 3D character I’ve designed and rigged. My filming setup was quite modest, yet the results were good enough for Deepmotion to track my movements.
My personal project
I have some experience with Motion capture suit Rokoko, so it was quite easy for me to try out Deepmotion. I started by creating a 3D character and rigging it according to armature guides given by Deepmotion. Rigging was the most challenging part of this whole project, as you need to weight paint carefully the whole mesh. I had to edit it 14 times to get it working perfect. It was worth it, because once it’s done well, you don’t need to retarget anything and it saves a lot of time in the end.
I managed to create short 10-20 second clips in just a day after finalising the rig. Now this would be pretty efficient in case I would need to quickly visualize something for a client.
Deepmotion’s default character on the left and my custom character on the right. I compared the rigs to each to other in order to find any mistakes in my work.