As I was working on various parts of Seas of Reverence, my son told me that another game maker he follows had created a motion capture system for his Quest-based game using just the controllers and some inverse kinematics, and said I should do the same.
Here is the initial phase of that subproject that I’ve worked on for the last couple weeks:
This is the next iteration of the motion capture system I’m creating for Seas of Reverence. I have decided to make it a standalone utility rather than integrating it into the main game. This way I can reuse it for other projects as well.
It is a client/server program, to accommodate the Quest’s security requirements. I have hard-coded the retrieval of the rigged skeleton mesh, but the rest is done dynamically based on the contents of the mesh.
The interface is going to be done through a virtual tablet, and I demonstrate the basic functionality of the tablet.
It’s a very short video, so you may have to watch it more than once to see the comments.
This shows the progress I’ve made on the virtual tablet interface this morning. The interface can be made to be much slicker, but I’m more concerned with getting features implemented than making them look professional. After all, I’m targeting this program toward in-house use rather than releasing it for sale.
However, I might change my mind about licensing it later down the road. Nothing it set in stone, and this is very early in development, so who knows…
This video is dedicated to showing how the virtual tablet is used to load meshes from the server. The program is no longer hard coded to load the mesh. Meshes are loaded on demand using the virtual tablet.
Meshes can be rotated with the left controller joystick. The virtual tablet has shortcuts for rotating 180 degrees on the X and Y axes. I forgot I was working in 3D, and therefore neglected to add a 180 degree rotation shortcut on the Z axis. Whoops.
There can be a lot of bone names, and those names will clutter the view and make many of them unreadable. I’ve made it so bone names can be hidden via the tablet, but still be viewed by pointing at the relevant bone.
On the way to animating movement, I have implemented multiple joint selection. This will become necessary for animation down the road.
The list of existing mesh animations can now be shown to the user through the virtual tablet. Animations can’t yet be selected, but that’s coming soon.
Users can now click on the current mesh’s animations to have them play. There are unexplained errors in the animations, so I’ll have to find out why that is. I suspect it has something to do with the mesh not being reset to its initial state prior to the animation playing, but that’s just a guess.
If anyone knows why this would happen, I would love to hear from you.
I have changed the program to reload the current mesh upon selection of an animation, and that causes all of the animations to play correctly.The AnimationPlayer changes its internal state while an animation plays, and there is no way to reset that internal state when playing a different animation.
I find that behavior puzzling since playing different animations through an AnimationPlayer in a game seems to work fine. Or maybe I’ve just been lucky all these years…
The solution was to reload the mesh upon selecting an animation.
It may not be possible to reset because the tracks of different animations are different, that is, some animations do not use the tracks used by other animations. I don’t know if the animation switch will use the reset animation first, if so, you can add all tracks in the reset animation. You mentioned that it seems to work normally to play different animations through an Animation player in a game. It may be for this reason. When will you share this tool?
Thanks for your insight. It makes sense that different animations may use different tracks. Since I know that internal states of the AnimationPlayer change during playback, it explains the weird behavior.
I have elected to reload the entire mesh when an animation is selected, which also re-reads the mesh’s AnimationPlayer. That eliminates the errors.
I have not decided when to make this tool (I call it “More Capture”) available. It has quite a way to go before it is useful for its intended purpose.
If you look at my more recent postings, you’ll see that I am removing this feature from Seas of Reverence. Instead, I am moving the motion capture functions to a standalone program (More Capture).
When joints are selected, the user can move them using the right controller (left controller support should be fairly easy to include later).
The next step is to have the joint movements reflected in the mesh’s armature.
It is like pinpointing (so it seems to me)… .
Maybe you could add spherical boundariers around any joint with attraction so it would “click” to it when near… ( so you have your guide… and you are pointing with some tinkerbel-isch thingy that is atraktes to the most nearby joint )
In my feel that would make a more snappy response mechanism than exactly pointing to a joint…
Just a brainfart… 
That’s a good idea. I’ll see what I can do.
When joints are moved, they normally move directly with whichever controller has selected them. This results in circular motion, which will move the joints in ways you normally don’t want. In those cases, the joints can now be constrained to move along selected axes.
I also made the joints’ selection area easier to hit.
The selected animation’s playback position can be controlled by pointing at the slider bar and dragging the slider button.
The slider button’s position can also be moved around via its position button. The slider works regardless of its position and orientation.
I thought this was going to be harder than it turned out to be, which is why I didn’t put it into my earlier video. In my prior video, you couldn’t help but notice that the bone joints didn’t move with the animation. Now they do.