[v0.1.0] 6. Before Rigging
Understanding the overall Motion Tracking process is required before the Bone Rigging, Controller, and MotionAdapter steps. It is recommended that you first read the Prerequisites page.
MYTY Kit supports motion tracking using Google's MediaPipe, a machine learning solution for live and streaming media. In particular, MYTY Kit utilizes MediaPipe's Face Mesh solution to track facial expressions in real-time. Since machine learning is employed to infer the position of facial and upper body landmarks as 3D coordinates, motion tracking requires just a single camera input, without the need for a dedicated depth sensor.
The process of taking this 3D coordinate information and connecting it to the 2D NFT Avatar can be broken down into smaller steps of bone rigging, Controller, and MotionAdapter. Before you proceed further, please familiarize yourself with the following key terms.
Terms | Definition |
---|---|
MediaPipe | Google’s machine learning solution that tracks user’s real-time movements on camera as 3D Motion tracking data. |
Motion tracking data (Motion data) | Movement data tracked by MediaPipe. |
Motion tracking module | A module for tracking a subset of movements with MediaPipe. |
Motion Adapter | A MYTY Kit tool that processes motion tracking data and connects it to another MYTY Kit tool, the Controller. |
Controller | A MYTY Kit Tool that controls the Bones and Sprites (Image) of MYTY Avatars. |
MYTY Avatar | An avatar created using the MYTY Kit. It is both a Live Avatar and an NFT Avatar, and is compatible with the MYTY Ecosystem. |
Bone rigging | The process of connecting Bones to the 2D Avatar Image to enable joint movements. |
MYTY Kit tracks the user's facial expression and movement using
MediaPipe
and delivers this information to the MYTY Avatar
through the Motion Adapter
and Controller
tools.- MediaPipe tracks the user’s movement and presents it as raw motion data.→ This raw motion data is processed by the Motion Adapter and delivered to the Controller.→ The Controller uses the processed motion data to affect applicable bones and sprites.→ A MYTY Avatar that mirrors the user’s movements is created.

- The MYTY Kit provides the following motion tracking modules. Any movements within a module can be tracked and applied to MYTY Avatar.
With the MYTY Kit v0.1, only upper body movements (Head & Face, Chest) can be applied to the MYTY Avatar. If you’d like to use more tracking modules, please contact us through the Community Channel.

- Head & Face: Module for tracking head and facial muscle movements
- Chest: Module for tracking torso movement
- Hips: Module for tracking movement of the hip joints
- Neck: Module for tracking movements of cervical spine joints
- LeftUpperArm & LeftLowerArm: Module for tracking movement of joints in the left arm
- RightUpperArm & RightLowerArm: Module for tracking movement of joints in the right arm
- LeftUpperLeg & LeftLowerLeg: Module for tracking movement of joints in the left leg
- RightUpperLeg & RightLowerLeg: Module for tracking movement of joints in the right leg
- LeftHand: Module for tracking movement of finger joints in the left hand
- LeftWrist: Module for tracking movement of joints in the left wrist
- RightHand: Module for tracking movement of finger joints in the right hand
- RightWrist: Module for tracking movement of joints in the right wrist
You can check the input values in real-time: Play Unity Project, select
MediaPipe > RiggingElement > Module Object
. Input values are displayed on the Inspector panel.

You can see real-time motion tracking data in the right-hand Inspector window.
Which Controllers and MotionAdapters will you need to make your desired animation?

- The MYTY Kit tools
Controller
andMotionAdapter
connect the user’s Motion tracking data to their MYTY Avatar.- MotionAdapter is a tool that processes raw Motion input and delivers it to the related Controller.
- Controller is a tool that uses the processed Motion input to control an avatar’s Bones and Sprites.
- Let’s find out more about the four types of Controllers included in the MYTY Kit: what they do, and which MotionAdapters they can be used with.
Bone1D Controller | Bone2D Controller | Sprite1D Controller | Sprite2D Controller | |
---|---|---|---|---|
Input | The float value of 1 MotionAdapter output. | The float values of 2 MotionAdapter outputs. | The float value of 1 MotionAdapter output. | The float values of 2 MotionAdapter outputs. |
Output | Transformation of connected bone(s). | Transformation of connected bone(s). | Selection of appropriate sprite image(s) to display. | Selection of appropriate Sprite image(s) to display. |
Related MotionAdapter | Bone2DZAdapter EyeCoverAdapter | Bone2DAdapter EyeBoneAdapter | EyeSpriteAdapter | EyeBrowAdapter MouthSpriteAdapter |
Usage Example |
|
|
|
|
There are two ways to making an eye-blinking animation.

An example of making the eyes blink using Bone animation.
You can make it using the
Bone 1D Controller
and EyeBone Adapter
- 1.Rig the EyelidBone to the Eyelid Sprite.
- 2.Create an empty GameObject (Naming example: EyeBoneController) within Controller Group, and add the Bone 1D Controller Component.
- 3.Open Bone 1D Controller Window. Specify EyeBoneController, and add EyelidBone Object to the list in the Hierarchy window.
- 4.Save the EyelidBone’s positions for when the eye is open (Max) and closed (Min).
- 5.Add EyeBone Adapter Prefab to Hierarchy. Specify Face model and EyeBoneController to the Model and Controller, respectively.
- 6.Press the Project play button to check that the animation works as intended.

An example of making the eyes blink using Frame animation.
You can make it using the
Sprite 1D Controller
and EyeSprite Adapter
.- 1.Create an empty GameObject (Naming example: EyeBoneController) within Controller Group, and add the Sprite 1D Controller Component.
- 2.Open Sprite 1D Controller Window. Specify EyeBoneController, and add Eyelid Object to the list in the Hierarchy window.
- 3.Press Auto Label to automatically bring up Sprite Library Label.
- 4.Assign Min and Max values to the Sprite Label for when the eye is open, to determine the data range within which an open-eye Sprite will be sent.
- 5.Assign Min and Max values to the Sprite Label for when the eye is closed, to determine the data range within which a closed-eye Sprite will be sent.
- 6.Add EyeBone Adapter Prefab to Hierarchy. Specify Face model and EyeBoneController to the Model and Controller, respectively.
- 7.Press the Project play button to check that the animation works as intended.
Have a question or an idea?
If you have a question to ask or an idea to share, participate in the MYTY Kit Community. We’d love to hear from you.
😄
Last modified 5mo ago