6. Before Rigging
This document has been updated along with MYTY Kit v1.0 release on October 2022.
Understanding the overall Motion Tracking process is required before the Bone Rigging, Controller, and MotionAdapter steps. It is recommended that you first read thePrerequisites.
MYTY ecosystem aims to enable Avatar Creators and metaverse app developers to create and use various live avatars easily. Accordingly, MYTY Kit solves the following needs of Avatar Creators and metaverse app developers.
- 1.Even if I am not familiar with the technical background of motion capture, I want to apply motion capture data to my avatars freely.
- 2.I want my avatar works well with compatibility across various metaverse apps.
- 1.I want to use various motion capture solutions to fit my multiple platform applications and apply them to MYTY Avatars.
- 2.I want to use live avatars without compromising the Avatar Creator's motion design intent.
To achieve the above purpose, MYTY Kit provides DefaultMotionTemplate and MotionAdapter. The overall motion data pipeline covered by MYTY Kit is shown below.
Motion Data Pipeline of MYTY Kit v1.0
MYTY Kit's motion capture data pipeline is divided into a part handled by metaverse app developers and a part handled by Avatar Creators.
- 1.When we move our body or face in front of the camera, that movement is captured and transmitted as data.
- 2.The captured data is categorized through TemplateBridge of MYTY Kit and flow to MotionTemplate.
- 3.Motion data delivered to MotionTemplate is used as input for MotionAdapter and flows to the Controller connected to the adapter.
- 4.The Controller expresses the movement by transforming the bones or sprites of the connected avatars.
- 5.An avatar that follows you in real-time is complete.
MYTY Kit provides a MotionTemplate that can work with various Motion capture solutions such as Google's MediaPipe. This is called DefaultMotionTemplate. By using this, Avatar Creators can obtain stable motion data automatically without applying a separate motion capture solution.
MYTY Kit's DefaultMotionTemplate supports motion tracking using Google's MediaPipe, a machine learning solution for live and streaming media. In particular, MYTY Kit utilizes MediaPipe's Face Mesh solution to track facial expressions in real-time. Since machine learning is employed to infer the position of facial and upper body landmarks as 3D coordinates, motion tracking requires just a single camera input, without the need for a dedicated depth sensor.
Click
Menu > MYTY Kit > CreateDefaultMotionTemplate to add respective object to the Hierarchy.
It is highly recommended that you test your work-in-progress (WIP) avatars using the MYTY Avatar Viewer (https://viewer.myty.space/). This platform provides the most accurate representation of what your avatar users will encounter when using your creations before they are officially released.
Alternatively, you can also test your avatars in real-time using Unity. Please refer to the document below:
The process of taking this 3D coordinate information and connecting it to the 2D NFT Avatar can be broken down into smaller steps of bone rigging, Controller, and MotionAdapter. Before you proceed further, please familiarize yourself with the following key terms.
MYTY Avatar | An avatar created using MYTY Kit. It is both a Live Avatar and an NFT Avatar, and is compatible with MYTY Ecosystem. |
Motion capture | Motion capture(Mo-Cap) is the process of capturing the movement of people, animals or objects, transferring it to a 2D or 3D model and animating it with the set of recorded movements. |
Motion tracking data (Motion data) | Movement data tracked by Mo-Cap solution. |
Motion Adapter | A MYTY Kit tool that processes motion tracking data and connects it to another MYTY Kit tool, the Controller. |
Controller | A MYTY Kit Tool that controls the Bones and Sprites (Image) of MYTY Avatars. |
Bone rigging | The process of connecting Bones to the 2D Avatar Image to enable joint movements. |
Sprite Mesh | The sprite's mesh type is all about how Unity will render your sprite in your game. The traditional way of rendering them is in full rect mode. |
In DefaultMotionTemplate of MYTY Kit, motion data is classified into 47 templates.
MYTY Kit v1.0 provides a stable motion tracking environment only for Face parameters and Head. Other motion data for full-body tracking will be provided in a later version.
Which Controllers and MotionAdapters will you need to make your desired animation?
- MYTY Kit tools
ControllerandMotionAdapterconnect the user’s Motion tracking data to their MYTY Avatar.- MotionAdapter is a tool that processes raw Motion input and delivers it to the related Controller.
- Controller is a tool that uses the processed Motion input to control an avatar’s Bones and Sprites.
- Let’s find out more about the four types of Controllers included in MYTY Kit: what they do, and which MotionAdapters they can be used with.
Bone 1D Controller | Bone 2D Controller | |
|---|---|---|
Input | The float value of 1 MotionAdapter output. | The float values of 2 MotionAdapter outputs. |
Output | Transformation of connected bone(s). | Transformation of connected bone(s). |
Related MotionAdapter | 👉 BoneTiltOrPosition
👉 (Bone) EyeBlink & EeyBlinkAll
👉 (Bone) EyeBrow & EyeBrowAll | 👉 Bone2D 👉 (Bone) Pupil & PupilAll |
Usage Example |
|
|
Sprite 1D Range Controller | Sprite 2D Nearest Controller | Rigged Sprite 2D Nearest Controller | |
|---|---|---|---|
Input | The float value of 1 MotionAdapter output. | The float value of 2 MotionAdapter output. | The float value of 2 MotionAdapter output. |
Output | Selection of appropriate sprite image(s) to display. | Selection of appropriate sprite image(s) to display. | Selection of appropriate bone-rigged sprite image(s) to display. |
Related MotionAdapter | 👉 (Sprite) EyeBlink & EyeBlinkAll
👉 (Sprite) EyeBrow & EyeBrowAll | (Sprite)MouthSprite | (Sprite)MouthSprite |
Usage Example |
|
|
|
There are two ways to make an eye-blinking animation.
An example of making the eyes blink using Bone animation.
You can make it using the
Bone 1D Controller and (Bone)EyeBlink Adapter- 1.Rig the EyelidBone to the Eyelid Sprite.
- 2.Create an empty GameObject (Naming example: EyeBoneController) within Controller Group, and add the Bone 1D Controller Component.
- 3.Open Bone 1D Controller Window. Specify EyeBoneController, and add EyelidBone Object to the list in the Hierarchy window.
- 4.Save the EyelidBone’s positions for when the eye is open (Max) and closed (Min).
- 5.Add (Bone)EyeBlink Adapter Prefab to Hierarchy. Specify the SimpleFaceParam template and EyeBoneController to the Template and Controller, respectively.
- 6.Press the Project play button to check that the animation works as intended.
An example of making the eyes blink using Frame animation.
You can make it using the
Sprite 1D Range Controller and (Sprite)EyeBlink Adapter.- 1.Create an empty GameObject (Naming example: EyeBoneController) within Controller Group, and add the Sprite 1D Range Controller Component.
- 2.Open Sprite 1D Range Controller Window. Specify EyeBoneController, and add Eyelid Object to the list in the Hierarchy window.
- 3.Press Auto Label to automatically bring up Sprite Library Label.
- 4.Assign Min and Max values to the Sprite Label for when the eye is open, to determine the data range within which an open-eye Sprite will be sent.
- 5.Assign Min and Max values to the Sprite Label for when the eye is closed, to determine the data range within which a closed-eye Sprite will be sent.
- 6.Add (Sprite)EyeBlink Adapter Prefab to Hierarchy. Specify the SimpleFaceParam template and EyeBoneController to the Template and Controller, respectively.
- 7.Press the Project play button to check that the animation works as intended.
Have a question or an idea?
If you have a question to ask or an idea to share, participate in the MYTY Kit Community. We’d love to hear from you.
😄