[v0.1.0] 9. Apply Motion Tracking

What is the Motion Adapter?

• The Motion Adapter takes motion capture data from MediaPipe, processes it, and feeds it to the Controllers.

• There are different Motion Adapters for different types of data processing. Data processing type in turn depends on the type of motion data and the type of Controller(s) that will be used.

• Theoretically, if there are M number of motion data types that can be tracked and N number of Controllers, a total of M*N number of Motion Adapters can exist.

• The MYTY Kit v0.1 provides 10 types of Motion Adapters to handle the more common animation scenarios.

Theoretically, Motion Adapters will allow you to take various motion data and freely connect them to different types of Controllers. But, we are still working through all animation possibilities with the MYTY Kit v0.1. Changing the Motion Adapter prefabs provided in the MYTY kit may lead to compatibility issues.

If the MYTY Kit doesn’t have an appropriate Motion Adapter for the animation that you’d like to make, please contact the MYTY Team through the Community channel or email.

Types of Motion Adapters in the MYTY Kit

• You can see all MotionAdapters under Assets > MYTYKit > MotionAdapter folder.

• You can select an appropriate Motion Adapter by taking into account the Controller and the Controller’s input (Motion data) that you’ll use.

Adapter Name	Motion Input	Related Controller
Bone2DAdapter	Joint Model (Chest or Head)	Bone2D
Bone2DZRotate	Joint Model (Chest or Head)	Bone2D
EyeBrow	Face Model; leftEyeBrow, rightEyeBrow	Bone2D, Sprite2D
LeftEyeBone	Face Model; leftEye	Bone1D
LeftEyeCover	Face Model; leftEye	Bone1D
LeftEyeSprite	Face Model; leftEye	Sprite1D
MouthSprite	Face Model; mouthX, mouthY	Bone2D, Sprite2D
RightEyeBone	Face Model; rightEye	Bone1D
RightEyeCover	Face Model; rightEye	Bone1D
RightEyeSprite	Face Model; rightEye	Sprite1D

Inspector windows for each Motion Adapter

• Each Motion Adapter has its own Inspector window.

• For all Motion Adapters, it is recommended that you do not touch the areas shaded in red, as they are settings for advanced animation.

• Clicking on the ◉icon next to an input box will bring up an object list from which you can select an option.

Bone2DAdapter

Variables

• Joint: Select the joint model script to be applied. The MYTY Kit v0.1 provides the Head.cs and Chest.cs scripts.

• From: The direction that will be referenced for the avatar’s up vector.

• Plane: Motion will be applied on this plane.

• Flip: Changes the up vector to the opposite direction.

• Controller: Select the Controller to be applied to the joint. For the objects, you can select ones with a Bone2DController component from the Hierarchy window.

Bone2DZRotate

Variables

• Joint: Select the joint model script to be applied. The MYTY Kit v0.1 provides the Head.cs and Chest.cs scripts.

• From: The direction that will be referenced for the avatar’s up vector.

• Plane: Motion will be applied on this plane.

• Flip: Changes the up vector to the opposite direction.

• Controller: Select the Controller to be applied to the joint. For the objects, you can select ones with a Bone2DController component from the Hierarchy window.

EyeBrow

Variables

• Face: Select the Face Model.cs as the Motion Input data. This will bring the leftEyeBrow and the rightEyeBrow's motion data.

• Con: Select the Controller to be applied to the EyeBrow motion data. For the objects, you can select ones with either a Bone2DController or Sprite2DController component from the Hierarchy window.

LeftEyeBone & RightEyeBone

Variables

• Face: Select the Face Model.cs as the Motion Input data. This will bring the leftPupil and rightPupil motion data.

• Con: Select the Controller to be applied to the leftPupil or rightPupil motion data. For the objects, you can select ones with either a Bone1DController or Sprite1DController component from the Hierarchy window.

LeftEyeCover & RightEyeCover

Variables

• Face: Select the Face Model.cs as the Motion Input data. This will bring the leftEye and rightEye motion data.

• Con: Select the Controller to be applied to the leftEye or rightEye motion data. For the objects, you can select ones with either a Bone1DController or Sprite1DController component from the Hierarchy window.

LeftEyeSprite & RightEyeSprite

Variables

• Face: Select the Face Model.cs as the Motion Input data. This will bring the leftEye and rightEye motion data.

• Con: Select the Controller to be applied to the leftEye or rightEye motion data. For the objects, you can select ones with either a Bone1DController or Sprite1DController component from the Hierarchy window.

MouthSprite Adapter

Variables

• Face: Select the Face Model.cs as the Motion Input data. This will bring the mouthX and mouthY motion data.

• Con: Select the Controller to be applied to the mouthX and mouthY Motion data. For the objects, you can select ones with either a Bone2DController or Sprite2DController component from the Hierarchy window.

How to apply Motion Adapters

Add MediaPipe Prefab

1. In the Project window, Select Assets > MYTYKit > Prefabs. Drag Mediapipe prefab to a blank area in the Hierarchy window.

2. You can check the incoming input values in real-time, by selecting a RiggingElement object and clicking ▶️(Play).

Does a MediaPipe object fail to track your movement? Refer to our Troubleshooting & FAQfor a solution.

Add Motion Adapters

1. Right-click on a blank area of the Hierarchy window, and select Create Component to create a GameObject for the MotionAdapter's Group.

2. In the Inspector window, click Add Component and apply the Motion Adapter Group Component.

3. In the Project window, go to Assets > MYTYKit > MotionAdapter, click and drag the prefabs you want to apply into the MotionAdapter object in the Hierarchy window.

4. In the Inspector window, adjust the settings for each MotionAdapter, selecting the appropriate scripts, Controllers, and objects.

5. If nothing appears when you click the ◉ icon, it may be due to either of the following:

1. The MediaPipe prefab is not in the Hierarchy window → Check Add MediaPipe Prefab

2. The appropriate Script(s) are not applied to the Controller → Check #2.-create-controller

6. Click ▶️(Play) to check that Motion Tracking works as intended.

Repeat steps 1 to 6 for each animation.

Have a question or an idea? If you have a question to ask or an idea to share, participate in the MYTY Kit Community. We’d love to hear from you.😄