Control Avatar (Mediapipe)

Describe how to control MYTY Avatar movement with Mediapipe

Background

We should provide motion captured results to MYTY Avatars to make them mimic us. This interface shows how to deliver the Mediapipe results.

Input parameters

Face

Motion captured result of face

Pose

Motion captured result of pose

Width

Width of camera reference

Height

Height of camera reference

Example Usage

function OnHolisticResult(result: Results)
{
  const motionData = {
    face: result.faceLandmarks,
    pose: result.poseLandmarks,
    width: cameraRef.current?.videoWidth,
    height: cameraRef.current?.videoHeight
  }
  sendMessage("MessageHandler", "ProcessCapturedResult", JSON.stringify(motionData));
}