Control Avatar (Mediapipe)

Describe how to control MYTY Avatar movement with Mediapipe

Background
We should provide motion captured results to MYTY Avatars to make them mimic us. This interface shows how to deliver the Mediapipe results.
Input parameters
Face
Motion captured result of face
Pose
Motion captured result of pose
Width
Width of camera reference
Height
Height of camera reference
Example Usage
function OnHolisticResult(result: Results)
{
  const motionData = {
    face: result.faceLandmarks,
    pose: result.poseLandmarks,
    width: cameraRef.current?.videoWidth,
    height: cameraRef.current?.videoHeight
  }
  sendMessage("MessageHandler", "ProcessCapturedResult", JSON.stringify(motionData));
}

Choose View Mode

Control Avatar (ARKit)

Last modified 4mo ago