maestro-sdk-docs

Maestro Unreal SDK Guide

In order to use the Maestro Glove in Unreal Engine 4, the Maestro Glove Unreal plugin must be used. The plugin can be downloaded from here. Full Unreal plugin documentation can be found here.

Plugin Set-up

Before the plugin can be installed, verify that the following prerequisites are met:

Plugin Installation

To install the plugin for a given project, place the Maestro Glove plugin in the Plugins folder in your project’s main directory. If that folder does not exist, create it first. The next time the project’s editor is opened the plugin will be loaded and ready to use. This can be verified by opening the Plugins Browser in editor by clicking on Plugins under the Edit dropdown on the main toolbar. If the plugin was loaded properly, it should be listed under Installed. Verify that the enabled checkbox has been clicked for the project here before continuing.

It is also possible to install the plugin on the specific version of UE4 itself if desired. If the plugin is placed into the Plugins folder for the Engine itself, usually found under Program Files\Epic Games\UE_[VERSION]\Engine, the plugin will startup and be available for use in any Unreal project using that version.

Getting Started with the Plugin

The main objective of the plugin is to make it easier for developers and designers alike to interface with the Maestro Glove. For this reason, the plugin includes a Blueprint Function Library that exposes all of our C API functions to the Blueprint Editor under the Maestro Glove category. It also includes several Blueprints that should be useful when attempting to interface with the Maestro. In order to be able to see the content included with the plugin the “Show Plugin Content” checkbox under the View Options dropdown in the Content Browser must be selected. The included content is detailed in the Useful Blueprints section below.

If you are developing an application using C++ rather than Blueprint scripting, it may be more useful to simply include both of the C headers, maestro.h and maestro_types.h, and use those exposed functions directly. These headers are stored in the Source\Maestro\Includes folder of the plugin. As the included Blueprint Function Library is simply a wrapper for the C API, using the headers directly can eliminate some unnecessary overhead.

It should be noted that the plugin calls start_maestro_detection_service on start-up, so it is unnecessary to call this function to get the glove to connect provided the plugin is enabled and starts up properly.

Useful Blueprints

The list below is not meant to be a complete and comprehensive list of all included content with the plugin, but instead is meant to detail the purpose and member variables of a few of the ones that are most likely to be used. The blueprints below are also meant as an example for how to use the plugin.

MaestroHand

Example of animating a skeletal mesh with data from the Maestro. Blueprint handles haptic responses on overlap, as well as object pickup interactions. For this reason, all actors in the scene need to have overlap events enabled in order for the blueprint to react to them. All instance variables under the Maestro category are necessary for the blueprint to function properly, and their purposes are as follows:

Which Hand - Whether this hand is a left hand or a right hand. Blueprint uses this to decide how to draw the skeletal mesh and which glove to retrieve data from.

Vibration Effect - Byte representing the vibration effect played in the fingertips during an overlap. A full effect list can be found in the C API docs here.

Pull Amplitude - Byte representing the amplitude of the force feedback motors during an overlap, 0 being no pull and 255 being full pull. It is recommended to always set this to somewhere in the approximate range 40-200. Values above about 200 are clamped down to 200. This maximum value may change at our discretion.

Index Socket - Name of the socket on the MaestroHand’s skeletal mesh that it located at the tip of the index finger. This is where the index finger’s collider will be attached for haptics and pickup logic. Socket documentation can be found here.

Middle Socket - Same as Index Socket, but for the middle finger.

Ring Socket - Same as Index Socket, but for the ring finger.

Little Socket - Same as Index Socket, but for the little (pinky) finger.

Thumb Socket - Same as Index Socket, but for the thumb.

Palm Socket - Name of the socket on the Hand’s skeletal mesh that it located at the center of the palm. This is where the palm’s collider will be attached for pickup logic.

Finger Size - Diameter of colliders used on the fingertips.

Palm Size - Size of the collider used on the palm.

Additionally, it is generally useful to be able to calibrate based on keypresses, so three key variables are exposed in the Calibration category for this purpose. All three are “None” by default as to not interfere with any other key-based game logic.

Wrist Calibration Key - Key that calibrates the wrist for this hand while held.

Proximal Calibration Key - Key that calibrates the proximal joints (fingers) for this hand while held.

Thumb Calibration Key - Key that calibrates the thumb for this hand while held.

There is also a Debug category that may prove to be useful:

Displacements Debug Actor - Actor reference that implements the MaestroPassDisplacementsInterface. Hand will pass its displacements to this actor every tick if valid. See the DisplacementsSliderWall blueprint for an example of how this can be used.

Show Pickup Colliders - Boolean that says whether or not to draw the colliders on the fingertips/palm used for haptics and pickup. Colliders will appear as a blue bubble, and will turn red whenever they overlap with a valid object.

MaestroPawn

Pawn that tracks an HMD and two MaestroHands for use in VR projects. By dragging this pawn in the scene and setting its device IDs to the correct values the pawn will handle your camera and hand tracking in the scene.

Left Tracker Device ID - Device ID of the tracker attached to the left glove.

Right Tracker Device ID - Device ID of the tracker attached to the right glove.

Default Player Height - How far the HMD starts off the floor for Eye Level tracking systems (PSVR).

MaestroPickup

Actor component that can be attached to any overlap-enabled actor to allow the MaestroHand to pick it up.

Lift Constant - Rotation ratio above which a finger cannot grab an object.

Release Constant - Rotation ratio above which defines a release between a finger and the palm.

Pinch Threshold - How much your finger has to uncurl from the position it grabbed the object to define a release, specifically when the grab is between the thumb and a finger.

FingerTipCollider

Sphere collider that is attached to the end of a finger or to the palm to enable haptics/pickup. Spawned automatically by MaestroHand.

Index - Integer index defining which finger this collider is attached to. 0 is the thumb, 1-4 are the fingers, 5 is the palm. Assigned automatically by MaestroHand.

Parent - Reference to the MaestroHand actor that owns this collider. Used to decide where to parent object upon pickup and get the glove pointer for haptic responses.

Blueprint Interfaces

The Maestro Unreal SDK includes three blueprint interfaces to pass all the glove’s displacement data, displacement velocity data, and rotation data, respectively, to other blueprints. These allow you to easily use your own classes with MaestroHand without having to modify the MaestroHand’s blueprint itself. Full Unreal Blueprint Interface documentation can be found here.

MaestroPassDisplacementsInterface

Blueprint interface used to pass a DisplacementContext between blueprints. Used by MaestroHand to pass displacements to its Displacements Debug Actor.

MaestroPassDisplacementVelocitiesInterface

Blueprint interface used to pass a DisplacementVelocityContext between blueprints. Currently unused by MaestroHand.

MaestroPassRotationsInterface

Blueprint interface used to pass a RotationContext between blueprints. Used by MaestroHand to pass rotations to its skeletal mesh.

Example VR Project

The Maestro Unreal SDK also includes an example project based off the Unreal VR Template to show pickup and object interaction with the Maestro. The example MinimalUnrealVR project can be downloaded from here and includes the plugin itself. The Project is set up to use the MaestroPawn blueprint, so its variables will have to be set properly for the glove to track.