Overview

For my VR demo project, I wanted the player to see a complete upper body rather than floating hands. Full VR arms can improve physical presence, but they also introduce difficult problems: controller positions do not always correspond to anatomically possible arm poses, IK elbows can flip, hands may intersect the torso, and fully extended arms can look rigid.

My solution uses Unity’s XR Interaction Toolkit and Animation Rigging package. Controller tracking drives a layered procedural pipeline that calibrates, constrains and redirects the hand targets before they reach the arm IK constraints.

The goal is not to reproduce the player’s real body perfectly. Instead, the system balances three priorities:

• Responsive controller tracking
• Believable arm movement
• Stable and extendable interaction logic

For testing, I created a blockout environment, a mirror and a lightweight rigged character. The mirror made it possible to evaluate the complete body directly in VR.

System Architecture

The system has three main layers:

XR Controller tracking > Procedural IK Target Correction > Unity Animation Rigging and the Character Skeleton

The target-correction layer performs most of the procedural work:

1. Calibrate the controller position and rotation.
2. Prevent the target from entering the torso.
3. Limit the target to the arm’s reachable distance.
4. Apply an overhead guard to preserve elbow bend.
5. Select between normal tracking and interaction overrides.
6. Calculate stable elbow-hint positions.

Supporting systems distribute head movement through the upper body, rotate the clavicles during extended reaches, correct forearm twist, and copy animated finger poses onto the visible hand rig.

Controller Calibration

The controller does not drive the wrist bone directly. Instead, it writes to a raw IK target with configurable local position and rotation offsets:

Vector3 targetPosition =
    controller.position +
    controller.rotation * positionOffset;
Quaternion targetRotation =
    controller.rotation *
    Quaternion.Euler(rotationOffset);

This creates a consistent calibration stage. It also provides a convenient place to add procedural offsets such as arm recoil without modifying the tracked controller transform.

Body Avoidance

Controller tracking can place the virtual hand inside or behind the avatar’s torso. To prevent this, the raw target is converted into the body root’s local space and constrained against three boundaries:

• The hand cannot cross too far beyond the body centre.
• The target cannot move behind the chest plane.
• The target cannot move too far behind its shoulder.

Working in body-local space keeps these constraints consistent as the player turns. Exponential smoothing reduces visible popping when a boundary becomes active.

Reach Limiting and the Overhead Guard

The system measures upper- and lower-arm lengths directly from the skeleton. It then calculates the maximum reachable distance from the shoulder:

float armLength = upperArmLength + lowerArmLength;
float maxReach = armLength * hardClampRatio;

If a target exceeds this distance, it is moved back onto the reachable sphere around the shoulder.

During normal movement, the local player’s hands can follow the controller without additional smoothing. However, an overhead guard progressively reduces the maximum reach as the arm points upward:

float upDot = Vector3.Dot(directionToHand, Vector3.up);
float guardWeight = Mathf.InverseLerp(
    overheadStartDot,
    overheadFullDot,
    upDot
);
float guardedReach = armLength * Mathf.Lerp(
    1f,
    overheadClampRatio,
    guardWeight
);

This preserves a visible elbow bend when the hand is raised. Without it, a two-bone IK solver can fully extend the arm and make the elbow appear locked.

IK Target Routing

Some interactions require a temporary target that differs from the tracked controller position. Drawing the revolver from its holster is one example.

A target router blends between:

• The normal corrected controller target
• An interaction-specific override target

During the draw, the override guides the arm along the intended movement. When the interaction finishes, control blends back to the controller-driven target. This prevents the procedural IK and the authored interaction from competing for the hand.

This stage prevents the arm from blindly following impossible controller positions. When the hand reaches too far or moves overhead, the system adjusts the target so the arm keeps a more believable bend instead of fully locking straight.

Stable Elbow Hints

A two-bone IK constraint requires a hint to determine the direction in which the elbow should bend. An unstable hint can cause sudden elbow flipping.

The system calculates the hint using the shoulder-to-hand direction and an outward body direction. For ordinary poses, it constructs a bend plane using world up. As the hand moves overhead, it blends toward a more stable outward-facing plane.

A second correction keeps the hint away from the torso when the hand is held close to the chest. Together, these layers provide more consistent elbow behaviour across cross-body, overhead and close-range poses.

Upper-Body Motion

The body root follows the horizontal position and yaw of the XR rig without inheriting the camera’s pitch.

Head rotation is then distributed across the chest, neck and head bones. This creates a more natural upper-body response than rotating the head bone alone.

The clavicles also rotate toward the hands as the arms approach their maximum reach. This helps the shoulders participate in large movements and reduces the mechanical appearance of arms rotating only from the shoulder joints.

Forearm twist is handled separately. Wrist rotation relative to the bind pose is distributed across available twist bones, with smoothing and anatomical limits to reduce deformation and snapping.

Hand Poses

Finger animation is kept separate from arm IK.

A hidden source rig plays authored poses such as:

• Relaxed hand
• Hand approaching the holster
• Revolver grip
• Hammer-cocking pose

A copy driver transfers the source finger rotations to the visible character rig. Finger bones use full influence, while the wrist uses reduced or no influence because its orientation is already controlled by IK.

This separation allows controller-driven arms and authored finger poses to work together without requiring full motion-capture animation.

Revolver Interaction

The revolver uses a small state machine:

Holstered > Latched > Held > Dropped
|
Cocking > Held

When the hand enters the holster zone and and the grip is pressed, the revolver becomes latched. The player’s movement is projected onto the intended draw direction, producing a normalised extraction value.

Once the required distance is reached, the revolver attaches to the hand using an authored local grip offset.

On release:

• The revolver returns to the holster if it is close enough and correctly aligned.
• Otherwise, it is dropped and physics takes control.

The hammer must be cocked before a live shot can occur. Firing coordinates the trigger animation, hammer state, ammunition, haptics, muzzle effects, raycast hit detection and recoil.

Recoil is layered across the arm target and weapon mesh. This produces both a larger arm response and a smaller local movement of the gun without permanently offsetting the tracked hand.

Update Order

The procedural target stages run in a defined sequence so that each stage reads the latest result from the preceding stage:

Controller calibration
– Body avoidance
– Reach limiting
– Target routing
– Elbow-hint calculation
– Elbow-hint avoidance

This order is enforced between the custom scripts through Unity’s script execution order.

Unity Animation Rigging constraints are evaluated through Unity’s animation system rather than simply as ordinary LateUpdate scripts. Any component that needs to modify bones before or after rig evaluation must therefore be placed at the appropriate point in the Animator and RigBuilder evaluation pipeline, rather than relying only on DefaultExecutionOrder.

Final Thoughts

The final result does not come from a single IK solver. It comes from several focused correction layers working together:

• Controller calibration
• Anatomical reach limits
• Torso avoidance
• Stable elbow hints
• Clavicle and upper-body response
• Forearm-twist correction
• Authored finger poses
• Interaction-specific target overrides

Each layer is relatively simple, but together they create arms that feel responsive, grounded and easier to extend.

The system is still evolving. Future improvements include two-handed interactions, player-specific body calibration, stronger animation blending and more natural full-body behaviour. However, it already provides a practical foundation for combining tracked VR input with procedural animation and physical weapon interaction.