DexCompose: Reusing Dexterous Policies for Multi-Task Manipulation with a Single Hand

Anonymous Authors

Policy reuse · Dexterous manipulation

DexCompose in Three Minutes

From frozen single-task policies to 16 compositions, three-skill sequences, and Franka–LEAP deployment.

Three-minute overview with English narration and captions.

Abstract

Dexterous manipulation policies can solve individual skills, but composing them to perform multiple tasks with a single hand remains difficult. Different tasks often compete over the same action dimensions, causing destructive interference between preserving an existing manipulation outcome and executing a new one. We propose DexCompose, a role-aware residual composition framework that enables the reuse of pretrained dexterous policies through explicit finger-level action ownership.

Given two pretrained full-hand policies, DexCompose first collects successful post-task states from the first skill and performs release tests over candidate finger masks to estimate which fingers are necessary for maintaining the established held state. It then trains a bounded residual stabilizer for task preservation and a context-aware residual that adapts the frozen downstream policy only within the action subspace assigned to the new task.

We evaluate the framework on 16 composite dexterous manipulation tasks spanning four object-retention skills and four downstream interactions. Our method achieves 77.4% average composite success, and ablation studies confirm that structural action ownership combined with dual asymmetric residuals is more effective than conventional policy chaining or unmasked residual correction.

Method

DexCompose treats policy composition as an action allocation problem at the embodiment level. The framework discovers which fingers must preserve Task A, releases redundant fingers for Task B, and composes the two frozen policies using residual modules that are restricted to their assigned action subspaces.

Finger Attribution

Successful Task-A hold states are replayed while candidate finger subsets are released. A task-aware LLM selector balances retention, clean release, and residual dexterity to assign fingers for the downstream interaction.

Action Ownership

The selected finger mask defines disjoint action subspaces: Task A controls preserved fingers, while Task B controls the wrist and released fingers.

Dual Residual Stabilizer

A bounded Task-A residual preserves the held state, and a Task-B residual adapts the downstream policy under the constraints imposed by the maintained grasp.

Video Demos

Demos are grouped by the behavior they illustrate: competent single-task base policies, release-test diagnostics for finger ownership, successful composite rollouts across all task pairs, and representative failure or ablation cases.

Primitive Skills

Base Policy Primitives

The primitive policies solve their own tasks before composition, so the challenge is controlled reuse under one shared hand action space.

Retention

GraspBall

Retention

PickStick

Retention

PickCan

Retention

PourMug

Downstream

OpenDoor

Downstream

PushButton

Downstream

OpenMicrowave

Downstream

TurnOnSwitch

Finger Ownership

Release-Test Diagnostics

Each row compares one successful release mask with three failure cases for the same retained skill, illustrating how DexCompose chooses fingers that preserve the held object while freeing useful dexterity.

Task A