Manipulation IK

Purpose

Control manipulator arms with differential IK, schema-native poser workflows, grasp frames, fixed-joint grasping, and hybrid IK plus joint-space motion.

Prerequisites

• Built Isaac Sim ($ISAAC_SIM_DIR or _build/linux-x86_64/release).
• NVIDIA GPU with a current driver (nvidia-smi).
• Shell env contract from isaac-sim-orchestrator: $ISAAC_SIM_DIR, $ISAAC_LAB_DIR, $WORKSPACE_DIR.

Limitations

• Targets Isaac Sim 6 / Kit 110 unless a section states otherwise.
• Does not replace official NVIDIA documentation for unsupported edge cases.

Troubleshooting

Error / symptom	Cause	Solution
Extension or import not found	Wrong $ISAAC_SIM_DIR or stale build	Point env vars at _build/linux-x86_64/release or rebuild
Black or empty frames	Missing lights or non-RTX render mode	Add dome/key light; confirm RTX / PathTracing settings
Hang on stage load or first render	MDL compile or oversized stage	Follow isolation steps in isaac-sim-troubleshooting

Patterns reference Isaac Sim docs and local example files; embedded code is a pattern sketch, not the canonical source. Always read the linked example; upstream code reflects the installed Isaac Sim version.

Available Scripts

Script	Purpose	Arguments
scripts/differential_ik_sketch.py	Conceptual sketch for custom Jacobian-based differential IK	see script --help
scripts/robot_poser_example.py	Schema-native IK + named-pose workflow using isaacsim.robot.poser	see script --help

Running scripts

From agent runtimes that expose skill execution helpers, invoke helpers with run_script():

run_script("scripts/differential_ik_sketch.py", args=["--help"])

From a built Isaac Sim tree, run the same file with ./python.sh (Linux) or python.bat (Windows) from _build/*/release, or execute shell helpers directly when they do not require the simulator.

When to use

• Control an articulated arm to reach, grasp, transport, place.
• Set up IK-based end-effector control (vs joint-space).
• Store reusable robot poses as named poses and apply them later.
• Set up grasping (FixedJoint, SurfaceGripper, contact-based).
• Validate manipulation success with a feedback loop.

Pick the right IK stack

Stack	Module	When
Differential IK on Articulation	isaacsim.core.experimental.prims.Articulation + custom Jacobian solver	specialized direct end-effector control; no maintained example wrapper
Schema-native IK + named poses	isaacsim.robot.poser.RobotPoser (LM solver via isaacsim.robot.poser.IKSolverRegistry)	offline pose authoring, persisted "pick_position" / "approach" poses
Obstacle-aware reactive	isaacsim.robot_motion.cumotion.RmpFlowController via motion-generation	dynamic obstacle avoidance, reactive trajectories
Pinocchio / PINK	isaacsim.robot_motion.pink.PinkIKController	alternative full IK stack with joint limits / task hierarchies
Lula motion generation	isaacsim.robot_motion.lula + isaacsim.robot_motion.motion_generation	legacy; supported but use one of the above for new work (rename map)

Local example files (canonical source)

Relative to $ISAAC_SIM_DIR/source/standalone_examples/api/isaacsim.robot_motion.examples/manipulation/:

Topic	Path
Follow target	follow_target.py
Pick and place	pick_place.py
Stacking	stacking.py
Multiple tasks	multiple_tasks.py

Shared implementations are under source/extensions/isaacsim.robot_motion.examples/isaacsim/robot_motion/examples/manipulation/ (ManipulationScenario, robot configurations, controllers, and interactive task backends).

Legacy/deprecated examples under $ISAAC_SIM_DIR/source/standalone_examples/deprecated/api/isaacsim.robot.manipulators/.

Migration: use isaacsim.robot_motion.examples for maintained manipulation examples.

Docs references

Topic	URL
Pick-and-place tutorial	https://docs.isaacsim.omniverse.nvidia.com/latest/robot_setup_tutorials/tutorial_pickplace_example.html
Setup a manipulator (import / assemble)	https://docs.isaacsim.omniverse.nvidia.com/latest/robot_setup_tutorials/tutorial_import_assemble_manipulator.html
Physics fundamentals (joints, schemas)	https://docs.isaacsim.omniverse.nvidia.com/latest/physics/simulation_fundamentals.html
Python scripting index	https://docs.isaacsim.omniverse.nvidia.com/latest/python_scripting/index.html

Differential IK pattern (modern Articulation)

The conceptual solver does:

1. Get the Jacobian: self.get_jacobian_matrices() (shape includes a virtual base for fixed-base robots; always slice past the base DOFs).
2. Compute the 6-DOF pose error from current EE pose to goal.
3. Apply the chosen solver to map error -> joint delta:
   • damped-least-squares: dq = J^T (J J^T + lambda^2 I)^-1 . error (default).
   • pseudoinverse, transpose, singular-value-decomposition also available.
4. Push as set_dof_position_targets(current + dq, dof_indices=arm_dofs).

differential_ik_step(arm, end_effector, end_effector_link_index, target_pos, target_quat, arm_dofs, method, damping, scale) — compute Jacobian, solve IK delta, apply via set_dof_position_targets. This is a conceptual pattern; use the maintained RMPflow examples for production control.

See scripts/differential_ik_sketch.py.

Tuning (start conservative, increase after stability):

Parameter	Conservative	Moderate	Aggressive
damping	0.1	0.05	0.01
max_delta per step	0.02 rad	0.05 rad	0.10 rad
Drive stiffness	200	400	800

Aggressive settings cause PhysX divergence under payload.

Hybrid IK + joint-space (arms with < 6 DOF)

Pure differential IK on under-actuated arms fails on:

• Large lateral transport with payload.
• Configurations near kinematic singularities.
• Sweeping through joint limits.

Pattern: IK for precision (approach, descent, final placement), joint-space interpolation for long transport (lift, traverse, descend). See the maintained manipulation controllers for sequenced examples.

Schema-native IK + Named Poses (RobotPoser)

For pose authoring, persistence, and replay use isaacsim.robot.poser. It owns the kinematic chain and IK implementation and stores named-pose data as IsaacNamedPose prims on the robot.

solve_and_store_pose(stage, robot_prim, start_prim, end_prim, target_pos, target_orient, pose_name) — validate schema, solve IK, apply joints, store as named pose. apply_stored_pose / export_all_poses for replay and persistence.

See scripts/robot_poser_example.py.

Standalone helpers (no RobotPoser needed) for FK / DOF target application:

from isaacsim.robot.poser import apply_joint_state, apply_joint_state_anchored
apply_joint_state(stage, robot_prim, joint_values)          # FK off-sim / DOF targets when playing
apply_joint_state_anchored(stage, robot_prim, joint_values, # keep anchor at world pose
                           anchor_prim=base_link_prim)

The IK solver is pluggable via isaacsim.robot.poser.IKSolverRegistry; the bundled LM solver (isaacsim.robot.poser.lm_ik) is the default.

Obstacle-aware motion controllers

For cuMotion and RMPflow controller setup, including world binding, RobotState, control-loop timing, reset sequencing, supported-robot configs, and obstacle synchronization, use motion-generation. This skill only decides when that stack is appropriate and validates the manipulation/grasp side. Use documented loaders; do not hand-construct controller configs.

Grasp frame discovery (do this first)

Most assets do not ship with a grasp frame. Before any IK:

1. Inspect the gripper USD; find the frame at the closed-finger center.
2. If absent, add a child Xform of the gripper link positioned at the grasp center; mark it with IsaacSiteAPI (ApplySiteAPI from robot_schema) so downstream tools recognize it.
3. Use that site as the IK target. The goal pose is where the object center sits when grasped, not where the gripper body is.

Grasping

FixedJoint (assisted rigid grasps only)

Pattern source: the maintained interactive pick_place_task.py plus UsdPhysics.FixedJoint. Always compute the gripper -> object relative transform at the moment of contact; never hardcode the offset. Hardcoded offsets + high stiffness produce PhysX snap and explosion.

Friction-only parallel grasps on a free rigid body are marginal: they may hold on lift but slip under transport acceleration. When the task allows an assisted grasp, prefer FixedJoint over tuning grip force or friction: attach at contact, keep the gripper visually closed, and remove the joint on release so the object settles under gravity.

For strict contact-only tasks, the object must move through the gripper's collision/contact forces. Do not use FixedJoint, D6 joints, attachments, pose-follow, object pose writes, kinematic holds, disabled dynamics, or post-release stabilization as the success path.

SurfaceGripper (vacuum / magnetic, used by UR10 example)

Pattern source: manipulation/ur10_palletizing.py (DirectSurfaceGripper).

from isaacsim.robot.surface_gripper import _surface_gripper as surface_gripper

iface = surface_gripper.acquire_surface_gripper_interface()
gripper_path = f"{end_effector_path}/SurfaceGripper"
iface.close_gripper(gripper_path)   # attach
iface.open_gripper(gripper_path)    # release
status = iface.get_gripper_status(gripper_path)  # GripperStatus.{Open,Closed}

Authored on the robot via usd.schema.isaac.robot_schema.CreateSurfaceGripper.

Grasp dataset workflow

For generating grasp datasets, see isaacsim.replicator.grasping (GraspingManager, GraspPhase) and source/standalone_examples/api/isaacsim.replicator.grasping/grasping_workflow_sdg.py.

Grasp validation (feedback loop)

Before executing or capturing a manipulation demo, validate the target object. Any object claimed as picked/pulled/placed/pushed/grasped must be physics-backed: a rigid body or articulation state, collision geometry, task-appropriate mass/inertia, and runtime pose readback (physics view or prim API). Visual-only meshes are fine for probes and debug, but the object in a final claimed result must be physics-backed.

After executing, validate object state (not just tool pose) at three gates, in order, stopping at the first failure:

Gate	Required evidence	Failure action
Grasp / contact	fingers around object; object within ~2 cm of grasp frame	adjust grasp frame offset or IK target
Lift / hold	object leaves the support and holds above the lift threshold for a measured window; XY drift bounded; velocity settles	gripper not engaged; grasp offset wrong (assisted: FixedJoint missing/wrong)
Place / release	gripper opens, object settles on support within ~5 cm; final pose/velocity meet thresholds; previously placed objects still pass	transport trajectory missed target

• Success is measured from object pose/orientation, not tool pose: a tool can converge while the object slips, ejects, or stays high. Smooth motion is necessary but not sufficient.
• Placement phase gates should use released object pose, not only end-effector convergence.
• Treat motion-phase timeouts as failures unless the phase is an intentional dwell or settle.
• Multi-object: revalidate the already-placed prefix after every later approach, place, and retreat.
• Require visual evidence from the latest run: fixed-camera video or screenshots showing the robot, object, support surface, grasp/contact area, and markers.
• Strict contact-only tasks must succeed through contact forces alone (no FixedJoint or other assistance, see above). If a gate fails, report it with numeric and visual evidence; never add hidden pose assistance to make the output look successful.

Rules

1. Read the local example first; this skill describes patterns, not syntax.
2. Always create or identify a grasp frame (IsaacSiteAPI) before IK.
3. Start conservative with IK gains; increase only after confirming stability.
4. The URDF importer applies PhysxArticulationAPI automatically; if you author articulations manually, apply it on the base link.
5. Run standalone scripts with $ISAAC_SIM_DIR/python.sh, not isaaclab.sh -p, when using SimulationApp directly.
6. Jacobian column layout: [virtual base DOFs | real DOFs] for fixed-base robots. Always slice past the virtual base.
7. Store reusable poses with store_named_pose; do not re-solve IK from scratch every session.
8. print() is unreliable in headless mode; use file writes for debug logging.
9. Validate visually at every phase. Smooth motion is not successful manipulation.
10. Hybrid IK + joint-space is the pragmatic default for arms with < 6 DOF.

Lessons (2026-04-08)

• SO-101 5-DOF: pure DLS IK converges for local moves (~0.008 m error) but diverges on lateral transport under load. Hybrid is required.
• FixedJoint with hardcoded offset + high stiffness causes PhysX snap and explosion. Compute the offset at grasp time.
• Jacobian virtual-base offset: easy to miss; breaks IK silently. Always slice past the base.