perceiving-sorting-pairs

A loop-head perception skill for sorting into a visible labelled 2x2 container. discover_regions runs once: it reads the four printed labels and reconstructs each compartment floor from calibrated depth. select_pair runs on every loop: it observes only the described source region, selects one remaining object, matches its type to one of the perceived labels, segments its graspable part, and returns its OBB/mask/cloud together with the matching destination OBB. No object-to-region table or simulator goal state is used.

The identity table comes from the graph

The script knows no object. The graph supplies, as literals on the select node:

• identity_hints -- the visual identities of the labels in play and the TARGET phrase to answer for each (for example "HAMMER is the cyan T-shaped tool and TARGET must be 'blue hammer handle'; ..."). When empty the prompt asks generically for the narrow graspable handle or body.
• canonical_targets -- a JSON object mapping a label to the exact TARGET phrase; a reply whose TARGET differs for a listed label is rejected. Labels absent from the table accept any TARGET.
• allowed_labels -- a comma-separated or JSON list restricting the layout to a subset of its labels, for instructions that sort only some categories.

Already-attempted labels are excluded through exclude_label_1..3; finished is returned when none remain or the model replies DONE.

The reply parser accepts semicolon or newline separators, a missing PIXEL (the box centre stands in), and coordinates given in crop pixels, 0-1000 normalised units, or full-frame pixels -- the first reading that fits the crop wins. Masks under 100 px are rejected as fragments.

The two fixed-camera views are sufficient for layout and object perception. An eye-in-hand camera may improve a downstream grasp skill, but is not required by this skill and is never used to infer the sorting association.