Grasp synthesis and planning

Lesson 4 answered a question about a grasp you already have: does it hold, and how well? Grasp synthesis asks the reverse question. Given an object, a hand and a task, where should the contacts go? In practice this is a search problem. The space of possible hand poses is large, most of it is useless, and the useful part is shaped by the object's geometry, the hand's kinematics, the surrounding clutter and the task that follows. This lesson covers the main families of grasp synthesis methods, the practical constraints that prune the search, and how a chosen grasp turns into a motion the arm can execute.

The methods fall into two broad groups. Analytic methods use models of the object and contact mechanics to compute grasps that satisfy closure and quality criteria. Data-driven methods learn, from examples or experience, which grasps tend to succeed, often directly from sensor data without a full object model. Modern systems mix both: a learned network proposes candidates, and analytic checks reject those that collide or lack closure.

The grasp synthesis problem

A grasp candidate for a parallel-jaw gripper can be described by six numbers: the 3-D position of the gripper's centre point, the approach direction and the rotation of the jaw axis about that approach, plus the opening width. For a multi-finger hand, the finger joint angles add more dimensions. The synthesis problem is to find candidates that satisfy four kinds of requirement at once: the grasp must hold the object (closure and quality), the hand must be able to reach the pose without colliding (reachability and collision), the object must not be damaged (force and contact area limits), and the grasp must suit what happens next (task compatibility).

Each requirement prunes the space. Closure eliminates most hand poses that touch the object at all. Reachability and collisions eliminate grasps on the side of a part facing a wall or the bin floor. Task compatibility eliminates grasps that would block the feature that must later be inserted, or that would leave the object in the wrong orientation for placing. A good planner applies the cheap filters first and the expensive ones last.

Analytic synthesis and antipodal sampling

For parallel-jaw grippers, antipodal sampling is the workhorse analytic method. Sample a point on the object surface, cast a ray inward along the surface normal until it exits the object, and record the exit point. If the angles between the line joining the two points and each surface normal are within the friction cone half-angle, the pair is antipodal, and lesson 4 showed that implies force closure in the plane. Check that the distance between the points fits inside the gripper's opening range, give the candidate a score, and repeat thousands of times.

Scoring usually combines several terms: the misalignment angle (smaller is more robust to friction uncertainty), the distance of the grasp axis from the centre of mass (smaller reduces the moment gravity applies about the grasp), the width compared with the gripper's range (a width near the limit leaves no tolerance), and clearance from neighbouring objects. Weighted sums are common; the weights are tuned on test objects.

# Antipodal sampling on an ellipse (semi-axes a, b): score pairs by misalignment and offset from the middle
import numpy as np
rng = np.random.default_rng(7)
a, b, mu, max_open = 0.05, 0.03, 0.4, 0.09          # m
half = np.degrees(np.arctan(mu))
def point(t):  return np.array([a*np.cos(t), b*np.sin(t)])
def normal_in(t):                                     # inward unit normal
    n = -np.array([np.cos(t)/a, np.sin(t)/b]); return n / np.linalg.norm(n)
def exit_point(p, d):                                 # second intersection of ray p + s d with the ellipse
    A = (d[0]/a)**2 + (d[1]/b)**2; B = 2*(p[0]*d[0]/a**2 + p[1]*d[1]/b**2)
    return p + (-B / A) * d                           # the other root is s = 0
best, kept = [], 0
for t in rng.uniform(0, 2*np.pi, 2000):
    p1, n1 = point(t), normal_in(t)
    p2 = exit_point(p1, n1)
    t2 = np.arctan2(p2[1]/b, p2[0]/a); n2 = normal_in(t2)
    axis = (p2 - p1) / np.linalg.norm(p2 - p1)
    ang1 = np.degrees(np.arccos(np.clip(axis @ n1, -1, 1)))
    ang2 = np.degrees(np.arccos(np.clip(-axis @ n2, -1, 1)))
    width = np.linalg.norm(p2 - p1)
    if max(ang1, ang2) < half and width < max_open:
        kept += 1
        mid = (p1 + p2) / 2
        score = (half - max(ang1, ang2)) / half - 5 * np.linalg.norm(mid) / a
        best.append((score, width, max(ang1, ang2), np.linalg.norm(mid)))
best.sort(reverse=True)
print(f"cone half-angle {half:.1f} deg; kept {kept} of 2000 samples")
for s, w, ang, off in best[:3]:
    print(f"score {s:+.2f}  width {1000*w:.1f} mm  worst angle {ang:.1f} deg  offset from middle {1000*off:.1f} mm")

cone half-angle 21.8 deg; kept 663 of 2000 samples
score +1.00  width 60.0 mm  worst angle 0.0 deg  offset from middle 0.0 mm
score +1.00  width 60.0 mm  worst angle 0.0 deg  offset from middle 0.0 mm
score +0.99  width 60.0 mm  worst angle 0.0 deg  offset from middle 0.0 mm

The best candidates pinch the ellipse straight across its short axis, 60 mm wide, where the surface normals point exactly at each other and the grasp passes through the centre. The equally well-aligned pinch along the long axis is 100 mm wide, more than the 90 mm opening, so it is rejected, and many oblique pairs fail the angle test; only about a third of the samples survive. On a real part the same sampler runs on a mesh or point cloud, and the surface normals come from the model or are estimated from neighbouring points.

Grasp databases and template matching

When the same objects recur, as in many factories, grasps can be planned once offline and stored. A grasp database associates each known object model with a ranked list of grasps in the object's frame. At run time the perception system estimates the object's pose (ROB-330), the stored grasps are transformed into the world frame, and the first one that is reachable and collision-free is executed. This approach is fast and predictable, and engineers can inspect and approve every grasp. Its weakness is that it requires an accurate model and pose for every object.

Template and shape-primitive methods sit between databases and full analytic search. An object is approximated by boxes, cylinders and spheres, each with a set of standard grasps: pinch a box across its narrow dimension, wrap a cylinder around its axis, pinch a sphere through its centre. These rules handle many household and warehouse items surprisingly well and are easy to explain.

Reachability, approach and retreat

A grasp that holds the object is useless if the arm cannot get there. Every candidate passes through an inverse kinematics check (ROB-210): is there a joint configuration that places the gripper at the grasp pose within joint limits? Many grasps around an object are unreachable, or reachable only at awkward configurations near singularities. Planners often prefer grasps with a comfortable configuration, measured by distance from joint limits or by manipulability, because those leave room for corrections.

The gripper also has to arrive and leave without hitting anything. The standard pattern is a pre-grasp pose a few centimetres back along the approach direction, a straight-line approach to the grasp pose, closing, a straight lift or retreat along a chosen direction, and then a free-space motion to the next waypoint. Each straight segment is checked for collisions with the scene and, during retreat, with the object now held in the gripper. Approach directions roughly along the surface normal at the grasp point are most forgiving, because small position errors then move the fingers along the surface rather than into it.

Robust grasp planning

Because the pose estimate, the friction coefficient and the object model are all uncertain, robust planners evaluate a grasp not once but many times under sampled perturbations. Shift the grasp pose by random errors drawn from the perception uncertainty, sample \( \mu \) from its measured range, recompute closure, and count how often the grasp survives. The fraction of successful samples estimates the probability of success. Ranking grasps by this probability, rather than by a single nominal quality value, prefers grasps that sit in broad, forgiving regions over grasps that are excellent only at one exact pose.

Data-driven grasp synthesis

Learned grasp detectors take an image or point cloud and output grasp candidates directly, together with a predicted success score. Training data come from simulation, where millions of grasps can be evaluated with analytic metrics and physics engines, from human labels, or from physical robots trying grasps and recording results. Such systems handle novel objects without a model, which is why they dominate mixed-item warehouse picking. Lesson 6 looks at their perception side in more detail.

Learned methods have their own failure modes. They can be confidently wrong on objects unlike their training data, such as transparent items or unusually shaped packages, and their scores are not guaranteed to be calibrated probabilities. Good practice combines them with analytic checks: reject candidates that collide, that exceed the gripper width, or that fail a simple closure test on the observed point cloud, and track real success rates by object type so weak categories can be retrained or routed to another station.

Task-oriented grasping

How an object is grasped should depend on what happens next. A mug picked to be washed can be grasped anywhere; a mug picked to be handed to a person should leave the handle free; a screwdriver picked to drive a screw must be grasped by its handle with the shaft aligned for torque. Task-oriented grasping adds constraints or scoring terms that encode these needs: keep a feature exposed, align an axis with the tool direction, place the grasp so the object can be set down in the required orientation without regrasping, or resist the specific wrenches the task will apply.

Placement is the most common task constraint in industry. If a part must be inserted into a fixture with a particular face down, only grasps that leave that face accessible and that let the arm reach the placement pose are useful. Planners therefore search grasp and placement together, a theme lesson 8 develops when regrasping becomes necessary.

Putting synthesis into a pipeline

A practical grasp planning pipeline for a parallel-jaw gripper typically runs: segment the target object; generate a few hundred candidates by sampling or by a learned network; filter by gripper width and simple collision checks against the scene; check reachability with inverse kinematics; score the survivors on quality, robustness and task terms; and try them in order of score until one succeeds, with the pre-grasp, approach, close, verify and retreat sequence. Logging every attempt, with the outcome, builds the data needed to improve the scoring over time.