Vision
Hand-Eye Calibration
Hand-eye calibration is the procedure that solves the rigid transform between a robot's camera frame and the robot itself, so pixel detections can be converted into robot coordinates the arm can actually move to.
What it is
Hand-eye calibration finds the fixed geometric relationship between a vision system and the robot's kinematics. A camera reports where a feature sits in its own image or camera frame; the robot plans and moves in its base or flange frame. Hand-eye calibration is the transform that ties those two worlds together so a detection at some pixel becomes a pose the arm can reach.
There are two setups, and they solve for different transforms. In eye-in-hand, the camera is mounted on the robot wrist and moves with it, so you solve for the transform from the flange (tool mount) to the camera. In eye-to-hand, the camera is bolted to a fixture looking at the workspace, and you solve for the transform from the robot base to the camera. Confusing the two is the most common way a calibration goes wrong.
How it actually works
You collect a set of paired observations: the robot moves to several known poses, and at each one the camera measures a known target (a checkerboard or dot grid whose geometry you supply). Each pose gives one equation relating the robot's motion between stations to the camera's observed motion of the target. Classic formulations write this as AX = XB, where A is the robot's flange-to-flange motion between two poses, B is the camera's target-to-target motion, and X is the unknown hand-eye transform you're solving for. Solve the stack of equations and you get the rotation and translation of X.
Accuracy scales with how varied your poses are. A dozen poses that mostly translate and barely rotate under-constrain the rotation part of X, so techs deliberately spread poses across large orientation changes and different standoffs. Once X is solved, a detection at, say, pixel (640, 480) maps through the camera intrinsics and the hand-eye transform into an XYZ pose in the robot frame, which is what feeds the motion plan. If the target board or robot poses were sloppy, that error rides straight through into every part the robot picks.
How it differs
- TCP / Tool Center Point · A TCP defines where the tool's working point sits relative to the flange; hand-eye calibration defines where the camera sits relative to the flange (eye-in-hand) or base (eye-to-hand). Both are frame offsets, but one describes the tool geometry and the other describes the sensor geometry. You can have a perfect TCP and still miss the part if the hand-eye transform is wrong.
- User Frame · A user frame is taught by touching known points to define a work coordinate system. Hand-eye calibration instead solves a camera-to-robot transform from vision observations. The end goal overlaps (both express positions in robot coordinates) but hand-eye calibration is the vision-specific step that lets the camera speak the robot's language.
Where you meet it in the field
- 3-point user frame setup · Hand-eye calibration produces the camera-to-robot transform; a taught user frame is the manual cousin that anchors vision results into the same robot coordinate system.
- FANUC CRX-10iA · A collaborative arm frequently run as a vision-guided pick station, where an eye-in-hand or eye-to-hand calibration is what turns camera detections into reachable poses.
Common questions
- Do I need to recalibrate hand-eye if I only move the camera, not the robot?
- Yes. The whole point is the fixed relationship between camera and robot. If you loosen, reposition, or re-mount the camera (or change its lens or focus enough to shift the intrinsics), the transform you solved no longer holds and detections will land off. A knocked bracket is a common silent cause of drift.
- Eye-in-hand or eye-to-hand, which is better?
- Neither is universally better; they solve different problems. Eye-in-hand lets one camera inspect many locations by moving the arm, and it keeps standoff consistent, but the camera adds mass and cabling to the wrist. Eye-to-hand gives a fixed, stable view of the whole workspace and never moves, but resolution is fixed by where you bolted it. Pick based on part size, cycle needs, and whether the arm must reach into varied spots.
- My calibration reprojection error is tiny but picks are still off. Why?
- Low reprojection error only proves the camera model fits its own target images. It doesn't prove the robot poses you fed in were accurate. A bad TCP, backlash, a target that shifted between stations, or poses that barely rotated can all leave a small reprojection number while the hand-eye transform itself is wrong. Check the pose spread and the robot-side accuracy, not just the vision residual.