Short answer: bin picking is hard because of perception and planning, not gripping. A bin-picking system must scan a container of randomly piled parts, work out the position and orientation of individual pieces it can see only partially, choose which piece to take and exactly where to grasp it, plan a collision-free path in and out, and confirm the pick actually happened. The camera, the grasp-planning software and the gripper succeed or fail as one system — which is why bin picking is bought as a tested application, never as components.
The six steps of one pick
- Scan. A 3D sensor — structured light, stereo or laser profiling — captures a point cloud of the bin's surface. This is why the 2D-versus-3D decision matters, the subject of our guide to 2D vs 3D vision for robots.
- Identify. Software matches the visible surfaces against the part's CAD model or a trained representation, and estimates each candidate part's full pose — including the parts of it hidden under others.
- Rank. Candidate grasps are scored: parts on top before buried ones, stable grasp points before marginal ones, poses the robot can actually reach before theoretical ones.
- Plan. A path is computed that avoids the bin walls, the other parts and the gripper's own bulk — the step most demos skip showing.
- Grasp and extract. Often the slowest-looking step is deliberate: a controlled lift that lets a tangled neighbour fall back rather than come along.
- Verify and place. The system confirms something is in the gripper — by weight, vacuum, vision or a sensor check — before placing, and knows what to do when the answer is no.
The parts that defeat systems
Shiny machined metal reflects structured light into noise. Thin flat parts stack into what looks like one thick part. Oiled parts slide at the moment of grasp. Wire forms, springs and castings tangle mechanically, so the chosen part is physically linked to its neighbours. None of these is an argument against bin picking — all are arguments for testing with production parts in production condition, including the oil, rather than with the clean samples in a vendor's demo bin.
Why Amazon showed the world a sparrow
The most public large-scale picking programme is not in a machine shop but in fulfilment: Amazon announced its Sparrow handling system in 2022, describing a robot that identifies and handles individual items from the enormous variety of a fulfilment centre's inventory. The significance for factory buyers is the architecture, not the setting — Sparrow's public story is the same lesson as the machine shop's: item identification and gentle, verified handling are the hard layers, and they are software problems sitting on top of adequate hardware.
Acceptance-testing a bin-picking system
Score a pilot on your parts, in your bin, at your lighting: pick success rate across hundreds of cycles (and what counts as a failure — a dropped part and a double-pick are different events), cycle time including retries, behaviour when the bin is nearly empty, and the exception path — who is called, and how the cell recovers without a full stop. The sensing side lives in the directory's machine vision and 3D sensing category and the gripping side in end effectors and dexterous hands; a credible system supplier should be able to explain, for your part list, which side of that pair carries the risk.
Where bin picking is heading
The direction of travel is clear even if the pace is debated. Learned grasp models trained on large volumes of simulated and real picks are steadily replacing hand-tuned pose pipelines, which improves performance on exactly the awkward parts — reflective, varied, deformable — that rule-based systems handle worst. Sensing keeps getting cheaper and faster, and cells increasingly rehearse new parts in simulation before the physical trial. None of this removes the acceptance discipline above: the measure of a bin-picking system in 2026 is still what it does with your parts, in your bin, across a full shift — not what its model architecture promises.
Frequently asked questions
Why is bin picking considered so hard for robots?
Because the robot starts from disorder. Parts arrive jumbled, overlapping and partly hidden, so the system must estimate full poses from partial views, choose a grasp that will not disturb or drag neighbouring parts, and plan a path out of a container — all in seconds, thousands of times a day. Humans solve this with touch and judgement; a bin-picking cell replaces those with 3D sensing, pose estimation and grasp-ranking software, which is where most failures originate.
What kind of camera does bin picking need?
Almost always a 3D sensor — structured light, stereo vision or laser profiling — because the system needs depth to estimate part poses and grasp points in a pile. A 2D camera can locate flat, separated items, but it cannot tell which part lies on top of another or at what angle a buried edge sits. The choice between 3D technologies is covered in our comparison of 2D and 3D vision for robots.
Which parts are the hardest to bin-pick?
The recurring troublemakers are shiny or reflective metal (which corrupts depth data), thin flat parts that stack into a single apparent slab, oily parts that slip at grasp, and shapes that mechanically tangle — springs, wire forms, hooks and open castings. Dark, matte, rigid parts of moderate size are the friendly end of the spectrum. Any vendor claim should be tested against your hardest production part, not your easiest.
How should I test a bin-picking system before buying?
Run a structured pilot on your own parts in your own bins: measure pick success rate over hundreds of cycles, count how failures divide between missed grasps, drops and double picks, time the full cycle including retries, and test the nearly-empty bin, where poses are worst. Agree in advance what the system does on failure — retry, skip, or call an operator — and make the acceptance numbers part of the contract rather than the demo.