Passive video has no action stream
Scraped footage ships semantics at best. Camera pose, hand geometry, contact, and metric scale must all be synthesized before a frame is worth anything to a policy.
grip.ai converts any egocentric human video into training-ready, embodiment-specific manipulation data — metric-grounded geometry, contact, and per-frame feasibility — and proves it works with a transfer benchmark we publish rather than a claim we assert.
There is roughly 1000× more human video of hands doing tasks than teleoperated robot data. It is also the wrong shape: internet video shows what the world looks like, not what it takes to act in it. The bottleneck is not architecture — it is grounded, cross-embodiment data.
Scraped footage ships semantics at best. Camera pose, hand geometry, contact, and metric scale must all be synthesized before a frame is worth anything to a policy.
Rigs produce clean trajectories at painfully low throughput, on one embodiment, at a cost structure that cannot reach foundation-model scale.
There is no neutral answer to "how much does this data improve my robot?" Without that number, data quality is a sales adjective instead of a measurement.
One perception pass over human video produces an embodiment-independent artifact — hand keypoints, camera trajectory, wrist SE(3) motion, object tracks. Retargeting to a given robot is then a cheap, CPU-only pass. Adding a robot costs one pass, not a re-run.
Bimanual hand pose, SLAM camera trajectory, and wrist SE(3) deltas, fused into one unified schema with per-frame confidence.
Hand-size priors and hand–object contact constraints resolve monocular scale ambiguity, so actions live in meters in a world frame — validated against depth ground truth.
Human keypoints to robot joint trajectories for multi-finger hands, with joint-limit slack, penetration, and velocity checks emitted as a per-frame quality signal.
An automated QA battery gates every shard, and source, license class, and transform chain travel with the data — so redistributability is a queryable field, not tribal knowledge.
Collect, ground, retarget, train, evaluate, mine failures, collect again. What makes the loop trustworthy is where it is measured: once on the labels, before any robot is involved, and once on the policy those labels produce. A defect that the first gate misses and the second catches is the most valuable thing the loop can find.
Egocentric video plus whatever free semantic labels the source already ships.
IndependentHand pose, camera pose, object tracks, wrist motion — the expensive geometric layer.
IndependentAre the labels right? Runs at full corpus scale, on human video, with no robot at all.
IndependentA named profile — backend, DoF, URDF, joint limits. The hinge where economics are set.
Per-embodimentHuman hand to joint trajectories, CPU-only, resumable, namespaced per robot.
Per-embodimentVision-language-action policy over the wrist-delta and finger-joint action chunk.
Per-embodimentDo those labels teach a policy anything? The only signal that says useful, not merely self-consistent.
Per-embodimentError by slice, failure taxonomy — which verbs, scenes, and hands the corpus is failing.
IndependentThe same machinery as stage 1 with a targeting spec attached. This is why the loop closes cheaply.
IndependentThe loop splits deliberately: stages 1–3 and 8–9 are embodiment-independent and run once, while 4–7 are per-robot. One perception pass feeds every robot in the zoo — which is what makes cross-embodiment data economical instead of aspirational.
A robot is one declarative profile: backend, degrees of freedom, URDF, finger layout, joint limits. Perception never re-runs when a hand is added — which is why the zoo grows by a config entry and a cheap pass rather than a pipeline rebuild.
| Profile | DoF | Role |
|---|---|---|
| leap | 16 | Primary target — inexpensive, clean URDF, widely used in academic work |
| allegro | 16 | Second dexterous hand for cross-hand generalization |
| shadow | 24 | High-DoF ceiling for the retargeting problem |
| inspire | 12 | Lower-DoF dexterous point on the curve |
| sharpa | 22 | Reference profile, retained for shard compatibility |
| gripper | 1–2 | In build — parallel-jaw floor, proving transfer across a large DoF gap |
Every data vendor claims quality. We are building the neutral, reproducible scorecard that settles it: for a fixed policy and a fixed target embodiment, how many real robot demos does it take to reach a success threshold when pretrained on data source D? Report the curve, not a single point.
Robot demos required to hit a success threshold, per data source: engine-produced vs. raw human video vs. teleop-only.
Multi-finger tasks that parallel-jaw-only pipelines structurally cannot score. Our differentiator, made measurable.
Cost and wall-clock per curated episode, published. Nobody else reports it, and it is the number that decides whether a corpus is affordable.
With and without object grounding, metric scale, and feasibility filtering — so each engine component has to earn its keep in public.
We are working with a small number of teams training manipulation policies — VLA labs, dexterous-hand makers, and sim/eval vendors. If you have a robot and a data problem, we would like to run your embodiment through the engine.
Or reach us directly at info@grip-ai.net