GENOMA LABS / Simulation and Real-to-Sim

Simulation and Real-to-Sim

Most robot learning fails not in the policy but in the world it trains on. GENOMA Labs builds simulation models that mirror the physical state of working machines into synthetic worlds, so behavior can be trained, stressed, and measured before it reaches hardware.

The loop runs in both directions. Real machines stream perception and state into simulation, simulation returns training data and tested behavior, and nothing ships back to hardware without passing a deterministic gate. We verify by running, not by inspection.

Mirroring physical state into learnable worlds

Real-to-sim starts with a faithful copy. The tracked crawler machines in our visual fleet are reconstructed in simulation, and recorded machine state is mirrored into that copy. Once the mirror holds, the same scene can be replayed, perturbed, and multiplied into synthetic variants that no field program could afford to stage.

These synthetic worlds are training material. They are one of the main data sources for our robotic world models, which are coming to fal.ai, and they give every downstream policy a place to fail cheaply before it touches a machine.

SENSE, perception as ground truth

A mirror is only as good as the state estimate behind it. SENSE is GENOMA's own perception console, built to hold that estimate in real time:

  • Fuses lidar point clouds and radar returns into one live scene.
  • Tracks vehicles, walkers, and stationary entities around the machine.
  • Carries live crane telemetry channels alongside the perception feed.

The same fused scene serves two consumers at once. Operators watch it as a console, and the simulation can be checked against what the sensors actually saw, which keeps the mirrored world honest about what the machine actually saw.

Data acquisition loops, from play to gate

Simulation needs data about how lifting actually gets done, and that data is expensive to collect on real jobsites. Craneformerz is our answer: an AI videogame and data-acquisition platform for robotic lifting, where play itself produces structured task data.

Acquired and synthetic data flow into the same training systems that drive our reinforcement learning and vision-language-action work. Every model generation coming out of those loops is checked by deterministic gates before it ships. If a behavior cannot pass a gate we can run, it does not leave the lab.

Questions

What does real-to-sim mean at GENOMA Labs?

It means mirroring the live state of a physical machine into a simulated copy of it and its surroundings. Once the mirror holds, that scene becomes training material: it can be replayed, varied, and used to train and evaluate models at a scale the field cannot support.

What is the SENSE perception console?

SENSE is GENOMA's own perception console. It fuses lidar point clouds and radar returns in real time, tracks vehicles, walkers, and stationary entities, and carries live crane telemetry channels. The same fused scene drives both the operator view and the simulation's ground truth.

How does Craneformerz relate to simulation?

Craneformerz is an AI videogame and a data-acquisition platform for robotic lifting, live at craneformers.io. Play generates structured data about lifting tasks, and that data feeds the synthetic training worlds and learning loops described on this page.