Data Pipeline
Automated task generation before data collection and trajectory cleaning and augmentation after.
This page covers the automated engines that bookend human data collection: task generation (before), and verification, cleaning, and augmentation (after) — plus the correction aggregation path that feeds post-training.
Pipeline 1: Task Generation

Before anyone touches a controller, the system automatically constructs task specifications and simulation-ready environments through the Task Generation Engine.
How Tasks Are Built
Given a high-level instruction (e.g., "pick up the toy car on the table"), the pipeline runs through several stages:
Task Manager
Parses the language instruction into structured configurations: the manipulation goal, required object interactions, workspace context (supporting surface, clutter level), and a difficulty level ranging from 1 to 5.
Assets Manager
Sources 3D objects from a database of 2,000+ assets. Assets can be retrieved from existing repositories or generated through an image-to-3D conversion pipeline. Generated meshes undergo a normalization step that rescales objects to plausible physical dimensions before database entry.
2.5D Layout Generator
Proposes a spatial arrangement: which objects appear on the workspace and their approximate positions. Objects are categorized as task-involved (required for completion) and decorative (added for visual diversity and scene realism).
Layout Supervisor
Validates the 3D instantiation against the intended layout. When inconsistencies are detected, the system iteratively corrects through relocation, re-orientation, or partial regeneration until the scene passes validation.
Success Checker
A task-specific success checker is generated per task, encoding geometric completion conditions (e.g., "object final pose within goal zone") and constraint verification for automatic evaluation during and after teleoperation.
Scenario Sampling
After a valid task layout is constructed, the platform defines distributions over initial conditions to generate diverse task instances. These distributions include variations in object poses, clutter configurations, and camera viewpoints — enabling large-scale scenario sampling while preserving core task semantics.
As of October 2026, the system has published 1,600+ tasks covering pick-and-place, push, sort, pour, open/close, stack, articulated-object interaction, and long-horizon multi-stage sequences.
Pipeline 2: Verification & Cleaning

Verification by Replay
Before any submission is accepted, it is replayed in simulation and checked for physical consistency. Because every rollout is seeded and the scene configuration is recorded with the trajectory, the run can be reproduced exactly as recorded — and anything that cannot be reproduced is rejected. Invalid submissions never reach training, and well-formed contributions are not diluted.
Cleaning
Teleoperation data can contain artifacts: jitter from keyboard input, stalls, unintended oscillations, or uninformative motion. Arctivar provides a cleaning pipeline with two goals: filter unusable episodes and refine borderline ones.
Filtering
Trajectories are rejected if they contain corrupted or incomplete data, extreme state discontinuities, implausible state transitions, or fail task-specific success constraints. The system computes state differences between adjacent frames and ensures deltas do not exceed physically plausible thresholds. Only a small share of submissions are filtered out this way.
Static Frame Removal
Frames where the absolute variation across all joints falls below a small threshold are classified as static and removed. This eliminates artificial pauses caused by operator hesitation.
Smoothing
A Savitzky-Golay filter smooths robot action trajectories, reducing high-frequency teleoperation artifacts — abrupt corrections, hesitation jitter — while preserving the overall motion shape. Gripper joints are excluded from smoothing to maintain sharp open/close transitions.
Resampling
Teleoperation input arrives at an adaptive control rate that rises during active manipulation (dragging, auto-approach). Cubic spline interpolation resamples every trajectory to the uniform control frequency downstream policy training expects. Non-robot data (object states) bypasses the smoothing filter and is interpolated separately to maintain temporal alignment with the robot's control steps.
Results
Across benchmark tasks (LIBERO suite), the cleaning pipeline produces substantially smoother trajectories — markedly reducing mean acceleration and jerk while preserving the overall motion shape. Cleaned trajectories are better suited for policy learning and cross-simulator replay.
Pipeline 3: Realistic Augmentation
Cleaned trajectories are replayed on GPU servers running IsaacSim. The augmentation engine transforms each trajectory into a rich, multi-modal training set.
What Gets Randomized
| Domain | Variations |
|---|---|
| Lighting | Direction, intensity, color temperature, shadow properties |
| Materials & Textures | Surface materials on table, objects, background; color randomization |
| Physics | Mass, friction, compliance (minor noise to enhance robustness) |
| Camera | Multiple virtual viewpoints generating multi-view RGB, depth, and segmentation |
| Background | Scene surroundings and distractor objects |
Augmentation Outputs
Each augmented trajectory produces:
- Multi-view RGB (and optional depth/segmentation) image streams
- Randomized but semantically consistent rollouts
- Per-frame annotations (poses, contacts) when available
These outputs are formatted for direct consumption by model and imitation learning training pipelines.
Pipeline 4: Correction Aggregation
Post-training tasks produce a second data stream: human-gated corrections recorded while a trained policy executes the task. Each correction record contains the seeded initial state, the policy-driven segments, the takeover points, and the human-supplied corrective actions — anchored to the exact states where the policy left a reliable regime.
Verified corrections are aggregated into the task's correction dataset, following the DAgger recipe: supervision collected on the policy's own induced state distribution, folded back into training. Model-policy gap analysis identifies the samples with the highest corrective value, keeping dataset growth quality-aligned rather than volume-driven.
As corrections accumulate, the policy is retrained and redeployed, and deployment feedback drives the next round of generation: failure mining, weakness-targeted task construction, and adaptive distribution reweighting determine which post-training tasks open next. See Model Training for the method and Architecture for the system view.
Next: Training →