2026-08-12 · CoRL · 11 citations

Enhancing Visual Domain Robustness in Behaviour Cloning via Saliency-Guided Augmentation

Zheyu Zhuang, Ruiyu Wang, Nils Ingelhag, Ville Kyrki, Danica Kragic

Imitation & diffusion policies Safety & evaluation

Published at CoRL (the arXiv record still lists it as a preprint). 11 citations, 1 of them influential, as of the last refresh.

Abstract

In vision-based behavior cloning (BC), conventional image augmentations such as Random Crop and Color Jitter often fall short under substantial visual domain shifts, including changes in shadows, distractors, and backgrounds. Superimposition-based augmentations, which blend in-domain and out-of-domain images, have shown promise for improving generalization in computer vision, but their suitability for BC remains uncertain because task-critical semantics, spatiotemporal relationships, and agent-target interactions must be preserved. To address this, we introduce RoboSaGA, a Saliency-Guided Augmentation method within the superimposition family tailored for vision-based BC. RoboSaGA dynamically adjusts augmentation intensity at the pixel level using policy-driven saliency, enabling aggressive augmentation in task-irrelevant regions while preserving task-critical information. It integrates seamlessly into existing architectures without requiring structural modifications or additional learning objectives. Experiments in both simulated and real-world settings show that RoboSaGA preserves in-domain performance while substantially improving robustness to visual domain shifts, including distractor and background changes, as well as lighting and shadow variations. Code is available at https://github.com/Zheyu-Zhuang/RoboSaGA.

arXiv comment: Accepted at the Conference on Robot Learning (CoRL) 2024

Ten-minute slide kit

Six slides is the whole talk: what was broken, what people tried, what these authors did, what the numbers say, where it falls over, and the sentence people should remember.

SLIDE 1

The problem

Vision-based behaviour cloning (BC) has made significant strides in transferring behaviour from expert demonstrations to robot skills \[1, [2\]](#page-8-0). These demonstrations involve intricate spatial-temporal interactions between the agent and its environment and often require integration among multiple sensory modalities. However, given the high costs associated with collecting demonstration data, especially in real-world settings \[3, [4\]](#page-8-0), datasets often prioritise task-related variability over…

SLIDE 2

What came before

A typical vision-based BC policy πξ {zj} t j=t−T , parameterised by ξ, derives actions from a temporal sequence of observation embeddings z over a window of length T [\[7\]](#page-8-0). At time step t, the observation embedding zt is defined as: where x vi t and pt represent the visual input from the i th visual modality or camera view, and the proprioceptive state. The visual encoders fθi , parameterised by θi ,…

SLIDE 3

The method

Not recoverable from the parsed text. Read this section in the paper yourself.

SLIDE 4

What they measured

Manipulation Tasks and Out-of-Domain Settings. As illustrated in *Fig*.2, in simulations, we use expert human demonstrations from the Robomimic environment [\[7\]](#page-8-0) to evaluate four tasks of varying complexity and visual diversity: *Lift* (pick), *Square* (pick and insert), *Can* (pick and place), and *Transport* (dual-arm pick, handover, place). Real-world experiments focus on *Toy* (pick and place). Most tasks use a second-person and an eye-in-hand camera, except *Transport* with four cameras, all at…

SLIDE 5

Where it breaks

The implementation of a saliency buffer improves efficiency, but the computational demands remain considerable. For example, saliency computation with a ResNet18 encoder on 84×84 inputs takes about 1.5 times longer than training. To improve efficiency, approximating saliency at mid-feature layers or exploring alternative saliency extractors could

SLIDE 6

One-line takeaway

Experiments show that RoboSaGA preserves in-domain performance while substantially improving robustness to visual domain shifts, including distractor and background changes, as well as lighting and shadow variations.

Assembled from the paper's own PDF, parsed with its layout intact so tables and equations survive, 67,231 characters of it, then split on the paper's own section headings. Extractive, not generated: every sentence here is lifted from the paper. Check it before you present it.

Figures worth putting on a slide

Figure 1: RoboSaGA's core components and the broadened visual domains. (a) Saliency map s, derived from visual feature output fθ(x), guides the overlaying of in-domain and the OOD
Figure 2: Experiment environment setups.
Figure 3: Augmentation with BC-MLP.
Figure 4: Examples of RoboSaGA against Real-World Visual Domain Shifts, including lighting changes, occlusion, object clutter, and background variations.
Figure 5: Saliency Maps across Two Views during different stages of task execution (BC-RNN).
Figure 6: Examples of out-of-domain images for data augmentation