Available soon...
Adversarial camouflage has garnered attention for its ability to attack object detectors from any viewpoint by covering the entire object's surface. However, universality and robustness in existing methods often fall short as the transferability aspect is often overlooked, thus restricting their application only to a specific target with limited performance. To address these challenges, we present Adversarial Camouflage for Transferable and Intensive Vehicle Evasion (ACTIVE), a state-of-the-art physical camouflage attack framework designed to generate universal and robust adversarial camouflage capable of concealing any 3D vehicle from detectors. Our framework incorporates innovative techniques to enhance universality and robustness, including a refined texture rendering that enables common texture application to different vehicles without being constrained to a specific texture map, a novel stealth loss that renders the vehicle undetectable, and a smooth and camouflage loss to enhance the naturalness of the adversarial camouflage. Our extensive experiments on 15 different models show that ACTIVE consistently outperforms existing works on various public detectors, including the latest YOLOv7. Notably, our universality evaluations reveal promising transferability to other vehicle classes, tasks (segmentation models), and the real world, not just other vehicles.
| Photo-Realistic Simulation Demo | Real World Demo |
|---|---|
To generate universal and robust adversarial camouflage, we propose the ACTIVE framework, comprimising: (1) Triplanar Mapping with Neural Texture Renderer, (2) Stealth Loss, (3) Random Output Augmentation, and (4) Smooth and Camouflage Loss. The overall framework is illustrated in following figure:
We present the utilization of triplanar mapping, a texture mapping approach that applies textures to objects by projecting them from three directions based on their surface coordinates and normals, which can be derived from the depth image and camera parameter. This technique proves highly advantageous for generating object-independent adversarial textures, as it doesn't depend on object-specific texture maps. Consequently, we can optimize a single adversarial camouflage that works universally across multiple vehicles, rendering our attack instance-agnostic.
Figure: Triplanar Mapping pipeline and mechanism.
(a) Triplanar mapping pipeline.
(b) Triplanar mapping mechanism.
We present Neural Texture Renderer (NTR), the improvement of Differentiable Transformation Network (DTN) by our previous work in the term of efficiency. While it effectively synergizes with our triplanar mapping approach to apply physical transformations from the reference image, we identified redundancies within DTN, specifically in the Transformation Features (TF) and the training process. The improvement of NTR over DTN is illustrated as follows:
Redundancy in Transformation Features: DTN, employs four Transformation Features (TF): TF Subtractor + ReLU, TF Adder, TF Multiplier, and TF Final Adder, intended to encompass basic transformations. We theorize that the initial two layers are redundant, with the last two layers sufficient to encode TF. Thus, our final TF, mimic to weights and biases in Neural Network design, comprises only TF Multiplier and TF Final Adder. Our NTR architecture, illustrated in Fig. (a).
Redundancy in Dataset: DTN's training dataset encompasses 50 flat random color textures. We contend this approach is inefficient, especially when training the network for multiple objects. To foster generalization of colors, we propose training the network with eight boundary colors in RGB space and gray [128, 128, 128] as the reference color. These eight boundary colors include primary (red, green, blue), secondary (magenta, yellow, cyan), white, and black, as depicted in Fig. (b).
(a) Our Implemented NTR Architecture.
(b) Selected Boundary Colors in RGB Space
We propose stealth loss a novel attack loss function to make the object undetectable. The loss function is designed to minimize both the objectness score and the maximum confidence score across all classes of valid detection (greater than the IOU threshold), making our attack class-agnostic. This approach not only leads to misclassification but also considers the possibility of the bounding box being empty, providing a more comprehensive adversarial influence.
(a) Texture optimized by DTA's L_atk. The cars are missclassified.
(b) Texture optimized by Stealth Loss. The cars are undetected.
We introduce the ROA module to enhance texture robustness through diverse digital transformations (i.e., EOT). This module takes the output of NTR, which is the adversarial example, and augments it by applying random transformations like scaling, translation, brightness, and contrast. While NTR offers robustness against physical transformations, ROA adds a layer of robustness through digital transformations, simulating real-world changes to a certain extent. Notably, this consideration of digital transformations is often absent in conventional adversarial camouflage methods.
Ilustration of applying Random Output Augmentation (ROA) to the output of NTR. ROA applies random transformation to enhance the robustness of the adversarial camouflage by simulating real-world changes.
We employ a smooth loss (i.e., Total Variation (TV) loss) to enhance the texture's smoothness anda camouflage loss to achieve adversarial camouflage, encouraging texture-background similarity. The smooth loss enhances naturalness, while the camouflage loss facilitates seamless blending with thebackground, rendering the vehicle less imperceptible to human observers. Both losses enhance the natural appearance of the adversarial camouflage. Our study shows that omitting smooth loss outputs a rough texture, whereas excluding camouflage loss makes it more colorful and bright.
[Click the link to show sample demo videos]
[Click the link to show sample demo videos]
[Click the link to show sample demo videos]
[Click the link to show sample image]
@InProceedings{Suryanto_2023_ICCV,
author = {Suryanto, Naufal and Kim, Yongsu and Larasati, Harashta Tatimma and Kang, Hyoeun and
Le, Thi-Thu-Huong and Hong, Yoonyoung and Yang, Hunmin and Oh, Se-Yoon and Kim, Howon},
title = {ACTIVE: Towards Highly Transferable 3D Physical Camouflage for Universal and Robust
Vehicle Evasion},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {4305-4314}
}
ACTIVE: Towards Highly Transferable 3D Physical Camouflage for Universal and Robust Vehicle Evasion
Naufal Suryanto, Yongsu Kim, Harashta Tatimma Larasati, Hyoeun Kang, Thi-Thu-Huong Le, Yoonyoung Hong, Hunmin Yang, Se-Yoon Oh, Howon Kim