Adversarial training improves discrimination, not alignment

When you train a deceptive model to stop its bad behavior, it could learn to:
1) Never do the bad behavior, or
2) Stop getting caught
Adversarial training makes it always learn to stop getting caught.
It never generalizes from adversarial examples to remove or even reduce the deceptive behavior.

Evan Hubinger - Michael Trazzi (Feb 12, 2024)
https://youtu.be/S7o2Rb37dV8?t=1119

Видео Adversarial training improves discrimination, not alignment канала Moloclips