Adversarial robustness is not free: decrease in natural accuracy may be inevitable. Silver lining: robustness makes gradients semantically meaningful (+ leads to adv. examples w/ GAN-like trajectories) arxiv.org/abs/1805.12152 (Dimitris Tsipras Shibani Santurkar Logan Engstrom Alex Turner)

Aleksander Madry Dimitris Tsipras Shibani Santurkar Alex Turner Some bonus GIFs from paper - adversarial training induces GAN-like optimization trajectories for adversarial inputs. "Primate" to "bird" (1/2):

Aleksander Madry Dimitris Tsipras Shibani Santurkar Alex Turner "Primate" to "dog": (2/2)