January 16, 2015

Easily Fooling Deep Neural Networks

My guest this week is Anh Nguyen, a PhD student at the University of Wyoming working in the Evolving AI lab. The episode discusses the paper Deep Neural Networks are Easily Fooled [pdf] by Anh Nguyen, Jason Yosinski, and Jeff Clune. It describes a process for creating images that a trained deep neural network will mis-classify. If you have a deep neural network that has been trained to recognize certain types of objects in images, these "fooling" images can be constructed in a way which the network will mis-classify them. To a human observer, these fooling images often have no resemblance whatsoever to the assigned label. Previous work had shown that some images which appear to be unrecognizable white noise images to us can fool a deep neural network. This paper extends the result showing abstract images of shapes and colors, many of which have form (just not the one the network thinks) can also trick the network. The core of the confusion, as Anh shares, seems to be in the discriminative nature of these networks. Their objective is to find any features which allow them to objectively distinguish between their available training image. This creates the opportunity for a type of over-fitting (or perhaps one could argue in some cases it's under fitting) yielding this situation.

We discuss the paper and it's implications including how this might effect security system uses of neural networks or even self driving cars.

I highly recommend checking out their original paper (pdf link above) for examples of the images. There is also a great youtube video Deep Neural Networks are Easily Fooled that explains the phenomenon in a clear and visual way.

For his benevolent recommendation, Anh suggests listeners might find the Caffe image recognition software package. And please check out work by Anh and his collaborators at the Evolving Artificial Intelligence Laboratory.

Enjoy this post? Sign up for our mailing list and don't miss any updates.

Have a word to say? Propose a specific change to the blog post.