DNN Attack Detection

Enabling attack detection for DNNs

This project, titled “Backdoor Detection via Eigenvalues, Hessians, Internal Behaviors and Robust Statistics” aimed to create effective tools to detect backdoor attacks on Deep Neural Networks (DNNs). It was completed in 2024.

DNNs, a type of machine learning algorithm, achieve impressive performance but can be challenging to train. As a result, it is common to outsource the training of a model (known as Machine Learning as a Service, MLaaS) or to use third-party pre-trained networks and then perform fine-tuning or transfer learning. However, these practices open security vulnerability because it is possible for an adversary (e.g., the MLaaS provider) to change the model architecture in a way that leaves a “back door” that can later be exploited. This can be done, for example, by polluting training data or directly changing the model weights, creating so-called Trojans within the model.

To address this issue, we designed mechanisms to understand the effects of Trojans on DNNs’ internal behaviors. For this work, we used ideas from robust statistics, scientific computing, and random matrix theory, providing a unique perspective to enable the development of effective tools for backdoor detection.

Project Team

About