DEPARTMENT OF COMPUTATIONAL AND DATA SCIENCES
Ph.D. Thesis Defense
Speaker : Ms. Sravanti Addepalli
S.R. Number : 06-18-02-17-12-18-1-15587
Title : “Efficient and Effective Algorithms for Improving the Robustness of Deep Neural Networks”
Thesis examiner: Prof. Vineeth Balasubramanian, IIT Hyderabad
Research Supervisor: Prof. Venkatesh Babu R
Date & Time : August 12, 2024 (Monday) at 11:00 AM
Venue : # 102 CDS Seminar Hall
ABSTRACT
Deep neural networks (DNNs) have achieved remarkable success across various domains, yet their vulnerability to adversarial attacks and distribution shifts remains a significant challenge. This thesis presents novel methodologies to enhance DNN robustness, focusing on efficiency, effectiveness, and practical applicability.
The first part of the thesis concentrates on developing computationally efficient adversarial defenses. Traditional adversarial training methods are often computationally intensive due to the generation of adversarial examples through multiple optimization steps. To address this, we introduce Bit Plane Feature Consistency (BPFC), a regularizer that promotes robustness without requiring adversarial examples during training. Furthermore, we propose Guided Adversarial Training (GAT) and Nuclear Norm Adversarial Training (NuAT) to mitigate the gradient masking issue prevalent in single-step adversarial training, leading to improved robustness without sacrificing computational efficiency.
The second part focuses on improving the effectiveness of adversarial training. While adversarial training enhances robustness, it comes at the cost of reduced accuracy on clean data. To address this, we introduce Feature Level Stochastic Smoothing (FLSS), a method that combines adversarial training with detection to boost robustness and accuracy. Additionally, we propose Oracle-Aligned Adversarial Training (OAAT) to address the robustness-accuracy trade-off at large perturbation bounds. To further enhance adversarial training, we explore the integration of data augmentation techniques through Diverse Augmentation based Joint Adversarial Training (DAJAT).
The third part of the thesis focuses on improving the efficiency and effectiveness of self-supervised training for robust representation learning. We investigate the potential of combining the popular instance-discrimination task with auxiliary tasks such as rotation prediction to reduce noise in the training objective and improve the quality of learned representations. We further utilize these self-supervised pretrained models in a teacher-student distillation setting for training adversarially robust models without labels using the proposed method Projected Feature Adversarial Training (ProFeAT).
The final part of the thesis addresses the brittleness of DNNs to distribution shifts. We propose the Feature Replication Hypothesis (FRH) to explain the underlying causes of vulnerability to distribution shifts. To mitigate this, we introduce the Feature Reconstruction Regularizer (FRR) that encourages the learning of diverse feature representations. Additionally, Diversify-Aggregate-Repeat Training (DART) is proposed to improve generalization of DNNs by training diverse models in parallel, and aggregating their weights intermittently over training. We finally propose Vision-Language to Vision – Align, Distill, Predict (VL2V-ADiP), a teacher-student setting to utilize the superior generalization of Vision-Language Models (VLMs) for improving the OOD generalization in vision tasks.
Through these contributions, this thesis advances the state-of-the-art in DNN robustness by providing practical and effective solutions to address the challenges posed by adversarial attacks and distribution shifts. The proposed methods demonstrate significant improvements in both robustness and accuracy, paving the way for more reliable and resilient models.
ALL ARE WELCOME
