publications

2026

1. Conference

   Why LoRA Fails to Forget: Regularized Low-Rank Adaptation Against Backdoors in Language Models

   Hoang-Chau Luong, and Lingwei Chen

   Association for Computational Linguistics, 2026

   Abs Bib

   

   Low-Rank Adaptation (LoRA) is widely used for parameter-efficient fine-tuning of large language models, but it is notably ineffective at removing backdoor behaviors from poisoned pretrained models when fine-tuning on clean dataset. Contrary to the common belief that this weakness is caused primarily by low rank, we show that LoRA’s vulnerability is fundamentally spectral. Our analysis identifies two key factors: LoRA updates (i) possess insufficient spectral strength, with singular values far below those of pretrained weights, and (ii) exhibit unfavorable spectral alignment, weakly matching clean-task directions while retaining overlap with trigger-sensitive subspaces. We further establish a critical scaling threshold beyond which LoRA can theoretically suppress trigger-induced activations, and we show empirically that standard LoRA rarely reaches this regime. We introduce Regularized Low-Rank Adaptation (RoRA), which improves forgetting by increasing spectral strength and correcting alignment through clean-strengthened regularization, trigger-insensitive constraints, and post-training spectral rescaling. Experiments across multiple NLP benchmarks and attack settings show that RoRA substantially reduces attack success rates while maintaining clean accuracy.

   @article{luong2026lora,
     title = {Why LoRA Fails to Forget: Regularized Low-Rank Adaptation Against Backdoors in Language Models},
     author = {Luong, Hoang-Chau and Chen, Lingwei},
     journal = {Association for Computational Linguistics},
     year = {2026},
     url = {https://arxiv.org/pdf/2601.06305},
     dimensions = {true},
   }

2. Conference

   Understanding SAM’s Robustness to Noisy Labels through Gradient Down-weighting

   Hoang-Chau Luong, Thuc Nguyen-Quang, Dat Ba Tran, and 1 more author

   International Conference on Artificial Intelligence and Statistics, 2026

   Abs Bib

   

   Sharpness-Aware Minimization (SAM) was introduced to improve generalization by seeking flat minima, yet it also exhibits robustness to label noise, a phenomenon that remains only partially understood. Prior work has mainly attributed this effect to SAM’s tendency to prolong the learning of clean samples. In this work, we provide a complementary explanation by analyzing SAM at the element-wise level. We show that when noisy gradients dominate a parameter direction, their influence is reduced by the stronger amplification of clean gradients. This slows the memorization of noisy labels while sustaining clean learning, offering a more complete account of SAM’s robustness. Building on this insight, we propose SANER (Sharpness-Aware Noise-Explicit Reweighting), a simple variant of SAM that explicitly magnifies this down-weighting effect. Experiments on benchmark image classification tasks with noisy labels demonstrate that SANER significantly mitigates noisy-label memorization and improves generalization over both SAM and SGD. Moreover, since SANER is designed from the mechanism of SAM, it can also be seamlessly integrated into SAM-like variants, further boosting their robustness.

   @article{luong2026understandingsamsrobustnessnoisy,
     title = {Understanding SAM's Robustness to Noisy Labels through Gradient Down-weighting},
     author = {Luong, Hoang-Chau and Nguyen-Quang, Thuc and Tran, Dat Ba and Tran, Minh-Triet},
     journal = {International Conference on Artificial Intelligence and Statistics},
     year = {2026},
     url = {https://arxiv.org/pdf/2411.17132},
     dimensions = {true},
   }

2024

1. Conference

   Fundamental Convergence Analysis of Sharpness-Aware Minimization

   Pham Duy Khanh, Hoang-Chau Luong, Boris S Mordukhovich, and 1 more author

   Advances in Neural Information Processing Systems, 2024

   Abs Bib HTML

   

   The paper investigates the fundamental convergence properties of Sharpness-Aware Minimization (SAM), a recently proposed gradient-based optimization method (Foret et al., 2021) that significantly improves the generalization of deep neural networks. The convergence properties including the stationarity of accumulation points, the convergence of the sequence of gradients to the origin, the sequence of function values to the optimal value, and the sequence of iterates to the optimal solution are established for the method. The universality of the provided convergence analysis based on inexact gradient descent frameworks (Khanh et al., 2023b) allows its extensions to the normalized versions of SAM such as VaSSO (Li & Giannakis, 2023), RSAM (Liu et al., 2022), and to the unnormalized versions of SAM such as USAM (Andriushchenko & Flammarion, 2022). Numerical experiments are conducted on classification tasks using deep learning models to confirm the practical aspects of our analysis.

   @article{klmtneurips,
     title = {Fundamental Convergence Analysis of Sharpness-Aware Minimization},
     author = {Khanh, Pham Duy and Luong, Hoang-Chau and Mordukhovich, Boris S and Tran, Dat Ba},
     journal = {Advances in Neural Information Processing Systems},
     year = {2024},
     url = {https://arxiv.org/abs/2401.08060},
     dimensions = {true},
   }