QL-PGD: An efficient defense against membership inference attack

In membership inference attacks (MIAs), attackers exploit the overfitting phenomenon when training deep learning models to identify whether a specific data sample was used to train a victim model. Consequently, many defenses have been introduced to mitigate the risk of being attacked by MIAs. However, state-of-the-art defenses against MIAs often suffer from poor privacy-utility balance and high training or inference computational time. To overcome these limitations, we propose Quantized Layer-wise Perturbed Gradient Descent (QL-PGD), a novel, lightweight and effective generalization method to protect machine learning models from Membership Inference Attacks. The key idea of QL-PGD is to regularize the model to prevent overfitting by adjusting the injected noise added to the gradient at each layer which explicitly regularizes the gradient passing through it to achieve a stronger privacy defense while maintaining similar levels of accuracy. Moreover, quantization is further performed on both model weight and gradient to reduce computational overhead. Extensive experiments are conducted to evaluate the performance of our method compared to other state-of-the-art generalization defenses against multiple attacks. The results show that QL-PGD can withstand both black-box and white-box attacks and preserve the target model's utility with efficiency in terms of speed, memory and energy.