Least privilege learning for attribute obfuscation

As machine learning systems become ever more prevalent in everyday life, the need to secure such systems is becoming a critically important area in cybersecurity research. In this work, we address the “feature misuse” attack vector, where the features output by a model are abused to perform a function that they were not originally designed for, such as determining a person’s gender in a facial verification system. To mitigate this, we take the security concept of “least privilege”, where a system can only access resources it explicitly needs to complete its task, and apply it to training deep neural networks. This “least privilege learning” ensures features do not contain information regarding protected attributes that are superfluous to the primary task, reducing the potential attack surface for feature misuse and reducing undesired information leakage. In this paper, we present two main contributions. Firstly, a novel training paradigm that enables least privilege learning by obfuscating protected attributes in verification and re-identification scenarios. Secondly, a comprehensive evaluation framework for models trained with least privilege learning, encompassing multiple datasets and three application settings: verification, re-identification, and attribute prediction.