Information Bottleneck Methods for Fairness and Privacy in Machine Learning
Loading...
Authors
Gronowski, Adam
Date
Type
thesis
Language
eng
Keyword
machine learning , artificial intelligence , deep learning , machine learning fairness , information bottleneck , Rényi divergence
Alternative Title
Abstract
Designing machine learning algorithms that are accurate yet fair, not discriminating based on any sensitive attribute, and also private, not revealing users’ personal information, has become of paramount importance for society to accept the widespread use of artificial intelligence (AI) for critical applications. In this thesis, we investigate the use of variational Information Bottleneck (IB) methods for fair and private machine learning.
We present a novel fair representation learning method termed Rényi Fair Information Bottleneck (RFIB) which incorporates constraints for utility, fairness, and compactness of representation. In contrast to prior work, we consider both demographic parity and equalized odds as fairness constraints, allowing for a more nuanced satisfaction of both criteria. We use the Rényi divergence in developing a loss function involving classical IB measures and show that its parameter α provides an extra degree of freedom that results in performance benefits. Applying the method to image classification, we study the influence of the α parameter and two other tunable IB parameters on achieving utility/fairness trade-off goals, and evaluate it using various utility, fairness, and compound utility/fairness metrics on three different image datasets (EyePACS, CelebA, and FairFace), showing that RFIB outperforms current state-of-the-art approaches.
Furthermore, we investigate how the problem of privacy relates to the problem of fairness and present a related method to jointly improve fairness and privacy termed Rényi Fair and Private Information Bottleneck (RFPIB). Using the Rényi divergence and IB measures, we develop a loss function designed to improve both privacy and multiple fairness metrics while also ensuring utility. Experimenting on the CelebA and EyePACS datasets, we study trade-offs between fairness, privacy, and utility. We significantly outperform a baseline ResNet50 network and show that tuning the Rényi divergence’s α parameter can be used to simultaneously achieve these three desired criteria.
Description
Citation
Publisher
License
Queen's University's Thesis/Dissertation Non-Exclusive License for Deposit to QSpace and Library and Archives Canada
ProQuest PhD and Master's Theses International Dissemination Agreement
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.
Attribution-NonCommercial 3.0 United States
ProQuest PhD and Master's Theses International Dissemination Agreement
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.
Attribution-NonCommercial 3.0 United States