MIT researchers have devised a way to make machine learning models less biased against minority subgroups.

It's not always easy for workers who use machine-learning models to make judgments to determine when to trust a model's predictions, especially when these models are so intricate no one knows how they function.

Fairness Criteria

Users sometimes use a method called "selective regression," in which the model figures out how sure it is about each prediction and rejects those for which it isn't sure enough. Then a person can look at each case, find out more information, and decide by hand. But while selective regression has to improve the overall performance of a model, researchers at MIT and the MIT-IBM Watson AI Lab have found that it can have the opposite effect on underrepresented groups of people in a dataset. This effect is because with selective regression, as the model's confidence increases, so does its chance of making the correct prediction, but this doesn't always happen for all subgroups.

For example, a model that predicts loan approvals might get it wrong less often on average, but it might get it wrong more often for Black or female applicants. The MIT researchers created two algorithms to address the issue after discovering it. They demonstrate that the algorithms lessen performance discrepancies that have impacted underrepresented minorities using real-world datasets.

Prediction or not, prediction

A method for estimating the connection between dependent and independent variables is regression. Regression analysis is in machine learning for prediction problems, such as estimating the price of a home given its attributes (number of bedrooms, square footage, etc.) With selective regression, the machine-learning model has two options for each input: it can predict anything or refrain from predicting something if it is unsure of its choice.

When the model abstains, the coverage—the portion of samples on which it bases predictions—decreases. The model's overall performance ought to increase by restricting its predictions to inputs, and it is pretty sure. However, this can potentially accentuate dataset biases when the model lacks sufficient data from particular subgroups. Underrepresented people may make mistakes or poor forecasts as a result of this. The MIT researchers wanted to ensure that selective regression improved each subgroup's performance, and so did the model's overall error rate. This danger is known as monotonic selective risk.

Focus on fairness

The researchers developed two neural network algorithms that use this fairness criterion to solve the problem.

• One algorithm ensures that the features the model uses to make predictions include all information about the sensitive attributes in the dataset, like race and sex, that are relevant to the target variable of interest. We can't use sensitive attributes to make decisions, usually because of laws or company rules. 
• The second algorithm uses a calibration method to ensure that the model makes the exact prediction for each input, even if that input has sensitive attributes.

Furthermore, researchers put these algorithms to the test on real-world datasets that we could use to make important decisions. For example, a crime dataset predicts the number of violent crimes in a community based on socioeconomic data. An insurance dataset is to predict the total annual medical costs charged to patients based on demographic data. Both sets of data have sensitive information about people. When they put their algorithms on top of a standard machine-learning method for selective regression, they got lower error rates for the minority subgroups in each dataset. This approach helped them reduce disparities. Also, researchers did this without significantly affecting the overall error rate.

Conclusion

Selective regression lets you not predict if you don't have enough confidence to make an accurate one. In general, if you give a regression model the option to reject, you can expect it to work better at the cost of less coverage (i.e., by predicting on fewer samples). In some circumstances, a minority subgroup's performance can decline as coverage does. "Selective regression" can amplify disparities between sensitive subgroups.

Because of these differences, the researchers have come up with new fairness criteria for selective regression that require the performance of every subgroup to get better as coverage goes down. Furthermore, the researchers show that the proposed fairness criteria meet sufficiency. 

Furthermore, the researchers propose two ways to reduce the difference in performance between subgroups: 

(a) by regularizing an upper bound of conditional mutual information under a Gaussian assumption and 

(b) by regularizing a contrastive loss for conditional mean and conditional variance prediction. 

These methods have worked well on both synthetic and real-world datasets.

Image source: Unsplash.