In an attempt to address the concerns about bias in AI results, researchers at IIT Jodhpur created a framework for scoring datasets on the ‘fairness, privacy, and regulatory’ scale for using algorithms in the Indian context. 

AI experts have consistently voiced concerns about the use of Western datasets in developing AI systems. These datasets tend to induce a bias in results, possibly rendering the system ineffective for the Indian context. 

The recommendations in the study “On Responsible Machine Learning Datasets with Fairness, Privacy, and Regulatory Norms” included collecting data from a diverse population with sensitive aspects such as gender and race, provided in a manner that protects the privacy of individuals. The framework, which also assesses if an individual’s personal data is protected, could aid in creating “responsible datasets” and is an attempt towards mitigating ethical issues of AI, the researchers said. 

The framework, developed with international collaborators, outlines criteria that assess a dataset’s “responsibility” -- fairness, privacy and regulatory compliance. The framework operates as an algorithm that produces an ‘FPR’ score. 

Fairness measures whether a dataset answers questions such as “Are different groups of people represented?” Privacy is assessed by identifying vulnerabilities that could potentially lead to a leak of private information. Regulatory compliance looks at institutional approvals and an individual’s consent to data collection. 

The researchers ran their auditing algorithm over 60 datasets from around the world, including widely used ones, and found that all highlighted “a universal susceptibility to fairness, privacy and regulatory compliance issues”. 

52 of the 60 datasets were face-based biometrics, with eight being chest X-ray-based healthcare. 

The team found that about 90 per cent of the face datasets were neither ‘fair’ nor ‘compliant’—scoring two or less out of a maximum of five on ‘fairness’ and zero or one out of three on ‘regulatory compliance’. 

“The framework would facilitate effective dataset examination, ensuring alignment with responsible AI principles,” the authors said in the study published in the journal “Nature Machine Intelligence”. 

Furthermore, under ‘regulatory compliance’, the framework analyzes whether a dataset respects an individual’s ‘Right to be Forgotten’, which states that if a person withdraws consent, their personal data must be quickly removed from the dataset. 

The IIT Jodhpur researchers also recommended enhancing data collection processes and addressing ethical and technical concerns while developing and managing databases. 

Source: Nature