Addressing AI system impartiality failures has grown in recent years. Unfortunately, current techniques for enhancing the fairness of AI systems concentrate on sweeping modifications to the norms, procedures, and algorithms companies use to implement these systems. 

Nevertheless, some organizations may require comprehensive methods to determine which user groups are disproportionately impacted by failures of specific system components. For example, failure mode and effects analysis (FMEA) is a popular safety engineering technique that helps organizations undertake "AI fairness impact assessments."

FMEA-AI

Researchers proposed "FMEA-AI," an extension to FMEA, as a modification to a familiar instrument for engineers and manufacturers that can incorporate moral sensitivity and ethical considerations into an organization's existing design process. 

FMEA-AI lets firms detect safety and fairness issues in numerous AI system failure modes. In contrast, current impact assessments concentrate on assisting regulators in identifying an aggregate risk level for the entire system. It explicitly identifies user groups and uses proportional claim satisfaction to calculate fairness-related failure likelihood and severity. This method can assist industry analysts in adapting a well-known safety engineering method to incorporate AI fairness concerns, promote moral sensitivity, and surmount resistance to change.

Features

The FMEA-AI work employs "impact assessment" rather than "risk assessment," which previous work characterized as an algorithm analyzing an individual's likelihood of defaulting on a loan, repeating a criminal activity, and so on. COMPAS, for example, is a risk assessment method used in US courts to determine the possibility of a human defendant committing a repeat offence. An impact assessment assesses the chances that an AI system will perform poorly, violate data privacy, be biased, etc.

Researchers assess fairness risk using failure mode probability. Still, formal verification and test methodologies might be applied with AI systems to establish allocations of products or likelihoods with high certainty.

Objective

An effective FMEA activity aids in identifying probable failure modes based on previous experience with similar products and processes—or on standard physics of failure logic. It is frequently employed in the development and production industries throughout the product life cycle. Examining the repercussions of those failures on various system levels is called effects analysis.

Both functional FMEA and piece-part (hardware) FMEA require functional assessments as input to discover proper failure modes at all system levels. An FMEA organizes risk reduction mitigation based on either reducing failure (mode) effect severity, lowering the chance of failure, or both. Although the FMEA is a thorough inductive (forward logic) study, the failure probability can only be estimated or lowered by knowing the failure process. As a result, FMEA may contain information on failure reasons (deductive analysis) to reduce the likelihood of occurrence by eliminating identified (root) causes.

Benefits

The following are the significant benefits of a properly implemented FMECA effort:

• It gives a defined technique for selecting a design with a high likelihood of success and safety.
• Early detection of single failure points (SFPS) and system interface issues that could jeopardize mission success and safety. They also give a way to ensure that switching between identical pieces is not undermined by single failures.
• It is a systematic strategy for assessing potential failure mechanisms, modes, and system functions.
• It is an efficient method for assessing the impact of proposed changes to mission design and operational procedures on mission success and safety.
• It is the foundation for in-flight troubleshooting processes, as well as the location of performance monitoring and fault-detection equipment.
• Criteria for early test planning.

Image source: Unsplash