Authors: Jan Simson, Florian Pfisterer, Christoph Kern

Published on: August 31, 2023

Impact Score: 8.12

Arxiv code: Arxiv:2308.16681

Summary

• What is new: Introduction of multiverse analysis for algorithmic fairness, turning implicit design and evaluation decisions into explicit ones to understand their impact on fairness.
• Why this is important: Biases in data can be reinforced or mitigated in the algorithmic decision-making process, but it’s often unclear how design and evaluation choices impact fairness.
• What the research proposes: A multiverse analysis method that makes design and evaluation decisions explicit, creates a grid of possible decision combinations (‘universes’), and computes fairness and performance metrics for each to investigate variability and robustness of fairness scores.
• Results: Demonstrated that evaluation decisions can significantly impact fairness metrics, showing the importance of transparent and robust evaluation processes to prevent the manipulation of fairness scores.

Technical Details

Technological frameworks used: Multiverse analysis

Models used: Predictive modeling for public health care coverage

Data used: Data on vulnerable populations for health care coverage prediction

Potential Impact

Healthcare providers, tech companies using ADM systems, policy makers, and fairness auditing services

Want to implement this idea in a business?

We have generated a startup concept here: FairVerse Analytics.