FERI: A Multitask-based Fairness Achieving Algorithm with Applications to Fair Organ Transplantation

Abstract

1. Introduction

Deep neural networks have revolutionized several areas within the medical field, including medical image processing¹, patient risk prediction², and the interpretation of clinical notes³. These advancements have enhanced personalized medicine and the development of effective treatments for various diseases⁴. The efficacy of deep learning models can be partly attributed to data-driven representation learning, which enables automatic extraction of useful features directly from data⁵. However, real-world data often encodes historical prejudices and demographic inequalities⁶. Therefore, reliance on potentially biased data risks amplification of unfairness in trained models. The issue of fairness in healthcare, in particular, has been a long-standing topic of discussion in various literature^7,8. The essence of fairness in healthcare requires addressing health disparities and reducing inequalities in both access to care and health resource allocation⁹. It requires implementing policy interventions targeted at underrepresented populations to bridge gaps in care. In recent years, attention has come to the fairness issues in healthcare prediction models¹⁰. Prediction models often emphasize accuracy enhancement without explicitly addressing issues of unfairness, potentially leading to inequitable outcomes.

Unfairness can originate from biases introduced in different stages of the machine learning pipeline, from data collection to model training. The source of bias often determines the approach used to achieve fairness. Based on where bias is addressed in the deep learning pipeline, fairness algorithms can be classified as pre-processing, in-processing, or post-processing methods¹¹. Pre-processing algorithms focus on addressing biases before modeling, i.e., by processing the input data to mitigate biases that can potentially cause unfairness in the prediction outcomes¹². Post-processing methods adjust model outputs after training to align predictions with fairness criteria. Examples include calibrated equalized odds for recalibrating predictions¹³ and a fairness-aware ensemble framework to shift decision boundaries¹⁴.

The goal of in-processing methods is to achieve model fairness during model training, and the critical approach is to seek strategies to balance model accuracy and model fairness^15,16. It can usually be done in various ways, for example, conventional methods using constraint optimization to add the fairness metric as a regularization term in the optimization target function⁶. The fairness can be modeled by the model performance disparity among different subgroups. Therefore, it can be easily expressed as the loss difference among subgroups^17,18. Several research studies have focused on directly modeling the fairness metrics as regularization terms¹⁹; however, the main difficulty is that most fairness metrics are “rate-based” expressions, which jeopardize the differentiability of the loss function²⁰. Typical methods to alleviate this is to use differentiable functions as surrogate functions²¹. Furthermore, alternative methodologies have been proposed. Agarwal et al. used game theory to convert constrained optimization into a gameplay problem with two players competing two-players²²; one optimizes the utility, and the other optimizes fairness. Kumar et al. treated the threshold as a learnable and differentiable parameter, making the rate-based fairness metrics a continuous variable²³. In recent years, other novel in-processing methods have been proposed beyond constrained optimization to achieve model fairness, such as adversarial learning²⁴ and implicit dynamic reweighting²⁵.

This paper uses a multitask learning framework to explore in-processing techniques for achieving group fairness in machine learning models. Achieving fairness in machine learning often requires careful calibration of model performance across different subgroups, especially those defined by sensitive attributes such as gender, age, race, and ethnicity. This goal of balanced performance aligns with the multitask learning framework, where a model can concurrently learn multiple tasks²⁶. The rationale for this analogy is on the concerns of the training speed disparity problem essentially²⁷. Techniques from multitask learning, including dynamic weighting scheme and specialized regularization, aim to equalize various learning speeds inherently²⁸. Meanwhile, applying such methods to fairness could also help address the disparity of the training speed between subgroups.

In this work, we introduce a novel algorithm, Fairness through Equitable Rate of Improvement (FERI), inspired by the Fast Adaptive Multitask Optimization (FAMO) to equalize subgroup training speeds²⁹. Our key insight in FERI is that the synchronized optimization framework of FAMO can be adapted to address unfairness in machine learning. We propose constraints on each subgroup loss to equalize the rate of decrease and smooth the learning trajectory, preventing any one subgroup from dominating the learning process. We build on multitask learning concepts to address fairness challenges in this novel way. Our technique promotes fairness by balancing subgroup optimization rather than directly regulating model parameters. Through experiments, we validate that this FAMO-inspired technique aligns with and even improves upon current methods by improving equity and mitigating bias.

Liver transplantation is a critical medical procedure that exhibits systemic inequalities related to gender, age group, and racial and ethnic backgrounds³⁰. For instance, Black patients are less likely to receive a liver transplant compared to white patients, highlighting racial disparities in access³¹. Similarly, there has been an increasing disparity in transplant rates between females and males³². The MELD score uses creatinine levels as one of the standards for liver transplant allocation, which are generally lower in women, leading to longer wait times and decreased access to liver transplantation for women³³. Age-related disparities are also evident, as pediatric candidates face challenges due to the scarcity of suitably sized organs. While previous papers concentrate on the multiple post-liver transplantation risk factors predictions^34,35 there is also work by Ding et al. that predicts graft failure in a distinct approach to address fairness challenges in liver transplantation³⁶. This paper focuses on these fairness challenges in liver transplantation, aiming to develop a more equitable and accurate model for predicting graft failure within a multitask learning framework. FERI aims to address the ethical complexities in real-world applications such as transplant allocation and post-transplant success, building on existing research while introducing novel fairness-oriented methodologies.

To summarize, the following are the key highlights of our research:

• Targeted Fairness: This study focuses on existing systemic biases affecting liver transplantation, emphasizing gender, age group, and race/ethnicity disparities.
• Methodological Innovation: Within a multitask learning framework, we propose a FERI algorithm to balance the subgroup training process through constraints on equitable loss improvements.
• Clinical Application: The paper aims to enhance equity in liver transplantation by creating a more accurate and fairer model for predicting graft failure, which is crucial for post-transplant outcomes.

2. Data Structure

We utilized a large national dataset of 160,360 liver transplant patients between 1987 and 2018, obtained from the Organ Procurement and Transplantation Network (OPTN) maintained by the United Network for Organ Sharing (UNOS)³⁷. This rich dataset includes comprehensive profiles of liver transplant recipients and donors, waiting list registrations, lab results, operative details, and longitudinal outcome data. A key variable we modeled is graft failure, a major determinant of post-transplant prognosis³⁸.

A common transplantation allocation framework is the Model For End-Stage Liver Disease (MELD) score, which estimates the 90-day mortality rate of patients without transplantation using lab tests for serum creatinine, bilirubin, international normalized ratio blood test, and serum sodium³⁹. In contrast to MELD, our study integrates a broader range of 117 features, including 52 recipient variables, such as blood type and BMI, and 65 donor variables. This expanded feature set enabled more detailed predictions of graft failure risk.

The demographic information for liver transplant patients is detailed in Table 1, which differentiates between those with and without graft failure. It covers gender, age, race, and ethnicity, highlighting potential disparities. For instance, while gender subgroups were relatively evenly distributed, disparities existed between adults (42.83% of patients) and pediatrics (33.99%) for graft failure. Similarly, differences across racial and ethnic groups exist. This suggests that age, race, and ethnicity contribute to unequal graft failure rates, which we aim to address. Additionally, the table underscores representational imbalances across each subgroup in sensitive attributes.

3. Methodology

3.3 FERI algorithm: Fairness through Equitable Rate of Improvement in Multitask Learning

We incorporated multitask learning into the Tab Transformer framework to enhance performance and fairness further. Multitask learning is particularly effective for our objectives as it often outperforms single-task models, especially when the tasks are correlated²⁶. In our implementation, we utilize a common approach known as hard parameter sharing, which allows the model to share hidden layers across all tasks while maintaining task-specific MLP output layers to produce predictions for each task⁴³. Each task in the multitask learning setup is carefully explained based on sensitive demographic attributes, such as gender, age group, and race/ethnicity. This design choice enables us to focus on optimizing the model’s performance for each subpopulation as a specific task, aiming to mitigate biases and ensure fairness in prediction across different demographics.

Inspired by foundational principles from existing research, we developed our own FERI algorithm that adjusts task losses to balance loss distribution in multitasking scenarios while extending to mitigate algorithmic biases in model predictions. We adapted the idea of dynamically fine-tuning the subgroup learning rate and efficiently managing training speed. Eventually, FERI adjusts the weights of each subgroup during training to ensure demographic fairness. Rather than solely focusing on balancing the learning rate per task, FERI also aims to robustly mitigate potential biases against sensitive attributes across various subgroups.

At first, we initialize various hyperparameters and attributes such as the number of tasks M, the learning rate α and β which control the step size in each update during training, a regularization term γ, and maximum gradient norm. We also consider a stability constant ε ∈ ℝ⁺(= 1 × 10^–8) to prevent division by zero. Notably, the logits for each task, denoted as w_t,m, starts at zero. As a result, the softmax-transformed weights z_t start by assigning equal importance to every task. The algorithm also starts with the model parameter θ_t with binary cross-entropy loss function L_m(θ_t).

Before any training begins, the combined task loss for the entire dataset is given by:

LossFERI(X,Y)=∑m=1MZt,m×Lm(θt)

(2)

here, Zt,m=ewt,m/∑m=1Mewt,m is the weight corresponding to task m at epoch t, which is determined by taking the m^th entry of the softmax transformation applied to the entire vector W_t = [w_t,1, w_t,2, … , w_t,M] for all tasks. For each training epoch t, the following steps are undertaken to adjust the loss for the next epoch t + 1:

a. Weight calculation

For each task m, the softmax function is applied to the logits of the current epoch to obtain z_t,m from the m^th entry as Softmax(W_t). This value represents the weight for task m at epoch t. Consequently, the vector z_t = [z_t,1, z_t,2, … , z_t,M] aggregates the weights for all M tasks at epoch t. To get a better understanding of the model’s current performance for each task, an adjusted loss is computed for every task as ∆_t,m= L_m(θ_t) + ε.

b. Model Parameters Update using Weighted Gradient Descent

We update the model parameters based on the current epoch weights as:

qt+1=qt−α(∇logΔt)ztrt

(3)

where ∆_t = (∆_t,1, ∆_t,2, … , ∆_t,M)^T. The rt=∑m=1mzt,m/Δt,m is used to re-normalize the task weighting back into the probability simplex by dividing a constant r_t per epoch.

c. Task Weight Logits Update using Weighted Gradient Descent

Using the updated model parameters q_t+1, we recompute the task-specific losses using Δ_t+1,m = L_m(q_t+1) + ε and collate them in the vector ∆_t+1 = (Δ_t+1,1, Δ_t+1,2, … , Δ_t+1,M)^T . This is crucial as it provides the necessary information to adjust the importance weights or logits of the tasks. Simultaneously, the logits for each task, which guide the weight determination, are also updated. We utilized the following update rule for logits:

wt+1=wt−β(δt+γwt)

(4)

where δt=∇Tzt,1(wt)⋮∇Tzt,M(wt)T(logΔt−logΔt+1).

The term log∆_t − log∆_t+1 denotes the change in the logarithm of the loss function after the parameter update. This difference indicates how the recent parameter adjustments impacted the overall model performance across tasks. Intuitively, a larger loss change on a task (faster training speed) will induce a smaller weight on this task in the next epoch to promote all tasks being trained at similar speeds. We adaptively adjust task weights by incorporating this into the logits’ update.

Initiating with the framework from Equation 2 to 4, the FERI algorithm iteratively adjusts its parameter for each epoch t. The weights z_t,m and parameters θ_t are updated to reflect the current epoch. The described algorithm FERI offers a dynamic task weighting mechanism in which each task’s importance is continuously updated, and the model parameters are fine-tuned accordingly in each epoch. As training progresses, these logits (and consequently, the weights) will be adjusted to potentially give more importance to tasks that are harder or less importance to tasks that are easier, depending on both individual task losses and the combined task loss from the prior epoch. This adaptive approach ensures the model shifts its emphasis on tasks based on their recent performance, aiming to optimize the combined task loss in subsequent epochs.

We summarized Algorithm 1 which dynamically adjusts the model’s weights and parameters, allowing for ongoing optimization and more accurate future forecasts. FERI reduces bias in liver transplantation predictions by dynamically recalibrating weights assigned to different demographic subgroups during training, ensuring no subgroup is favored due to the large sample size. This leads to more equitable outcomes across gender, age group, and race/ethnicity in liver transplant patients.

4. Model Evaluation

We assessed the FERI algorithm based on two primary objectives. First, we evaluated its predictive accuracy for liver transplant graft failure across various demographic subgroups, defined by sensitive attributes such as gender, age group, and race/ethnicity. We measured this using the Area Under the Curve of the Receiver Operating Characteristic (AUROC) and the Area Under the Precision-Recall Curve (AUPRC) metrics, comparing the performance of the FERI algorithm against a baseline model for each demographic subgroup. Secondly, we evaluated the model’s fairness using demographic parity and equalized odds as fairness metrics. For both training and validation, a five-fold cross-validation approach was employed. Loss plots were analyzed to understand how the convergence rate varies between the FERI and baseline models within each subgroup. Ultimately, the aim is to establish the FERI algorithm’s proficiency in balancing both predictive accuracy and fairness when forecasting liver transplant graft failure. For clarity, the fairness metrics used were defined as follows: Y^: the prediction on graft failure made by the model; Y: the true label or actual outcome on graft failure; S: sensitive attributes.

Demographic Parity:

Equalized Odds:

5. Experiments

In the following experiments, we monitor training loss rates across demographic subgroups for both the FERI and baseline models to minimize disparities during training. FERI’s efficacy in addressing fairness for liver transplantation was evaluated by adjusting subgroup losses and comparing performance to baseline models on demographic parity and equalized odds. FERI’s accuracy against the baseline was assessed using AUROC and AUPRC for subgroups within sensitive attributes to determine if FERI can improve fairness without sacrificing evaluation accuracy for predicting liver transplant graft failures.

5.1 Loss Progression Analysis: FERI vs. Baseline Across Subgroups

We comprehensively assessed the FERI algorithm’s behavior during training, evaluating its ability to adapt and be unbiased across diverse demographic groups. We compared the FERI algorithm to baseline models to evaluate its improvements in adaptability and fairness.

To monitor the trajectory of training loss across epochs, we observed changes in losses from one epoch to the next, specifically focusing on the demographic subgroups of gender (Male, Female), age group (Adult, Pediatric), and race/ethnicity (White, Black, Hispanic, Asian). Furthermore, hyperparameter tuning played a crucial role in this experiment. We varied the regularization coefficient (γ) from 1e-9 to 1e-2 and adjusted the learning rate for task logits α between 0.05 and 0.15. Using the optimized hyperparameters (γ is 1e-6 for both Gender and Age, and 1e-7 for Race/Ethnicity, along with α is 0.11 for Gender, 0.10 for Age, and 0.08 for Race/Ethnicity), Figure 1 provides a clear illustration of the effectiveness of the FERI algorithm. The algorithm achieved a uniform loss-reduction trajectory across various demographic subgroups. For example, both the male and female subgroups reached a loss value of 0.5 by approximately the 300-epoch benchmark. This consistency emphasizes the model’s capability to equitably enhance performance across subgroups. The FERI algorithm’s loss curves were curlier than the baseline because FERI updates each epoch’s losses based on the previous epoch’s outcomes. This step-by-step adjustment process allows FERI to incrementally refine the losses through continuous monitoring.

Open in a separate window

Figure 1.

Training Losses Over Epochs for Different Sensitive Attributes. The figure represents a comparative analysis of the baseline and FERI’s loss progression across epochs, categorized by settings: Gender, Age Group, and Race/Ethnicity. X-axis: Epochs, Y-axis: Loss Value.

5.2 Comparative Fairness Analysis: FERI vs. Baseline

We evaluated the effectiveness of our proposed FERI algorithm in addressing fairness issues against baseline models through an ablation study. We performed a five-fold cross-validation method to split the data into 60% for training, 20% for validation, and 20% for testing.

In Table 2, it shows the comparison across sensitive attributes of gender, age group, and race/ethnicity. This emphasizes the FERI algorithm’s superior performance in reducing disparities across sensitive attributes versus baselines. For example, when considering demographic parity with respect to gender, the FERI algorithm showed an obvious improvement. The baseline model had a demographic parity score of 0.0046 with a standard deviation of ±0.0037. In contrast, the FERI algorithm scored 0.0013 with a standard deviation of ±0.0017. The reduction was derived by taking the difference between the baseline model value and the FERI algorithm value, and then dividing the result by the baseline model value. This represents a reduction in disparity of approximately 71.74%. Similarly, in the age group category, the equalized odds metric saw a decline from the baseline model’s score of 0.0833 (with a standard deviation of ±0.0128) to the FERI algorithm’s score of 0.0496 (with a standard deviation of ±0.0211). This shows an approximate reduction of 40.46% in the disparity. The FERI algorithm also displayed improvements for race and ethnicity, underscoring its efficacy in promoting fairness across all sensitive attributes. In conclusion, the FERI algorithm consistently outperformed the baseline, indicating reduced unfairness across all examined attributes.

Table 2.

Disparity Analysis: Demographic Parity and Equalized Odds Comparisons between Baseline and FERI. Each entry provides fairness metrics value from a five-fold cross-validation. FERI algorithm performance on demographic parity and equalized odds are boldfaced and highlighted in green for each fold. A smaller value in our fairness metrics indicates reduced unfairness.

Open in a separate window

Open in a separate window

5.3 Accuracy Sacrifice Evaluation: FERI vs. Baseline

For a robust evaluation, we adopted a five-fold cross-validation approach for evaluating the accuracy using both AUROC and AUPRC metrics. The values presented in Figure 2 and Figure 3 were averages from these five folds for each subgroup within the specified sensitive attributes. We presented a detailed analysis that compares the predictive performance of our FERI algorithm against the baseline model. Remarkably, the FERI algorithm exhibited AUROC and AUPRC scores that closely align with those of the baseline model across all analyzed subgroups. The notation |∆| in the figures represents the absolute difference between the FERI and baseline values, highlighting a minimal trade-off. Specifically, the AUROC values for both models consistently exceeded the 0.7 threshold and AUPRC values are above 0.6. They provided robust evidence that our FERI algorithm succeeded in improving fairness metrics without sacrificing predictive accuracy. Meanwhile, it also underlined the model’s capabilities in ensuring an equitable prediction of graft failure outcomes in liver transplantation across various demographic subgroups.

Open in a separate window

Figure 2.

Evaluation of Accuracy Trade-offs in Baseline and FERI Using AUROC Metrics for Subgroup

Open in a separate window

Figure 3.

Evaluation of Accuracy Trade-offs in Baseline and FERI Using AUPRC Metrics for Subgroups

Through all these three experiments, FERI consistently outperformed the baseline models, demonstrating its effectiveness in balancing both predictive accuracy and fairness in predicting liver transplant graft failure across all sensitive attributes.

6. Discussion

Our multitask-based techniques equalize and optimize the training rates across all subgroups in different demographic populations, thereby improving fairness by mitigating disparate training speeds. However, it’s crucial to note that FERI is sensitive to hyperparameters such as learning rate and weight decay. Fine-tuning is often required for FERI to achieve optimal performance on specific datasets. Future work will extend beyond algorithmic validation, emphasizing real-world settings and focusing on diverse patient populations. We aim to develop an adaptive mechanism to counteract biases arising from imbalanced subgroup representation within the dataset. The goal is to optimize hyperparameters for generalizability and develop techniques to improve robustness to skewed subgroup distributions. Moving forward, our focus will include non-technical factors such as the varying prevalence of diseases among demographic groups to enhance our algorithm’s applicability across diverse clinical settings. We will refine our approach to better account for the above factors in future algorithm improvement. Overall, developing accurate and unbiased predictive tools is essential to reduce healthcare disparities and ensure clinically meaningful predictions for all patients. This work provides a valuable framework for future efforts to integrate both accuracy and fairness objectives.

7. Conclusion

In conclusion, the FERI algorithm provides an innovative approach to fair predictive modeling in healthcare. By utilizing multitask learning and dynamically adjusting loss function weights, FERI achieved comparable overall accuracy to baseline models based on AUROC and AUPRC metrics. Critically, it also demonstrated substantially improved fairness by equalizing improvement rates across demographic subgroups.

8. Acknowledgment

References