A core feature of a major machine learning (ML) tool that enables data scientists to detect bias in their data set prior using it for model training, review detected bias that showed up during model training, and better understand their machine learning model by portraying the impact of features on a model.
What was the
Problem
Bias detection is incredibly important, but a lot of manual work.
In machine learning, machine learning models are trained using data sets until the results are satisfactory. This process poses three major challenges. The first is introducing bias into the training model either when uploading a data set. The second is when there is bias in the training algorithm. The third is when biased actions in the real world inform the data.*
If a model is only trained using data from brown furry dogs, it may be less accurate when making predictions. Feed it data involving dogs with short black hair or curly white fur, and it might struggle. It becomes critical to reduce bias that can lead to discrimination. It is also required by institutions regulating the ethical handling of personal data.
A Fortune 100 company was looking to increase the competitive position of their machine learning platform. They wanted to add compelling bias detection features to their machine learning platform. These included bias detection in a data set, bias detection during model training, and model explainability.
*For example, at a bank, a loan officer might hold prejudice against a people group. When the officer meets with people to approve or deny the loan, their bias may show up through key data points. Examples might be the applicant’s ethnicity or the area code they live in. These data points could negatively influence the model’s training. If the model is used in an automated program, it would the bias of the officer and disadvantage applicants.
The Bonus Problem: COVID
COVID-19 impacted the way we communicate and collaborate. It challenged our assumptions about productive ways of working and learning. The team overcame these challenges to deliver a critical and compelling feature in the ML Tool. It was presented by C-level stakeholders at their annual conference in 2020.
What was the
Solution
We created an ML Bias Detection tool for the ML Platform to help data scientists detect bias in their projects and explain AI model predictions. The goal of this project was to visualize the complexity of bias metrics and model explanations in the simplest ways possible. I created a simple, human-readable design that effectively communicated the bias metric values. For model explainability, the team leveraged the familiarity of feature importance charts expressed as SHAP values.
What was my
Process
Started from the bottom now we're here.
Hours Learning ML
Mockups Designed
Usability Studies
This project was an ambitious attempt to explore a new frontier in machine learning. Many of the team members (like myself) were new to machine learning. The team consisted of multiple groups, such as design, product management, and software engineering. Project teams or “squads” consisted of product managers, machine learning experts, developers, and a designer.
Designers integrated with the project team and owned the product through delivery. We used a design system, managed by one member, and regularly met to ensure our projects seamlessly integrated into the ML Platform. There was no official process, so designers were expected to plan, organize, and apply the creative process to get things done.
Most members on my project team had never worked with a designer. It empowered me to guide them through the creative process with abundant grace. I stuck as close as I could to the creative process, but often had to work backwards and let my design inform requirements. Requirements changed often, resulting in a lot of churn and refinement.
My Role
My responsibilities included creating a design plan, interviewing subject-matter-experts, designing and testing the experience, documenting the work, and collaborating with the frontend engineering team to deliver a complete experience. This began in April of 2020 and we needed to finish before the client’s annual conference in November.
I worked closely (remotely) with the PMs and engineers to design and iterate on concepts. I used my knowledge of best design practices and consulted the team to guide us toward creating a simple solution for a complex product. I handed off design to the development team in phases, touching base frequently since we were a fully remote team.
Explaining Bias
Metrics
The bias metrics needed to communicate the significance of the returned value without confusing data scientists or driving them to external sources for more information.
Visualizing Model
Explanations
The visualizations for feature attribution needed to indicate the how a feature impacted a model during training as well as indicate the change in rank from one period to the next.
Introducing the
Final Product
Launched on December 8, 2020
Detect imbalances in data
The ML Bias Detection feature was designed to integrate with the platform’s data exploration tool. The exploration tool enabled data scientists to detect bias during data preparation. I created a simple form that collects attributes of interest, and runs the ML Bias Detection tool to check for bias in the attributes. The tool then displays a report with visualizations. The visualizations use a “Meter” design component, displaying a measurement and descriptions of the sources of possible bias. These help data scientists quickly identify and resolve the issues. For example, if a dataset only contains a 3 examples of car models the tool may detect an imbalance. When predicting car sales, this helps avoid training a model that favors one type of car over another.
Detect bias in trained models
Similarly, data scientists can check for bias in their trained model through ML Experiments. After training they can generate a similar report to identify different types of bias for each attribute. It might suggest whether one ethnic group receives more positive predictions compared to another.
Understanding a model
As models form predictions in training, it’s possible that some inputs (called features) are weighed more than others.
The ML Bias Detection tool plots feature importance in ML Experiments. It generates a graph detailing the features that weighed more in the model’s prediction after training. Data scientists can use this information when evaluating compliance requirements. Another use is to determine if a feature has more influence than it should on the model.
Monitor models for changes in behavior
Over time, the behavior of a deployed model can be affected and cause it to change. For example, a decline in home prices could cause a model to weigh income less when making predications on loan approval. The Bias Detection tool integrates with the Model Monitoring tool to plot the change in feature importance compared to the tolerance set by the user.
Related
Projects
Check out another high impact project I did for this client.
I designed enhanced debugging features for a major ML tool to help data scientists improve model training.