Naughty ML
Like all powerful technologies, machine learning can be a wonderful, awesome tool or the most direct path to an ethical quagmire. Google’s People + AI team underscores the importance of balancing what is technically possible with how it should be applied keeping issues of fairness, accessibility and the broader customer empowerment in mind. Like a board game of snakes and ladders, there are many opportunities to slip up from biased data to an over reliance on implicit and disempowering flows.
I teach (learn) Emerging Technology to Interaction Design seniors at CCA. This fall the students and I kicked things off with AutoML on GCS. Our goal in the class is to learn by doing and making so exploring the ethics of AI was no different. We divided the class up into four groups to build vision models for answering some very naughty questions — can we predict from a person’s photo alone if they are rich or poor; smart or stupid; achievers or losers; US residents or foreigners. “Wait a minute”, you say, “that sounds seriously dubious!” and yet there are real products claiming efficacy for exactly these kind of services.
First a little bit about the process…While still in beta, we decided to use AutoML from Google because the web interface makes it more accessible for designers while still preserving some of the key aspects of evaluating ML models such as confidence threshold, error classification and confusion matrix. Each of the four groups started by collecting image datasets for their classification. Even as designers trained to focus on empathy, it is surprisingly easy to fall into stereotypes when that is their objective function.
This was followed by training, evaluating, tweaking and re-training — your basic iterative cycle of building real world models. The students quickly developed a surprising level of intuition that led to high levels of “prediction accuracy”. This led to some interesting conversations about how these very cut and dry objective metrics can create a false sense of confidence in what is clearly a flawed enterprise. “But we have numbers to back it up…”
Finally the students presented their models and insights to the rest of the class. One of the groups was bold enough to run the prediction on themselves, live! In a real time demonstration of bias, one of the girls was labeled a “loser” until she replaced her fashionable, transparent eyeglasses with a more classic black frame. That was all it took to flip her to an “achiever”. The group working on US residents versus foreigners were careful to explicitly remove ethnic bias based on skin or hair color, focusing on fashion as a key predictor. However, in a surprising twist they found that their model preferred its US residents to be more serious and excessive smiling would re-classify the same person as a foreigner — go figure :)
The use of ML in products is sure to accelerate in the coming years given how transformative it can be to experiences. Being naughty with ML really reinforced for us, as a class, the need for designers to get involved in the building of customer oriented ML models from the very beginning to avoid the many ethical questions that are bound to come up.
If you are interested in exploring this topic further here are some good resources to get started — some first steps in accessing fairness in ML, types of bias, and delayed impact of data on machine learning.
A very special thanks to all of the students for sharing their work: Sam Anderson, Bibiana Bauer, Adam Brumley, Julian Crespo, Skylar Dann, Krystel Delos Reyes, Cegan Dodge, Avtandil Grigolia, Yang Hong, Joanne Lee, Grace Park, Sparsh Sharma, Kailen Swain. An amazing group that I would highly recommend if you are looking for a technically savvy and passionate designer. Also, a thank you to Google’s education initiative for giving us GCS credit to play with the different ML solutions. Jim Fleming from Fomoro for sharing some speaking to the class and sharing some interesting links on ML bias.
