Three Questions for Crown’s Amanda Cook on WEAPONS OF MATH DESTRUCTION
In this “Three Questions for an Editor” Igloo feature, Amanda Cook, Vice President, Executive Editor, Crown, offers personal insights into her work with author Cathy O’Neil on her newly published book, WEAPONS OF MATH DESTRUCTION, subtitled “How Big Data Increases Inequality and Threatens Democracy.”
Ms. O’Neil examines how some of the decisions that affect our lives—where we go to school, whether or not we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated. But as Ms. O’Neil reveals in this urgent and timely book, the opposite is true.
Before we present Ms. Cook’s “Three Questions” responses, here is a brief look at her background: She began her publishing career at Addison-Wesley in 1996, and went on to work at Basic Books and Houghton Mifflin Harcourt before joining Penguin Random House. Amanda specializes in idea-driven and narrative nonfiction and has worked with a wide range of journalists and scholars, including Erik Larson, Matthew Desmond, Natalie Angier, Deborah Blum, Steven Strogatz, Roland Fryer, Jo Marchant, and Michael Specter.
How did Cathy O’Neil first come to your attention and what were your first impressions of the WEAPONS OF MATH DESTRUCTION proposal?
I first heard about Cathy from her agent, Jay Mandel, a few months before the proposal was ready. I started reading her blog and immediately fell in love with her fearless voice and her mission: to expose the secret and disastrously unfair mathematical models that increasingly control our lives. I also learned her great backstory. After receiving a Ph.D. in math from Harvard, Cathy taught at Barnard, then went to Wall Street, eager to put her math skills to use predicting movements in the market. I think she was the only female quant at D.E. Shaw. But when she realized her hedge fund was betting against people’s retirement funds, she became deeply disillusioned. Math—her first love—was being used in a way she felt was immoral. Cathy left Wall Street and became a financial advisor to the Occupy Movement, leading their Alternative Banking Group, and then took a job as a data scientist for a New York start-up.
When the proposal arrived, I was even more intrigued. As a culture, we’d already started talking about inequality, but no one was looking at Big Data as a cause. I think we all tend to see math as objective, pure. Cathy certainly did. But as she learned first-hand, algorithms that score teachers, sort resumes, grant (or deny) loans, evaluate workers, and set parole are not static descriptions of reality. They change reality, by expanding or limiting the opportunities people have. And these models are opaque, unregulated, and unaccountable. As a mathematician deeply concerned with social justice, Cathy was the perfect person to put all the pieces together.
That “Big Data” and “mathematical models” shape so many important life decisions may be alarming to many. As her editor, in what ways did you work with Cathy to develop and present the most essential elements of the book?
Working with mathematicians is always fun—they’re comfortable with abstraction, so they tend to be good structural thinkers (though this is the first time I ever received an initial outline in the form of a spreadsheet!). Mostly what I helped Cathy do was widen the range of examples in the book. These “weapons of math destruction” are everywhere—in schools, in police departments, in hospitals, in workplaces—and we wanted readers to have a sense of that. Unfortunately, there has been very little reporting on this topic, precisely because the models are hidden, so finding stories of real people damaged by algorithms was a challenge.
Cathy tells the story of one excellent teacher who was fired after receiving a bad score on something called the Value-Added Model, which Cathy shows is no better at evaluating teachers than rolling a pair of dice. There’s another story about a college kid who went to Vanderbilt, and had almost perfect SAT scores. He had bipolar disorder but was managing it well, yet he kept getting turned down for minimum-wage jobs. Through a friend, he discovered he was being screened out by a computerized personality test, one that is now being used in 60-70% of workplaces around the country. The stories are out there and we’re going to be hearing more of them.
Who do you see as the primary audience for WEAPONS OF MATH DESTRUCTION and what are some of the key takeaways that may have the most significant impact on readers?
I think that people who read Paul Krugman, watch Rachel Maddow, and support Elizabeth Warren will love this book. But what Cathy writes about affects us all. E-scores are a great example. These are like credit scores, but they determine your creditworthiness using alternative data—things like your shopping habits, your age, your gender, your social networks, your zip code. Computers can churn through this data so quickly that some call centers even use e-scores to decide how long to make you wait for a customer service representative. If you live in a certain part of the city, or the algorithm doesn’t like who your friends are, you could be on hold for much longer.
In the case of e-scores, it means that a person with a bad score (because they live in a poor neighborhood, say) is less likely to get a loan, less likely to get a job when their score is used for hiring, which keeps them in the poor neighborhood, which in turn worsens their score. Cathy calls this a “death spiral.” It’s basically a new, digital form of redlining. We all need to be aware of it so that we can protect ourselves and advocate for change.