Yes, that’s right, some algorithms are biased; we have to work on improving the data we feed them

The story began on Twitter: David Heinemeier Hansson, DHH, a respected entrepreneur and creator of Ruby on Rails with more than 358,000 followers, shared a thread about how his wife, who had requested the same Apple Card as him, was given a credit limit 20 times lower than his, even though they filed joint tax returns and that, in addition, she had a better credit history than he did.

The somehow explicit tweet soon went viral, as people with similar experiences shared their stories and DHH said the company had ignored his appeals. In the meantime, he has triggered an investigation by the New York State Department of Financial Services for gender discrimination, while the card issuer, Goldman Sachs, has announced it would reassess how it calculates credit limits.

Coming from an experienced developer, DHH’s complaint, which blamed Apple exclusively for the problem, is curious: a machine learning system reflects the data we feed it, and even in developed countries (and more so in other parts of the world), there are still strong gender biases, particularly in financial environments. Obviously, this needs to be corrected, and in most of those developed countries this is what’s happening — we could argue about whether this is happening quickly enough — but that, I fear, does not address the issue of data already used to feed algorithms.

Biased machine learning algorithms do not necessarily indicate any kind of intentionality or discrimination on the part of the company that markets a product or service. Discrimination is the result of human intervention, and therefore must be corrected, typically as a result of consumers complaining. Believing that algorithms are biased by nature is like saying they’re infallible, when in reality, they are simple mathematical methods that process data. If the data is biased, those biases are reflected in the decisions algorithms make, and will remain so until the data is corrected or mathematically discounted by methods that correct the sample bias. This is a topic I have been discussing in my classes for years and is not so hard to understand or apply from a purely statistical standpoint.

Machine learning is one of the most interesting and promising technologies to benefit from much more sophisticated, powerful and flexible automation than simply programming variables and taking conditionals and loops into account. Applying machine learning means accessing much more intelligent, powerful and flexible rules, which are corrected as the sample data evolves, reflecting variations based on many factors, and which have the capacity to improve over time. There is even talk of using history as a huge database to feed algorithms, something that will require the participation of professional profiles of many different types, combining technical with humanistic disciplines.

Attributing algorithms of unreasonableness or accusing Apple of chauvinism is, at this stage of the game, as naive as assuming that they are possessed of some mystical power. Instead, machine learning will continue to generate biased algorithms due to the data we feed the algorithms, and those biases will have to be monitored and eventually corrected. Protesting makes sense, and companies must take action. Not all discrimination is the result of some kind of explicit intention or bias, sometimes they are implicit biases that are not necessarily easy to visualize and that are rooted in mathematics.

Machine learning will soon be an integral part of our lives and the Machine Learning as a Service (MLaaS) market will grow rapidly in the coming years. What’s more, companies that opt for automation will likely beat those that do not. When you use machine learning, assume that it is possible that biases that you have not taken into account may appear, so be ready to take the right response when they do. If you automate, be sensitive to the problems and biases that can arise from automation. We will have to learn to live with algorithms, and this will mean that both the companies that use them and the customers who experience them will know they aren’t perfect, and perhaps sometimes plain wrong. Let’s learn to recognize this without resorting to conspiracy theories and overreacting.

(En espa?ol, aquí)

?