Skittles: Jinni or Universe in a Pocket?

This was the big data economy, and it promised spectacular gains. A computer program could speed through thousands of resumés or loan applications in a second or two and sort them into neat lists, with the most promising candidates on top. This not only saved time but also was marketed as fair and objective. After all, it didn’t involve prejudiced humans digging through reams of paper, just machines processing cold numbers…The math- powered applications powering the data economy were based on choices made by fallible human beings.Some of these choices were no doubt made with the best intentions. Nevertheless, many of these models encoded human prejudice, misunderstanding, and bias into the software systems that increasingly managed our lives. Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domain. Their verdicts, even when wrong or harmful, were beyond dispute or appeal. And they tended to punish the poor and the oppressed in our society, while making the rich richer…Without feedback, however, a statistical engine can continue spinning out faulty and damaging analysis while never learning from its mistakes…They define their own reality and use it to justify their results. This type of model is self-perpetuating, highly destructive, and very common.

https://doi.org/10.35483/ACSA.Teach.2019.69

Volume Editors
Richard Blythe & Johan De Walsche

ISBN
978-1-944214-23-4