The inboxes of European internet users have been filling with messages from various service providers during the past couple of months, as companies have updated their customer data protection to conform to the newly implemented GDPR (General Data Protection Regulation).
According to Cathy O’Neil, who gave her keynote at SHIFT Business Festival on Wednesday, improving privacy protection is not enough to solve the ethical issues concerning the collection of digital information about private individuals, serving at most to slow down the rapid and troubling current development.
A world-leading data activist, O’Neil has focused on big data driven algorithms in her research. Algorithms are everywhere, and according to O’Neil, they have come to stay. This is where the problems begin: These algorithms are essentially automated decision-making, and they are being used by, for instance, insurance companies, banks, schools and even courts of justice – institutions that can greatly affect a person’s life. The process is not open, and people do not have access to information about how decisions are being made about them or how and where data collected about them is being used.
The biggest problem is that algorithms can be unfair. Unmonitored, they reflect the discriminating and racist structures of society. This is disadvantageous to those who are already in an underprivileged position, such as women, people of low income and minority groups. In the worst case, the results could be disastrous. The title of O’Neil’s book, Weapons of Math Destruction, refers to this potentially massively destructive effect.
In her work, O’Neil focuses on the United States, but she sees the same problems at work everywhere in the world. It’s not about the data itself, but about how and why it is used – in other words, it’s about power.
We are no longer talking about harmless advertisement targeting, but rather about decisions that can have a profound impact on an individual’s life. With power, O’Neil refers to the fact that people don’t have the option to refuse answering certain questions when dealing with banks, insurance companies or the authorities, for instance. Seemingly innocent questions about age, gender and marital status alone give the underlying algorithm enough to start profiling, and the result is not always to the individual’s advantage.
The pervasiveness of algorithms is still a new phenomenon, and regulation concerning them is lacking. O’Neil wants to introduce ethical considerations to the discussion around algorithms. This is not a simple task, as terminology is ambiguous, and racism, for example, has not been defined so that everyone would understand it the same way. What’s more, not many people understand mathematical models enough to be able to discuss their ethical ramifications, especially when on the surface everything appears to be working just fine. In O’Neil’s view, the important question is not whether the algorithm works, but whether it is lawful and fair.
As a solution to the Wild West of the algorithm, O’Neil suggests that in addition to the business and the customer, a third, independent party should oversee the legality and fairness of the system. The challenge lies in that the authority would need to understand both equality and the technical aspects of algorithms, and at the moment it is still difficult to find experts versed in both.
The data activist has got down to business herself by starting a company that provides algorithm auditing and risk consulting in the United States. Her customer base at the moment consists of small and medium-sized businesses that want to be able to prove the appropriateness and lawfulness of their algorithms to their own customers. In the future, O’Neil hopes her customers will be those who absolutely do not want her services. O’Neil says Facebook would be her dream customer.
Head of Marketing & Communications
+358 45 8684 808
SHIFT Business Festival connects groundbreaking technology with leaders of traditional industries for a discussion about the ethics of future development. The festival, attracting 4000 great minds and visionaries, is held May 22nd - 23rd 2018 in the abandoned prison of Kakolan Lääni, Turku, Finland.
SHIFT is organised by a non-profit organization with the purpose of shaping breakthrough technology to be responsible and sustainable.
SHIFT is not an event. It is a beginning.
© Koodiviidakko Oy - Y-tunnus 1939962-1