You're already living with AI systems, you just don't know it... and you almost surely can't protest how they work.

The AI is only as good as its training sets. 

Training sets do not understand subtlety. 

The social impacts of Artificial Intelligence are hard to see.

Sometimes AI techniques get it right, and sometimes they get it wrong. Only rarely will those errors be seen by the public: like the Vietnam war photograph, or when a AI ‘beauty contest’ held this month was called out for being racist for selecting white women as the winners. We can dismiss this latter case as a problem of training data — they simply need a more diverse selection of faces to train their algorithm with, and now that 600,000 people have sent in their selfies, they certainly have better means to do so. But while a beauty contest might seem like a bad joke, or just a really good trick to get people to give up their photos to build a large training data set, it points to a much bigger set of problems. AI and decision-support systems are reaching into everyday life: determining who will be on a predictive policing ‘heat list’, who will be hired or promoted, which students will be recruited to universities, or seeking to predict at birth who will become a criminal by the age of 18. So the stakes are high.

For example, the few studies that have been done into the use of AI and algorithmic decision-support systems in core social domains have produced troubling results. A recent RAND study showed that Chicago’s predictive policing ‘heat list’ — a list of people determined to be at high-risk of involvement with gun violence — was ineffective at predicting who will be involved in violent crime. However, it did lead to the increased harassment of those on the list. Similarly, a ProPublica exposé showed criminal risk-assessment software produced results that were biased against black defendants. To ensure people’s rights and liberties are upheld, we will need validation, auditing, and assessment of these systems to ensure basic fairness. Without it, we risk incorrect classifications, biased data, and faulty models amplifying injustice rather than redressing it.

Turing said in 1947 that if a machine is expected to be infallible, it cannot also be intelligent. What concerns us is that these fallible automated systems are being rapidly rolled out into the complex nervous system of society. These issues are front of mind for us, as we recently chaired AI Now, a White House symposium dedicated to the social and economic impacts of artificial intelligence in the next 10 years. AI Now also included an experts’ workshop, where leaders from academia, civil society, industry, and government discussed challenges in four thematic areas: social inequality, ethics, labor, and health — places where AI is already raising pressing questions.

The insights and diverse perspectives at AI Now were deeply informative, but they also revealed an uncomfortable truth: there are no agreed-upon methods to assess the human effects and longitudinal impacts of AI as it is applied across social systems. This knowledge gap is widening as the use of AI is proliferating, which heightens the risk of serious unintended consequences.

The core issue here isn’t that AI is worse than the existing human-led processes that serve to make predictions and assign rankings. Indeed, there’s much hope that AI can be used to provide more objective assessments than humans, reducing bias and leading to better outcomes. The key concern is that AI systems are being integrated into key social institutions, even though their accuracy, and their social and economic effects, have not been rigorously studied or validated.

There needs to be a strong research field that measures and assesses the social and economic effects of current AI systems, in order to strengthen AI’s positive impacts and mitigate its risks. By measuring the impacts of these technologies, we can strengthen the design and development of AI, assist public and private actors in ensuring their systems are reliable and accountable, and reduce the possibility of errors. By building an empirical understanding of how AI functions on the ground, we can establish evidence-led models for responsible and ethical deployment, and ensure the healthy growth of the AI field.