What we need to talk about when we talk about AI (for regulatory purposes)

Nowadays, Artificial Intelligence (AI) is ubiquitous. We can hardly open a newspaper or tune in to a news show without getting some story about AI. AI is probably the technology most talked about. But AI means different things to different people.

I’ve been working on the field of AI, both in industry as in academia since the late 80’s. Developed my first AI system in 86, an expert system to determine eligibility to social housing. Since then I’ve witnessed the deeps and the ups, the winters and the hypes in the field. Never before there has been this level of excitement, and fear, by so many, in so many areas, as we are seeing in the last couple of years. AI is breaking through in many different application domains, with results that impress even the most knowledgeable experts. Three main factors are leading this development: the increasing availability of large amounts of data, improved algorithms and substantial computational power. However, of these three only algorithms can be rightfully seen as a contribution from the AI field.

More recently, the awareness that ‘AI’ has the potential to impact our lives and our world has no other technology has done before, is rightfully raising many questions concerning its ethical, legal, societal and economical effects. Government, enterprises and social organisations alike are coming forward with proposals and declarations of their commitment to an accountable, responsible, transparent approach to AI, where human values and ethical principles are leading. This is a much needed development, one to which I’ve dedicated my efforts and research in the last few years. But, responsible development and use of AI begins with a proper AI narrative, one that demystifies the possibilities and the processes of AI technologies, and that enables all to participate in the discussion on the role of AI in society.

The current hype is concerning. All of a sudden every digital system is AI, AI is used everywhere, and all problems raising from the increasing use of digital technologies, are AI risks and concerns. AI is seem either as some magic ‘thing’ that no one understands how it works but that takes decisions about us, for us and instead of us, as an all knowing, all powerful ‘entity’ that will soon take over the world (for the good or for the worse depends on who is describing it), or as ‘business as usual’, just the next step in digitisation. The need to guide the use and development of such ‘thing’ is increasingly debated at all levels, and the ‘AI ethics’ field is fast growing, with individuals and organisations falling over each other in the rush to publish yet another recommendations, principles or guidelines.

My question is: what are we regulating? What is different between ‘AI’ and any other computer system that causes all this efforts, discussions and commissions? Why are we concerned with the outcomes of an ‘AI’ decision, but much less those made by, for example, people or by throwing a dice? Is it the fact that ‘AI’ systems use large quantities of data? Many other applications do too. Because ‘AI’ systems are not transparent? Which human organisation is fully transparent? Or because ‘AI’ systems are very often developed, controlled and owned by large, private, corporations outside the oversight of democratic practices? Or is it something else that I am failing to see?

What is ‘AI’? What are we concerned about? What do we want to regulate? In the following, I briefly describe some of ways the concept of AI has been used, and conclude with a reflection of the significance to the current efforts towards AI regulation.

AI is not the Algorithm

The “algorithm” is achieving magical proportions, used right and left to signify many things, de facto embodying, or seen as a synonym to, the whole of AI. AI has been around for give or take some 80 years, but algorithms are much older than that[1]. AI uses algorithms, but then so does any other computer program or engineering process. Algorithms are far from magic. In fact, the easiest definition of algorithm is that of a recipe, a set of precise rules to achieve a certain result. Every time, you add two numbers, you are using an algorithm, as well as when you are baking an apple pie. And, by itself, a recipe has never turned into an apple pie. The end result of your pie has more to do with your baking skills and choice of ingredients. The same applies to AI algorithms: for a large part the result depends on its input data, and the ability of those that trained it. And, as we have the choice to use organic apples to make our pie, in AI we also have the choice to use data that respects and ensures fairness, privacy, transparency and all other values we hold dear. This is what Responsible AI is about, and includes demanding the same requirements from the ones that develop the systems that affect us.

AI is not Machine Learning

Machine Learning, and in particular, Neural Networks, or Deep Learning, is a subset of AI techniques that uses statistical methods to enable computers to perceive some characteristics of their environment. Current techniques are particularly efficient in perceiving images, written or spoken text, as well as the many applications of structured data. By analysing many thousands of examples (typically a few million), the system is able to identity commonalities in these examples, which then enable it to interpret data that it has never seen before, which is often referred to as prediction. Even though, the results of current machine learning algorithms are, also here the process is far from magic, but is the result of applying well-known mathematical and statistical methods at very large scale. Moreover, perception is just one component of intelligence. AI applications that are able to identify patterns in data are usually very far from being able to understand the meaning of those patterns.

An attempt to define AI

AI includes Machine Learning and is based on algorithms. However, as a discipline the ultimate goal of AI is to develop computer systems that are able to simulate human-like intelligence. Besides machine learning, AI includes knowledge representation, planning, dealing with uncertainty, theorem proving, cognitive robotics and human-agent/robot interaction just to mention a few on the fields. The term Artificial Intelligence was coined in the 50’s by John McCarthy, who defined it as the endeavour to develop a machine that could reason like a human, was capable of abstract thought, problem solving and self-improvement. The challenge proved much harder than what those original scientists expected, and even current success of AI, in the area of Machine Learning, are very far from realising those objectives. More than the ability of perception, AI as a field of science is about reasoning, is about meaning.

Borrowing from the definition given in the seminal textbook on AI[2], I would say that AI is the discipline of developing computer systems that are able of perceiving its environment, and to reason about how to best act on it in order to achieve its goals, assuming that the environment contains other agents similar to itself. As such, AI is about the autonomy (or better automation) to decide on how to act, the adaptability to learn from the changes affected in the environment, and the interactivity required to be sensitive to the actions and aims of other agents in that environment, and decide when to cooperate or to compete.

A responsible, ethical, approach to AI goes further than the technology used, and needs to include the social, organisational and institutional context of that technology. It is this socio-technical ecosystem that needs to ensure transparency about how adaptation is done, responsibility for the level of automation on which the system is able to reason, and accountability for the results and the principles that guide its interactions with others, most importantly with people. In addition, and above all, a responsible approach to AI makes clear that AI systems are artefacts manufactured by people, for some purpose, and that people are responsible for the use and development of AI.

AI regulation: what are we regulating?

AI regulation is a hot topic, with many proposers and opponents. The European Commission has presented a concrete proposal in April 2021. The ‘definition’ above may be useful from the perspective of designing AI systems, but more is needed when the regulation of AI is concerned.

Firstly, it is important to decide what are the characteristics, or properties of a system, that are relevant to be regulated. By focusing on technologies, or methods, i.e. by regulating systems that are based on ‘machine learning , logic, or statistical approaches’, which are given as the AI definition used in the European Commission’s proposal, we run the risk of seeing organisations evading the regulation, simply by classifying their applications differently. Conversely, there are plethora of applications based on e.g. statistics that are not ‘AI’. Focusing on the techniques will quicker lead to some catch-22 situation than to meaningful regulation. It cannot be the purpose of regulatory efforts, to lead to the next ‘AI winter’ by creating a situation in which all will be ‘doing AI’ under other names.

A risk-based approach to regulation, as proposed by the European Commission, is definitely the direction to take, but needs to be informed by a clear understanding of what is the source of those risks. The design of any artefact is in itself an accumulation of choices and choices. These are biased by nature as they involve selecting some option over another. We should not merely focus on technical solutions at the level of the algorithms or the datasets, but rather develop socio-technical processes, and the corporate responsibility, to ensure that any discriminatory or unfair outcomes are avoided and mitigated.

At the same time, AI systems are computer applications, i.e. are artefacts, and as such subject to existing constraints, legislation, for which due diligence obligations and liabilities apply. As my colleague, Catelijne Muller always says “AI does not operated in a lawless world”. Before defining extra regulations, we need to start by understanding what is already covered by existing legislation.

In order to be future-proof, regulation should focus on the outcomes of systems, whether or not these systems fall in the current understanding of what is ‘AI’. If someone is wrongly identified, is denied human rights or access to resources, or is conditioned to believe or act in a certain way, it cannot does not matter which technology or method is used. It is simply wrong. But, it is not sufficient to consider only the outcomes of the system. The inputs, processes and conditions under which AI is developed and used are at least as important. Much has been said about the dangers of biased data, and discriminating applications. Attention for the societal, environmental and climate costs of AI systems is increasing. All these must be included in any effort to ensure the responsible development and use of AI.

Regulating these systems is needed. Independently of whether we call the system ‘AI’ or not.

[1] The word algorithm derives from al-?wārizmī ‘the man of ?wārizm’ (now Khiva), the name given to the 9th-century mathematician Abū Ja‘far Muhammad ibn Mūsa, author of widely translated works on algebra and arithmetic. (source Wikipedia)

[2] Russell and Norvig (2009): Artificial Intelligence: A Modern Approach, 3rd edition. Pearson Education.