human, all too human

SUMMARY

1.   Information Technology is not neutral. It disrupts society and its institutions, including government and the law. In a liberal democracy the job of law, courts, and lawyers, is to humanise technology and to uphold the rule of law. And yet IT “platform capitalists” and their adherents seek unfettered access to the institutions that might tame them. Our job as lawyers is to say “enough and no more”: we will use technology insofar as it serves human justice, but anything beyond that subverts or threatens to subvert the rule of law and liberal democracy. We must understand the technology we use (including IT) and be vigilant about its uses and abuses.

2.   As a caveat, and as will no doubt become apparent, I profess no technological expertise; on such matters I have read, digested, and (more heavily than usual) quoted from relevant experts in this article. My perspective is that of a court lawyer and trainer. The training perspective is essential: I have given critical thought not just to what lawyers do and why, but also how they work, what their work will look like in a few year’s time, and how their work will change. I have given particularly anxious thought to technology’s likely impact on access to both the profession and to justice. 

LAW AND TECHNOLOGY: A SLOW-MOTION REVOLUTION

Law in practice

3.   The law is not a closed system, operating behind locked doors, but is connected in hundreds of ways to our democratic culture. To disregard this structure of authority and replace it with a theory- whether philosophical, political, or economic in kind- is to erode our democracy at root.[1] 

4.   Rather, law is a human institution primarily concerned with human activities. Constitutional principles such as legality, accountability, transparency and other expressions of the checks and balances of the rule of law are core to liberal democracies. The rule of law implies that neither the legislature nor the executive get the last word on the meaning, interpretation or application of the law; judgement is reserved for the courts.[2]

5.   Society, especially in a liberal democracy, is messy. By way of illustration, the European Court of Human Rights neatly encapsulated the reasons underpinning Article 10 ECHR in Otto-Preminger Institute v Austria[3] para 49:

“freedom of expression constitutes one of the essential foundations of a democratic society, one of the basic conditions for its progress and for the development of everyone. Subject to paragraph 2 of Article 10 (art. 10-2), it is applicable not only to ‘information’ or ‘ideas’ that are favourably received or regarded as inoffensive or as a matter of indifference, but also to those that shock, offend or disturb the State or any sector of the population. Such are the demands of that pluralism, tolerance and broadmindedness without which there is no ‘democratic society’…”

6.   Code that! The Otto Preminger quote demonstrates that legal disputes may involve contested values, contested interpretations, as well as contested facts. Judges, faced with the task of deciding a dispute involving human activities, routinely have to produce reasonable and acceptable solutions in which the facts, the rules, and/or how they fit together, are controversial; the judicial function requires normative judgement and can only be legitimate when completed by a person who can subjectively understand what he or she is imposing on others when decisions are made.[4] As Sir Mark Hedley noted in the Family Court in London Borough of Southwark v A Family [2020] EWHC 3117:

“There is an element in human judgment that lies beyond cold rationality as every experienced trial judge soon comes to appreciate.”[5]

7.   Lawyers, analysing issues, giving advice, framing pleadings, leading evidence, and making argument deploy (amongst other skills and traits) logic, inference, probability, tacit and explicit knowledge, wit, psychology, common sense, experience, and their own character.

8.   For some tasks computers are good, better in some instances even than humans who have achieved mastery: Gary Kasparov was beaten by Deep Blue at chess, for example. As David Spiegelhalter reminds us, amazing machines such as Deep Blue

“are technological systems that use past data to answer immediate questions, rather than scientific systems that seek to understand how the world works: they are to judged solely on how well they carry out the limited task at hand, and, although the form the form of the learned algorithms may provide some insights, they are not expected to have imagination or have super human skills in every day life.”[6]

9.   Unless the “singularity” of artificial general intelligence is achieved, it seems likely that projects in computational law will treat traces of the legal process as the equivalent to the process itself. Thus, a computer may conduct a better analysis of an aspect of a legal problem than a human but is likely to do less well than a human lawyer overall. Analytical tools may be useful to lawyers for cross-checking, sense-checking, or discovery. Code-driven law presents even more difficult problems. Mireille Hildebrandt explains:

“What code-driven law does is to fold enactment, interpretation and application into one stroke, collapsing the distance between legislator, executive and court. It has to foresee all potential scenarios and develop sub-rules that hopefully cover all future interactions- it must be highly dynamic and adaptive to address and confront what cannot easily be foreseen by way of unambiguous rules. If it fails to do so, code-driven law must be subjected to appeal and contestation, based on, for instance, core legal concepts such as ‘unreasonableness’, ‘unacceptable consequences’, ‘good faith’, or the ‘circumstances and the context of the case at hand.’ This would imply reintegrating ambiguity, vagueness and multi-interpretability into the heart of the law.”[7]

 Human Agency

10. Human agency is the irreducible moral core of the rule of law. To that end, lawmakers ought to take steps to protect, preserve, and promote the natural ecosystem for human life. As a minimum, the physical wellbeing of humans must be secured: humans need oxygen, food and water, shelter, protection against contagious diseases, medical treatment if they are sick, and they need to be protected from attack. Further, the conditions for meaningful self-development and agency need to be constructed: there needs to be a sufficient sense of self and of self-esteem, as well as sufficient trust and confidence in one’s fellow agents, together with sufficient predictability to plan, so as to operate in a way that is interactive and purposeful rather than merely defensive. As Roger Brownsword puts it:

“With existence secured, and under the right conditions, human life becomes an opportunity for agents to be who they want to be, to have the projects that they want to have, to form the relationships that they want, to pursue the interests that they choose to have and so on. In the twenty-first century, no other view of human potential and aspiration is plausible.; in the twenty-first century, it is axiomatic that humans are prospective agents and that agents need to be free.”[8]

 Technology is not neutral: it has societal consequences.

11. Professor Carlota Perez describes a model of 5 industrial revolutions, which occur approximately every 50 years[9]. Broadly speaking, there have been 5 industrial revolutions:

1.   The “Industrial Revolution”, which started in Britain when Arkwright’s mill opened in Cromford in 1771

2.   The Age of Steam and Railways, which started in Britain with the test of the “Rocket” steam engine for the Liverpool-Manchester railway in 1829

3.   The Age of Steel, Electricity and Heavy Engineering, which started in the USA when the Carnegie Bessemer steel plant opened in Pittsburgh, Pennsylvania in 1875

4.   The Age of Oil, the Automobile, and Mass Production, which started in the USA when the first Model T came out of the Ford plant in Detroit, Michigan in 1908

5.   The Age of Information and Telecommunications, which started in the USA when the Intel microprocessor was announced in Santa Clara, California in 1971

6.   The Age of Platform Capitalists? 2010-2020ish?

12. Different phases within each industrial revolution change society. Within each revolution there is a “creative destruction”, a “great surge of development” that represents the turbulent process of diffusion of each technological revolution. As Perez notes,

“Each technological revolution is received as a shock and its diffusion encounters powerful resistance both in the established institutions and in people themselves. Hence, the full unfolding of its wealth-creating potential at first has rather chaotic and contradictory social effects, it later will demand a significant institutional recomposition. This will include changes in the regulatory framework affecting all markets and economic activities as well as the redesign of a range of institutions, from government, through financial regulation, to education, as well as modifications in social behaviours and ideas” [10]

13. Perez continues:

“Such changes tend to be forced by a mixture of pressures coming at first from the requirements of the rapidly changing economy and later from the consequences of the turbulent manner in which the new technology diffuses, leading to intense and sometimes violent social tensions. Ultimately, the most effective pressure for institutional change, and especially for some form of state intervention in the economy, comes from the recession following the collapse of the financial economy, which tends to occur a couple of decades after the initial big bang.

…

Each technological revolution, originally received as a bright new set of opportunities, is soon recognized as a threat to the established way of doing things in firms, institutions and society at large.

The new techno-economic paradigm gradually takes shape as a different ‘common sense’ for effective action in any area of endeavour. But while competitive forces, profit seeking and survival pressures help diffuse the changes in the economy, the wider social and institutional spheres where change is also needed are held back by strong inertia stemming from routine, ideology and vested interests. It is this difference in rhythm of change, between the techno-economic and socio-institutional speheres, that would explain the turbulent period following each big-bang and therefore the lag in taking full social advantage of the new potential.”[11] (emphasis added)

14. Perez describes what occurs during each technological revolution in four broad phases[12]:

1.   Irruption. There is a techno-economic split. Irruption of the technological revolution, the decline of old industries and unemployment

2.   Frenzy. Financial bubble time. Intensive investment in the revolution, a decoupling of the system. Polarisation of rich and poor. A gilded age.

3.   Synergy. The Golden Age. Coherent growth. Increase in production and employment.

4.   Maturity. Socio-political split. The last products and industries. Market saturation and technological maturity of main industries. Disappointment vs complacency.

15. It may be argued that IT has become the new “common sense” and that law will just have to cope. It is certainly true that some aspects of the Age of Information and Telecommunications have become common sense: it would be absurd to continue a paper-based system and judicious use of remote hearings seems reasonable and practical (although not yet familiar). Beyond that, as I argue below, we have either to tread carefully or not at all.

 Rule of Law or Rule of Technology?

16. It would have been unthinkable to hand over law-making and judicial functions to Cornelius Vanderbilt the railroad tycoon, to Andrew Carnegie, or to Henry Ford. History shows instead that from the Factories Acts to “railway spine” to the Human Embryology and Fertilisation Acts to copyright law (to name but a few), law has humanised technology; even Donoghue v Stevenson 1930 SC (HL) 31 might reasonably be seen as an instance of the law humanising mass production, preventing harm to (human) consumers. 

17. It ought to be equally unthinkable that we hand over law-making and judicial functions to the IT entrepreneurs and their adherents. Fairfield describes some of the ways that law has indeed regulated the activities of Facebook (election hacking and hate speech), Yahoo (stopping sales of Nazi memorability), and Google (the European right to be forgotten) but much, much more requires to be done to prevent the rule of technology.[13]

 Who benefits?

18. Referring to Lucius Cassius, the “most honest and most wise judge”, Cicero asked in in his defence of Roscius Amerino, “Cui bono?” (“who benefits?”). Information Technology, including machine learning, artificial intelligence, and databases such as blockchain, tend to be used for purposefully; in other words, they are not free-standing gimmicks or toys. Artificial intelligence and machine learning are used by “Big Tech” platforms such as Google, Facebook, Amazon, Apple, and Microsoft to make them some of the wealthiest organisations on the planet. Blockchain is the database underlying the Bitcoin cryptocurrency. There is a school of thought that:

“the ‘old’ state model is essentially a ‘vending machine’ where money goes in (tax) and public goods and services come out (roads, police, hospitals, schools). It is time [Tim O’Reilly] and others suggest ‘to rethink government with AI’ now that the technological change has ‘flattened’ the world, eroded state power, and provided models for uncoupling citizenship from territory. Only by seeing government as a ‘platform’ will it be possible to harness critical network externalities and ensure … ‘generativity’- the uncanny ability of open-ended platforms like Facebook or YouTUbe to create possibilities beyond those envisioned by their creators. For O’Reilly, Big Tech ‘succeeded by changing all the rules, not by playing within the existing system’. Governments around the world, he suggests, must now follow their lead.”[14]

19. As Frank Pasquale observes, there are at least two forms of pressure external to legal values that will make AI systems such as machine learning increasingly tempting to policymakers:

“First, the siren song of austerity amidst economic slowdowns will tempt policymakers to try to replace workers in the justice system with cheaper software. Second, high-margin technology businesses skilfully re-invest their profits to lobby governments to expand the use of their wares. In the large number of jurisdictions where such influence is common, we can foresee a predictable political economy of automation: the firms (and sectors) that are more concentrated, with a higher profit margin, will systematically bid legal work away from firms and sectors that are less profitable and less concentrated (and thus less able to coordinate long term investment in influence campaigns).”[15]

 Scottish courts

20. Scottish courts have, slowly, adapted to societal changes. Using the imperfect guide of legislation as an indicator of such adaptation, we can see very considerable changes to the civil and criminal courts in the middle years of the 19th century that might be explained by a society that was increasingly mobile due to railways. It may be no coincidence that changes to the courts coincided with radical changes to democracy and with societal discontent (to the point of revolution in continental Europe). 

21. While the courts no doubt evolved in the later 19th century and in the 20th century (as evidenced, perhaps, by the Sheriff Courts (Scotland) Acts of 1907 and 1971, the Court of Session Act 1988 and the Criminal Procedure (Scotland) Act 1995), much more radical reform has happened in the first two decades of the 21st century (see, for example, the Courts Reform (Scotland) Act 2014). Until recently, however, the courts would have been broadly recognisable to the Victorian time traveller; court processes resembled the “common sense” described by Perez. 

22. The measures taken in response to the COVID pandemic have introduced, or at least accelerated, unprecedented reforms, most notably involving electronic documents and remote video or telephone hearings. These have not yet achieved a “different common sense”. 

23. One consequence of the measures taken in response to the COVID pandemic has been the increased use of written argument either to replace or to focus remote, synchronous hearings. I sound a note of caution about over-reliance on written argument, invoking no less an authority than Socrates:

“It shows great folly… to suppose that one can transmit or acquire clear and certain knowledge of an art through the medium of writing, or that written words can do more than remind the reader of what he already knows on any given subject… The fact is, Phaedrus, that writing involves a similar disadvantage to painting. The productions of painting look like living beings, but if you ask questions they maintain a solemn silence. The same holds true of written words; you might suppose that they understand what they are saying, but if you ask them what they mean by anything they simply return the same answer over and over again. Besides, once a thing is committed to writing it circulates equally among those who understand the subject and those who have business with it; a writing cannot distinguish between suitable and unsuitable readers. And if it is ill-treated or unfairly abused it always needs its parent to come to its rescue; it is incapable of defending or helping itself.”[16]

The “parent” is, of course, the advocate who can defend, explain, correct, etc what has been written.

 THE SIREN SONG OF TECHNOLOGY

Justice systems across the world have been impacted by COVID; the response of the courts and the legal professions has, largely, been to introduce remote work, including video hearings. In a recent (now paywalled) article in The Times[17], Professor Richard Susskind concluded:

“In technological terms, video hearings are Stone Age stuff conceived in the 1980s. Their recent adoption offers a kick-start to the transformation of our courts. But the ad hoc Covid systems will be superseded this decade by asynchronous procedures, court-connected online dispute resolution, telepresence, virtual reality, blockchain, and artificial intelligence, terms as unfamiliar to most lawyers now as Zoom and Teams were 12 months ago.

If today’s video hearings are claimed to be the main answer to the future of courts, we are asking the wrong question. A sustainable, technology-enabled court service of the future will offer a blend of physical and online service — of which video is but one option — that is optimised for different types of cases, balancing the interests of court users with the practicalities of delivering a public service.

To get there we will need greater government investment and the support and vision of leaders across the legal community.”

24. Professor Susskind, who jokes that he has written the same book every 4 years for around 40 years[18] might be described as a techno-solutionist, who explicitly advocates “Outcome-Thinking”[19]. His views are in many ways consistent with LegalTech developers such as Ben Alarie who seek to usher in a new era of “smart” law built on AI and big data that will be more efficient and cheaper, and with superior decision-making capabilities than human lawyers and judges[20] and that LegalTech will improve access to justice, lower costs, and improve the efficiency of legal administration amongst other practical benefits.[21] Those who stand in the way of “progress” are damned as dinosaurs, luddites, unimaginative, self-interested, or all of the above.

25. The fundamental premise of the techno-solutionist’s position is that law and society ought to work like the natural sciences; because science can predict the regularity of the natural world, with enough computing-power, similar effects ought to be brought to bear on law and society[22]. As the first 23 paragraphs of this paper seek to illustrate, however, the analogy between the natural sciences and human society is misplaced; no amount of computing power will correct the false analogy; human justice cannot be replaced by technology and, if human justice is to be assisted by technology, great care and effort will be required to understand and constrain technology to ensure that it is ethical and properly serves the intended ends. Technology, including IT, is not neutral and cannot be assumed to be advantageous or even benign. The effect of IT on society and the quality of justice are not and cannot yet be known by Professor Susskind or anyone else.

26. The recent Court of Appeal decision in Hamilton and others v Post Office Ltd [2021] EWCA Crim 577[23] neatly demonstrates the risks of a lack of both accountability and transparency inherent in unquestioning reliance on computer systems. In Hamilton, Post Office Ltd (“POL”) used the Horizon accounting system. Horizon’s data contained serious discrepancies caused by bugs, errors, or defects in the system. Because audit data for Horizon was expensive and time-consuming to obtain and analyse, POL used unreliable management data that purported to show discrepancies. As a result, many sub post masters (“SPM”) were accused and convicted of fraud, theft and false accounting. As Holroyde LJ explains:

“First, we reiterate that POL deliberately chose not to comply with its obligations in circumstances in which its prosecution of SPM depended on the reliability of Horizon data. It did so against a background of asserting that SPMs were liable to make good all losses and could lose their employment if they did not do so. It did so despite the fact that POL itself had selected the SPMs as suitable persons to hold their position of trust. In the High Court proceedings, one of the agreed facts was that POL “incurs expense and time-costs in recruiting (including advertising for applicants and assessing and selecting applicants) and training new Subpostmasters”. Yet if Horizon showed a shortfall which an SPM did not accept as correct, POL invariably accepted the position shown by Horizon, refused to countenance any possibility that the apparent shortfall may be the result of an error or bug in the system, and was quick to assume dishonesty on the part of the SPM. As we have seen, in internal documents relating to at least some cases, an SPM who attributed a shortfall to a system error was dismissed as “jumping on the Horizon bandwagon”. These were very serious failures by POL to fulfil its obligations as a prosecutor. We are driven to the conclusion that throughout the period covered by these prosecutions POL’s approach to investigation and disclosure was influenced by what was in the interests of POL, rather than by what the law required.” (para 129)

27. Hamilton also explains the human toll of wrongful conviction and that justice is human:

“POL’s failings of investigation and disclosure (in Ms Busch’s phrase) “directly implicate the courts”. If the full picture had been disclosed, as it should have been, none of these prosecutions would have taken the course it did before the Crown Court. No judge would have been placed in the unhappy position of learning – as some judges (or retired judges) will do if they read this judgment – that they unwittingly sentenced a person who had been prevented by the prosecutor from having a fair trial.” (para 133)

28. Professor Richard Moorhead astutely asks, “Where were the lawyers?”[24] It is reasonable to suggest that more, well-trained lawyers, and less uncritical trust in technology, may well have avoided this gross miscarriage of justice.

29. The Faculty of Advocates of Scotland, The Bar Council of England and Wales, The Bar of Ireland, and the Bar Council of Northern Ireland issued a joint statement on 4th May 2021[25] that concluded:

“our unanimous stance is as follows:

1.   We are supportive of the continuing use of technology in our courts.

2.   We are supportive of remote hearings becoming the default position for short or uncontroversial procedural business. We recognise that the appropriate use of remote hearings will be vital in tackling accrued backlogs in each of our jurisdictions.

3.   However, for any hearing that is potentially dispositive of all or part of a case, the default position should be “in-person” hearings. Remote hearings should be available as an option in such cases where all parties (including the court) agree that proceeding in that way would be appropriate.”

30. While no doubt techno-solutionists would describe these statements as old-fashioned to the point of quaintness or, less generously, self-serving, these statements are expressly open to supporting the use of technology in our courts. What is implicit in these statements is that the technology should serve human justice.

 SO WHAT SHOULD WE DO?

31. Why should advocates at the Scottish Bar care about technology? Isn’t it enough that we are adapting to remote hearings? What has any of this got to do with training and education?

 TRAINING, STAYING ALERT, AND RESISTING

32. Right now, we have to adapt to remote hearings, where they are appropriate. In the longer term, we will have to be able to advise our clients on the impact of technology on the quality of justice they are likely to receive (cf Hamilton, and the Horizon/POL scandal). To that end, we will need to be neither cynical nor enamoured about Information Technology, but to remain curious and sceptical about IT. We will need to keep abreast of IT so we understand AI, singularity, blockchain, machine learning, etc sufficiently well to ask the for the right assistance from suitably qualified experts. We will need to understand who benefits from the technology at hand, to understand the ethical considerations arising out of IT, and to be prepared to take a moral stand against aspects of IT.

33. We should thus choose our battles: some technology, used wisely, helps us to prepare, to make decisions, or to attend remotely, for example. We require to train and learn how technology can best serve human justice by using technology proportionately, cost-effectively, and appropriately.

34. We should be wary about relying on exclusively or predominantly written argument. Written argument on complex, complicated matters is no substitute for rapport-driven, trustworthy, open, skilful oral argument that adds value by going beneath surface appearances and into the heart of the matter.

35. The above discussion ought to allow us better to understand the needs of our (human) clients and to shine a bright light on judicial decision making and the role of advocates in framing issues, in preparing and presenting cases, and on persuasion. 

AN ABRIDGED BIBLIOGRAPHY

1.   Online Courts and the Future of Justice, Susskind, Oxford University Press, 2019

2.   Is Law Computable? Edited by Deakin and Markou, Hart, 2020

3.   Runaway Technology, Can Law Keep Up, Fairfield, Cambridge University Press, 2021,

4.   AI Ethics, Mark Coeckelbergh, MIT, 2020

5.   An Artificial Revolution, On Power, Politics and AI, Bartoletti, The Indigo Press, 2020

6.   The Art of Statistics, Spiegelhalter, Pelican, 2019

7.   Technical Revolutions and Financial Capital, the Dynamics and Bubbles of Golden Ages, Perez, Edward Elgar Publishing, 2002

[1] J Boyd White, quoted by Frank Pasquale in his Foreword to Is Law Computable? Critical Perspectives on Law and Artificial Intelligence, Hart Publishing, 2020, ix

[2] Code-driven Law: Freezing the Future and Scaling the Past, Hildebrandt, in Is Law Computable?, p71

[3] (1995) 19 EHRR 34

[4] Frank Pasquale, Foreword to Is Law Computable?, vii

[5] Paragraph 188

[6] The Art of Statistics, Spiegelhalter, Pelican, 2019, p145

[7] Code-driven Law: Freezing the Future and Scaling the Past, in Is Law Computable?, p71

[8] Artificial Intelligence and Legal Singularity, in Is Law Computable?, p147

[9] Technological Revolutions and Financial Capital: The Dynamics of Bubbles and Golden Ages, Perez, 2002, pp4-5, with health warning at p49

[10] Perez, pp22-4

[11] Perez, pp25-6

[12] Perez, pp47et seq

[13] Runaway Technology, Can Law Keep Up? Fairfield, CUP, 2021, pp281-3

[14] From Rule of Law to Legal Singularity in Is Law Computable?, p3

[15] Is Law Computable?, pp v-vi

[16] Socrates, in Plato, Phaedrus, quoted in Tacit & Explicit Knowledge, Collins, The University of Chicago Press, p15

[17] https://www.thetimes.co.uk/article/video-hearings-have-transformed-courts-but-are-not-a-panacea-mcp77mjj7

[18] Online Courts and the Future of Justice, 2019, Acknowledgements, p ix

[19] Online Courts and the Future of Justice, Chapter 4

[20] From Rule of Law to Legal Singularity in Is Law Computable?, pp4-5. 

[21] From Rule of Law to Legal Singularity in Is Law Computable?, p16, which references Profesor Susskind’s work.

[22] For an overview, see Pasquale’s Foreword to Is Law Computable and the final chapter of Runaway Technology

[23] https://www.bailii.org/ew/cases/EWCA/Crim/2021/577.html

[24] https://lawyerwatch.wordpress.com/2021/04/23/the-post-office-where-were-the-lawyers-post/

[25] https://www.advocates.org.uk/news-and-responses/news/2021/may/statement-of-the-four-bars-on-the-administration-of-justice-post-pandemic