CSA Research

One of the most interesting aspects of writing about AI and LLMs right now is that if I say anything remotely positive, some people will accuse me of being a shill for Big AI. If I say anything remotely negative, others will accuse me of being insufficiently aware of the progress AI has made. So I will put out a few personal statements about AI that might clarify where I am on this: 1. AI is not intelligent, at least not in the human sense of the word. It is a sophisticated tool for drawing inference from binary data and thus operates *below* a symbolic level. 2. AI, at least in the guise of LLMs, is not going to achieve artificial general intelligence (AGI) now or in the future. 3. AI is getting much better at *approximating* human behavior on a wide variety of tasks. It can be extremely useful without being intelligent, in the same way that an encyclopedia can be very useful without being intelligent. 4. For some tasks – such as translating between two languages – LLMs sometimes perform better than some humans perform. They do not outperform the best humans. This poses a significant challenge for human workers that we (collectively) have yet to address: Lower-skilled workers and trainees in particular begin to look replaceable, but we aren’t yet grappling with what happens when we replace them so they never become the experts we need for the high end. I think the decimation of the pipeline for some sectors is a HUGE unaddressed problem. 5. “Human parity” is a rather pointless metric for evaluating AI. It far exceeds human parity in some areas – such as throughput, speed, cost, and availability – while it falls far short in other areas. A much more interesting question is “where do humans and machines have comparative advantage and how can we combine the two in ways that elevate the human?” 6. Human-in-the-loop (HitL) is a terrible model. Having humans – usually underpaid and overworked – acting in a janitorial role to clean up AI messes is a bad use of their skill and knowledge. That’s why we prefer augmentation models, what we call “human at the core,” where humans maintain control. To see why one is better, imagine if you applied an HitL model to airline piloting, and the human only stepped in when the plane was in trouble (or even after it crashed). Instead, with airline piloting, we have the pilot in charge and assisted by automation to remain safe. 7. AI is going to get better than it is now, but improvements in the core technology are slowing down and will increasingly be incremental. However, experience with prompting and integrating data will continue to drive improvements based on humans’ ability to “trick” the systems into doing the right things. 8. Much of the value from LLMs for the language sector will come from “translation adjacent” tasks – summarization, correcting formality, adjusting reading levels, checking terminology, discovering information, etc. – tasks that are typically not paid well.

Whenever I post about “Human at the Core” (versus “Human in the Loop” or HitL) as a model for human-AI interaction, I get a certain amount of pushback from some people who argue that it is, in essence, putting lipstick on a pig. They argue that this approach still cedes too much control and autonomy to machines and the companies that control them. It would be extremely unwise to dismiss thoughtful criticism like this, and I have to admit that these critics have a point. Although the label matters, just rebranding the same old approaches as human at the core (or augmented translation) would be a disservice. So the question is if the two approaches are actually the same or not. For me, the difference is that in an HitL model, the human is presented with a fait accompli and cannot control the process or the inputs. For the language sector, this is classic post-editing: You get a segment of MT output and you fix it, even if some other process would have been better for handling it. You fix things, but your inputs do not result in systematic change, at least not until the next retraining cycle. In human at the core – at least as I envision it and as CSA Research defines it – the primary difference is that the language professional is making the decisions about what resources to use and how to use them. If the resources are assistive and provide actionable and useful guidance, then they can help the human focus on where they add value. For the language sector, this is not post-editing, even though MT may be provided as a resource, because they translator is responsible for these choices and is free to ignore the MT and use other inputs, with assistance to make those decisions in an informed manner. In this ideal scenario, the translator is not forced to clean anything up, but instead is given some idea of where their input is needed and where they can safely leave some aspects to the machine, but they still can step in and make those decisions. They are not cleaning up after indiscriminate use of MT. In practical terms, will this difference matter? I suspect some companies will try to brand HitL as being more than it is. Some will talk of empowerment even as they quietly remove it. Others will provide genuine benefit. The key is that workers need to demand better than being converted into AI janitors. I’m not so na?ve as to believe that it’s just that simple and that companies won’t find more and more ways to convert humans into serfs on the AI manors, but we do see many cases where the opposite is true and where technology has liberated people to do far more. It's just that we tend to take those for granted and see the downsides of new cases. But the criticism of the concept of human at the core and its potential subversion in service of commoditized services and lower autonomy is one to keep clearly in mind as we resist that potential.

We’ve been seeing a lot of interest in knowledge graphs for multilingual content in the past few weeks. We (CSA Research) are working to finalize three pieces on this topic, with the next one due very, very soon. So if you are interested in this topic, keep your eyes out. I’ll be posting more soon.