Life, Universe And GEC
In this series, we will dive into a key component of deep tech at CACTUS LABS, which powers several of our AI and automated products and slowly has become one of our crown jewels.
This article explores the origins of our GEC tech that solves an important user problem. GEC (Grammatical Error Correction) may seem trivial on the surface but it is actually quite a complex beast when viewed together with its army of minions (sub-problems).
But first, some context. One of our business units,?Editage , is a team of 4000+ native English editors from across the globe, which includes one of the largest in-house editing teams. They have expertise spanning over 1,300 subject areas and are committed to delivering high-quality editorial support while adhering to the highest publication standards, ethical considerations, and compliance norms. In the mind of every professional academic editor resides a complex body of grammar rules that helps them tune in and filter out errors in spelling, punctuation, word choice, and sentence clarity and structure.
While this process is fairly well known in the non-technical world as language editing, in the world of natural language processing (NLP), this falls under a cool yet authoritative umbrella of solutions termed GEC aka Grammatical Error Correction.
Think of human editors as high-level expert systems with the primary aim of fixing grammatical errors and improving the clarity of the text. A certain persona comes to mind, one who constantly corrects people’s grammar mistakes; you know the word: it rhymes with kamikaze. Now, imagine an automated system, your personal companion, fixing the content and helping polish your writing skills over time without confrontations, and voila, you are starting to see where these systems fit in the big picture.
Over the past 3 years, we have built and refined a solution that can correct both trivial and complex writing errors with a blend of traditional AI heuristics and modern machine learning.
Typical Grammar Edit Systems
Here’s something to give you a glimpse of what our NLP team deals with on a regular basis.
As you can see, the quantum and type of edits required in a scientific manuscript can be as unpredictable as a certain virus.
The accuracy of the automated solution has a heavy dependency on the gold standard set by the human editor; but because of the subjectivity of human annotation and the variations in the editorial style of different editors, it becomes a bit difficult to generalize quality even for the same piece of text.
History of GEC
For one, this field touches upon several topics in linguistic analysis: phonology, morphology, syntax, semantics, pragmatics… the list goes on. The impact is far-reaching and wide since it affects millions of people who use English to communicate regardless of their native language.
In our case, even though our tool has processed millions of manuscripts over the years, the problem of human levels of grammar check aka manual review is still needed and there are things yet to be solved (in part due to a multitude of open, unresolved problems in AI technology).
There was a renewed interest in GEC since 2011 when the?Helping Our Own ?task was presented to promote the development of automated tools that can assist authors in writing tasks. This is what started the conversation around grammar correction. Since then, people have been actively following developments in this field, and with the emergence of large language models (the Age of BERTs and GPT), we are seeing giant leaps in model outputs; although we are still far from 100% automation, some of today’s state-of-the-art GEC systems perform solidly as assistive tools.
Current State of GEC at CACTUS Labs
Cactus Labs, our platform that serves to convert these homegrown R&D solutions into products that can, in turn, serve researchers better, is made up of many internal tribes that handle different components of GEC. We recently released the 3rd generation (iteration) of our proprietary AI model, with a primary focus on editing accuracy and the explainability revolving around these AI-powered edits.
A lot of our systems have been built to align with the formal, academic language used in a very specific domain (for example, medical science), and evaluation metrics for such custom GEC systems have been a challenge, one that’s felt even across the active research ecosystem. Still, we do follow industry norms of precision-recall and F-scores to try to keep our eyes on the prize as we cater to practically hundreds of research areas, ranging from applied physics to anthropology. And at the end of the day, machine learning, governed by mathematicians and statisticians, has the last laugh.
领英推荐
Our major tribe, named after a famous telescope, has a keen eye poised toward the sky to find bigger and better constellations — in other words, we are constantly tuning the technology such that it grows not only in size but also in leaps and bounds in terms of efficiency and other internal benchmarks. We have various dedicated focus groups working on preprocessing and big data, exploring various NMT (Neural Machine Translation) systems, language models, as well as better ways to tag and generate quality, training-ready data.
Just to give you a brief overview of how one would go about building a GEC system on their own, this is pretty basic maneuvering what we talked about in the previous section.
Future/Scope
As we venture from assistive editing (NLP) to assistive writing (NLG - Natural Language Generation), our journey brings a lot of promise as well as challenges.
We recently announced a new product in our ecosystem which helps researchers achieve better and cleaner manuscripts while providing real-time prompts and suggestions to improve their technical writing in English as well.
You can take it for a spin over here ->?https://paperpal.com/edit
Granted, there are still a lot of open problems tied to the core language and complex terminologies that come with various academic domains. But with these new large language models like GPT-3 and recent work by various NLP research groups, it seems we might be very near to replicating human-level performance or certain sub-problems (and we hope to cover them in future articles under this series)
While it’s difficult to find a “One model to rule them all,” an ensemble of countless different models is what makes the system thrive and evolve. And that is why, although technically not an AI problem — a system engineering one, it’s important to set your model versioning, data management policies, and A/B testing and rollout strategies in place. Nonetheless, the technology has successfully solved an important user problem, by being available to anyone who needs their own personal academic editor 24x7.
Does any of this sound interesting? Is NLP your true calling? We are always looking for more people to join our various tribes @ CACTUS Labs. Make sure you look at our?careers page .
Additional reading/repos to explore: