TX services & NLP development

Here at the DSC, we had the amazing opportunity to talk to Milena ?or?evi? a Data Scientist and Machine Learning Engineer, and ?? Milica Panic ?? a Data Scientist, Mathematician and Researcher from TX Services .

In this article, we asked representatives of their Data Science team a couple of questions regarding the future of NLPs, here are their answers

?

1.????What is your opinion on the current state of NLP research and development? Are there any particular areas you think are especially promising or in need of further exploration?

?Milena ?or?evi?:

Natural Language Processing has experienced exponential growth in the past few years, especially in the last 6 months. In both research and industry, we can see an effort being put into moving this technology for more everyday usage and to a bigger range of products and services relying on data. I don’t think this was something that most of us expected to happen, at least not at this speed. To put it mildly - feels like every few months in this field is the equivalent of a year’s worth of progress. I certainly see the potential of open-source LLMs and the effort the community is putting into it. The biggest challenges are still in evaluating and scaling these models, but also in expanding their ability to work better in multiple modalities.

??Milica Pani?:

NLP has experienced remarkable advancements in recent years, owing to the progress made in deep learning techniques. Many NLP models now exhibit exceptional accuracy in a wide array of tasks, encompassing sentiment analysis, language translation, named entity recognition, and more. Nevertheless, there are still numerous challenges within the realm of NLP that necessitate further investigation and innovation. One area of significant promise lies in the enhancement of models capable of contextual understanding in natural language. Additionally, an intriguing field of exploration focuses on the development of models with a heightened ability to comprehend the subtleties ingrained in languages, such as sarcasm, irony, and other forms of figurative expression.

2.????Can you discuss how recent advancements in NLP, ChatGPT and GPT-4 models, have impacted the field and what opportunities and challenges these models present? Additionally, can you explain the role of attention mechanisms in NLP and how they improve model performance?"

Models and services from OpenAI, but also a few others have made a large impact on both industry and research in AI and NLP. In my opinion, they showcased what LLMs can do to a wider audience and so in a more convenient way. Challenges that NLP had only a few years ago are now easily resolved, which opened up a space for exploration and a good set of opportunities for future applications. They raised the bar and introduced us to a whole new range of possibilities. Attention mechanisms are the key ingredient behind the widely used language models today. They allow the model to gain an understanding of the meaning in a text, as they aim to capture the relationship between different parts of the text. In this way, these mechanisms can better handle longer instances of texts and thus provide a better, and more accurate output.

?

3.????How has NLP been implemented in your work at TX Group? Could you provide us with an example of a successful application of NLP in your projects?

Natural Language Processing is a part of our daily work, whether it’s a simple text processing task, or building a text classification system. One of the projects which are predominantly using this technology is a platform for translating content to consumer segments. This platform allows us to connect two significant businesses of our group: media and advertising, and with it provide important insights from the data.

?

4.????What are some challenges that you have encountered while working with NLP? How did you overcome those challenges?

From an engineering perspective, these technologies are rather new, and some of the best practices are yet to be established. Therefore, I would say that the amount of experimentation and research needed to properly apply them is rather higher than in other computer science fields. I like this aspect of NLP because it makes space for more creativity in finding solutions and there’s often new stuff being developed and new approaches to try out.

?

5.????Can you speak to the ethical considerations involved in NLP development, particularly around issues such as bias, privacy, and fairness? What steps do you take to ensure your models are ethically sound??

I like this topic and I am glad it’s gaining even more traction in recent months. I will take here an example of text generation, where it often happens that the content produced by some language model doesn’t align with certain guidelines and ethical standards that society aspires to. This is considered an erroneous behaviour, and as such it creates a liability for its applications. Although, these kinds of mistakes are not comparable to for example false positives and false negatives. This happens because they can reflect the training data or the input of a model and appear to be correct, which makes them harder to be detected, especially taking into consideration the complexity of a topic such as ethics. For our services, we are doing a two-step evaluation: the first one is a quality assessment of the input data that we are using, and the second one is the assessment of the model itself and the outputs it produces. Both steps are important and evaluated by an adequate expert, meaning that the final say comes from human feedback.

6.????Can you share with us some exciting new developments in the field of Machine Learning that you find particularly interesting? How do you see these advancements impacting the future of NLP????

Milena ?or?evi?:

There’s been so many exciting developments in the last year alone, that it’s hard to narrow it down to just a few. If I am looking more into the area of NLP, I must say I was pleased to see the release of a holistic evaluation approach. This work was published earlier this year and can be applied across different tasks and methods/models, and most of them were evaluated through it, which I see as a nice benchmark for both researchers and practitioners in the field.

Aside from this, I am very interested in the development of frameworks using both reinforcement learning and attention mechanisms, as this is already producing great results with prompting LLMs. In light of that, significant progress was done by the OpenAI and the developments they’ve made so far have put the world of artificial intelligence on a faster track. It’s very worth mentioning that great progress was done also by an open-source community and I hope we see even more progress from their side. It’s an exciting time for AI and NLP in particular, which makes predicting the future a bit harder. Nevertheless, I hope we see more multimodal LLMs and even stronger work by the open-source community.

Milica Pani?:

NLP has been greatly transformed by the introduction of transformer architectures, which rely on self-attention mechanisms. These architectures, exemplified by the popular BERT and GPT models, have exhibited exceptional performance across various NLP domains, such as language modelling, question-answering, and text generation. As these advancements continue to unfold, we can anticipate a future where language understanding reaches unprecedented levels, enabling us to build more intelligent, context-aware, and human-like systems that revolutionize communication and information processing.

We would like to thank Milica and Milena for sharing their opinions and their experience, and we would like to thank TX Services for sharing their knowledge with us, and partnering with us.

#naturallanguageprocessing #ai #datascience #gpt #LLM #bert