Experimenting with LLMs for Developer Productivity

LLMs (or Large Language Models) is the latest craze in the world of Artificial Intelligence. It is getting a lot of attention due to its seemingly human like ability to mimic text based chat. It has a wide variety of applications such as customer support, search engines, life science research, marketing, finance, and legal.

I became curious as to what LLMs could do to help improve developer productivity. I recently concluded an evaluation on how well the most predominant LLMs were able to generate unit tests for an already existing, non-trivial, business focused microservice.

Perhaps the most pressing interests that many professionals have with LLMs is around job disruption. Will I lose my job to an LLM? That may well be (or already has been) the case in certain content creation roles but what about professional coders? Will those who toil behind the figurative curtain in the development of the web or mobile applications that you use every day be replaced by AI?

That is one of the central themes that I explore in the Experimenting with LLMs for Developer Productivity article recently published over at InfoQ.

Other questions or topics that are covered include the following. What would corporate adoption look like? What are the costs to using LLMs in this way? How effective are LLMs in generating unit tests? Do they really save the developer any time? What is the best user experience for this use case?

Does anyone else here have similar experiences? Feel free to share them as comments here or over in the article at InfoQ.