Generating code with Sourcegraph Cody on a scale

Please use this article only for learning purposes. Many risk aspects should be considered before using in real products

What is Cody?

Sourcegraph Cody is GenAI product which we are using in our team. This article is not promote the tool - but mainly to share the experience working with AI, the tooling for AI, and applying those tooling for specific goals. I'm quite sure there are plenty of other similar tools available and there will be many more.

Basically main Cody UI is not even the web app but VS Code extension. It's very similar to Co-pilot or another extensions.

Basically you do almost same as in chatGPT but it is also integrated with your code. There are even alternative IDEs like Cursor AI which are doing similar things in own IDE, not as extension.

Few years back I could only dream about something like this but having it now I started feeling like I want more, way more than this. I want to be able to write scripts, prompts, CI/CD pipelines and use AI features from there. The main tool to achieve this is API of course - it delivers all available features, can be called in complex scenarios, including streaming large files, parallel runs and all other cool stuff. But at the same time it comes with a cost of development and complexity of your solution. What can be the alternative? Usually is CLI.

What is Cody CLI?

Luckily Cody also comes not only with API but with CLI which despite being experimental is quite useful already.

The interface of CLI is mainly about the chat. So what you can do with it:

• chat with a custom message
• chat with a message from --stdin which makes it convenient for command pipelines
• provide additional files as context

As a result it returns back a text message with a free text response like:

Here's the TypeScript configuration file:

```json:tsconfig.json
{
  "compilerOptions": {
    ...
}
```

Create the main TypeScript file:
```typescript:src/index.ts
class Greeter {
  ...

```

Let me know if you want to know more!        

So as you can notice - it replies back literally like a human providing us back explanations but also file snippets embedded inside the response. If tools like VS Code extension can parse it automatically - here you need to this yourselves in a code.

Codygen CLI as a wrapper for Cody CLI

Being node/TS dev I decided to extend the existing tool with few more features which I miss in the original CLI.

First of it's parsing the response and automatic extraction of files.

So if this command will just print the response to the console

cody chat -m "Hello world ts app"         

a command like this will do same but will also store files in your project

cody chat -m "Hello world ts app"  | codygen extract        

additionally you can mention where you want to store newly generated files

cody chat -m "Hello world ts app"  | npx codygen extract -o dist/hello-world-ts        

This command will save those files physically:

Seems too simple to keep reading this article, right? But let's go further.

So if we can parse the response what if can also control the input for the chat in own way? Like we want to control also what is the prompt, may be we want to build dynamically. Or we want also to control what are context files?

That's how I came to the idea to implement also own chat command, but also backed with a programmable and hackable config?

Imagine we have a config like this

import { CodygenConfig } from 'codygen';

const { API, RUNTIME } = process.env;

export default CodygenConfig.define({
  prompt: `Create a Sample ${API} app using ${RUNTIME}`,
  output: `dist/${API}-${RUNTIME}`,
});        

So what we do here - we build the prompt dynamically by applying environment variables such as API ( consider the product spec ) and runtime ( like Node, Java, Python or anything else )

Then we call a command like

 API=petstore RUNTIME=node npx codygen chat --config codygen.config.ts        

and this command generates us pretty much the same as what we saw in the example, but already driven with parameter

But if we can do this for one pair - can we do this for a matrix?

Pretend we have several products ( backed by some specs ) and several runtimes we want to support. Now if we transform our command into something like this ( by taking 3 products and generating apps/servers/clients for them in different runtimes ) into something like this:

parallel -j3 --colsep ' ' "API={1} RUNTIME={2} npm run build:matrix:step" ::: petshop brewery forex ::: node java python        

Then what we have within few minutes ( time not really matters ) something incredible as this:

Of course - the goal is not to generate sample apps. The point is totally different. Look back - the only code we used to create it - is just a config where prompt was as simple as that:

Create a Sample ${API} app using ${RUNTIME},

But in real case the prompt will look differently. And it may be static prompt or generated by the code, or even pluggable and hackable prompt. Same for context, and output. Those things may be also defined programmatically. And this code what we see above allowed us to generate several thousands of lines covering different cases and touching any runtime unless the model knows about it. Moreover being also an ABAP developer I noticed that in general model is not really good in ABAP yet, but it is really good in template matching. So once you give it a real example - like a sample app. It can create you tons of other apps in the same way, but within a given context. It is unbeliavble.

How we can use it in SAP?

In my company we have really large number of different interfaces, mainly described with openapi or graphql specs. To integrate with them we need to do few things in ABAP:

• define the data structure and types ( including all nested types )
• implement serialisation/deserialization logic
• implement actual http calls
• implement custom authentication
• implement the mapping between your inputs and API inputs like path, method, query, headers, body
• parse responses properly
• implement error handling according to a spec.

Of course there are tools like https://abap-openapi.github.io/web-openapi-client/ developed by amazing Lars Hvam Petersen which allow us to generate the code, just by giving a spec. And we also have such an app developed in house. But problem is - whenever something is changing, you need to change the generator. And sometimes it can be costly. Like even the tool we developed - it was supporting swagger 2.0 and now it became openapi 3.0 and it already doesn't work well. Or what is more interesting - we want to generate the client in a different way. What does it mean for us? we need to write the code.

We needED and NOW we don't!

With genAI it is not needed to explain every single details - it's just enough to give it a pattern. So what i did - I created the code which is not even present in my system, but more like a code I would like my future code to look like:

And now I'm asking chat - please generate me the client for a certain spec, following the pattern i'd like to have and what it creates after it's already a working product:

And notice - it doesn't care about linting or any other thing - it just follows the pattern you gave. The best thing with this approach is that you don't even waste time for types declaration:

neither for methods declaration:

neither for exception object creation ( with exact types from the spec )

You need more files? abapGit integration? Not a problem. You just literally add one single line to your prompt.

and here it is - every file is already backed by a file descriptor:

You know - I'm not scared of AI anymore. What I do want is to encourage everyone to explore it and make it your slave. I don't feel like it's replacing me or doing my work, no! It's actually make me much stronger and more powerful.

However at the same time I understand that it may bring various risks and you should be very careful with using it. But once you control steps and make sure all proper controls are in place like SAST, unit test, integration testing and other important controls - I think it can be considered as a part of a product development lifecycle.

Thanks for reading till the end. Let me know what do you think about this.

Useful links:

• Codygen github project
• Codygen npm package