Now is the time to talk to your developers about generative AI

If you've played around with ChatGPT's GPT-4 model at all you know by now its stunning capabilities at generating code. While some wring their hands that this capability will replace programmers, I believe it will be another productivity tool that if used correctly, will make already labor-starved software engineering teams an order of magnitude more effective.

There are serious risks here. I am concerned that developers will generate code and deploy it without understanding it. Even when the code is correct, this greatly diminishes your team's ability to maintain the code. I also am not convinced generative AI can produce accurate code without carefully engineered prompts that clearly define the context. An easy example is the versioning of frameworks and APIs--where AI-generated code may confuse or even mix functionality and behavior from different versions of APIs or frameworks. In some cases, it may even hallucinate framework or API affordances into existence that simply aren't there.

The real risk is not that generative AI will find its way into your development organization, but that you won't introduce it yourself. And when your developers start using it (which they will if they aren't already) you won't be able to manage the risks or understand the rewards.

Now is clearly the time to develop a plan.

1. Gather your team leads and brainstorm. Task them with experimentation with tools like Copilot and ChatGPT's GPT-4. For example, ask them to take a few tasks off the backlog and see how these tools can be applied to those tasks. Come up with some learnings and incorporate them in the following steps.
2. Educate your team: Organize training sessions to help your team understand the benefits and risks of using generative AI in software development. Cover topics like what tools your leads found most effective, how to create prompts that best articulate context, their limitations, and how to interpret and validate the generated code.
3. Establish guidelines: Develop clear guidelines and best practices for using generative AI. This should include when and how to use AI-generated code, the review process, and the expected level of understanding that engineers should have before incorporating generated code into their projects. Consider tagging all AI-generated code in source control be sure to include who generated it, the prompt used, the model used, and who reviewed it.
4. Choose the right tools: Open up the debate to the entire team, but guide the discussion to settle on a limited tool set for now. Make sure the leads know how to use said tools, and can provide proper guidance to their teams. This will be a rapidly changing landscape, so revisit the "approved" tool discussion often.
5. Integration with existing tools and workflows: Explore the integration of generative AI tools with your existing development tools and workflows, such as IDEs, version control systems, and CI/CD pipelines. This will ensure the traceability of AI-generated code through your build/deploy process.
6. Implement a strong code review process: Require thorough code reviews for any AI-generated code, just as you would for code written by human developers. Team leads should ensure that engineers understand the generated code and that it meets your organization's coding standards, quality, and security requirements.
7. Encourage collaboration: Promote a culture of collaboration and open communication between team members. Encourage engineers to discuss any issues or uncertainties they encounter with AI-generated code, and provide guidance on how to resolve them. When new or interesting phenomena are discovered (i.e when the generative AI does something "weird"), make sure everybody is made aware.
8. Monitor and measure: Track the impact of generative AI on your team's productivity, code quality, and other key performance indicators (KPIs). This will help you assess the effectiveness of the AI tools and identify areas for improvement.
9. Continuously improve: Based on the monitoring and measurement results, make adjustments to your processes, guidelines, and tooling as needed. Continuously refine your team's understanding of generative AI and stay up-to-date with the latest advancements in the field.
10. Share success stories and lessons learned: Share success stories, challenges, and lessons learned with the rest of the organization to encourage broader adoption of generative AI and foster a culture of continuous improvement.

Developers are going to use these tools. So take the lead and help them be successful.