What if: AWS DevOps powered by LLMs

Imagine you are driving home from work and you receive a Critical Alert!

A system or environment is down and.....

you need to reboot an EC2 instance

This is a common scenario for Site Reliability Engineers, Sys Admins, and NOC staff who are constantly attached to their phones or laptops in order to solve issues during off-hours or weekends.

Over the years dozens of solutions have been built to help solve this issue. From amazing solutions like PagerDuty or Atlassian 's Opsgenie to Infrastructure as Code and automated runbook remediation solutions.

However, I've always had a fascination with ChatOps, and with the advent of #generativeai LLMs, I simply wondered if the combination of...

ChatOps powered by an LLM could be a utopia or a disaster

The goal is to help Tier 1 teams

In the quest to simplify or automate #devops, infrastructure as code technologies like #terraform and #cloudformation have risen to the forefront of system maintenance and automation.

While undeniably useful, these tools can present a significant hurdle to Tier 1 teams such as Helpdesk or NOC squads. These teams may not have the technical prowess to write, edit, and deploy Terraform or CloudFormation scripts, and therein lies the predicament.

But what if there was a way to make AWS management easier for these teams? To unify the approach, and leverage the same technology across the board, regardless of skill level?

AWS ChatOps powered by GenAI LLMs

Imagine simply being able to write in #slack "restart ec2 instance id-something" and having an LLM execute the change.

Suddenly, for better or worse, Junior teams can execute changes against an AWS account.

Alternatively, if voice-to-text translates your words incorrectly this could be a disaster for an environment!

We would need a checks and balance operation, a test process, approval process or more to ensure this solution does not bring down production environments, but its an very achievable future.

How To: Unleash LLMs on AWS Management

What you will need:

• An LLM that can accurately translate AWS tasks into code parameters. Recommendation - OpenAI, AI21Labs, or opensource WizardCoder
• Minor experience with the AWS boto3 SDK
• A opensource Localstack deployment for AWS API testing
• Instruct-Prompt Engineering skills

How it works TLDR:

• Insert a request such as "list out s3 buckets" into an LLM
• LLM responds with the boto3 requirements ("service", "command", "arguments")
• Dynamically insert and execute those three parameters a the boto3 function
• Profit?

Lets talk details

LLMs with the right prompt can understand the context of a task request, dynamically translate it into actionable code parameters, and execute AWS-specific commands using the boto3 library. The result? Your users can perform AWS management tasks directly from a chat console.

Here's a scenario

For instance, if a user asks to "list all s3 buckets," the LLM prompt will need to understand this request and generate the relevant boto3 code

The user receives a response from the LLM with three parameters, Service, Command, and Arguments which can be parsed and executed against a dynamic boto3 function like this:

For this experiment, I used a Localstack container to emulate the AWS API with boto3, ensuring no AWS account was harmed during this experiment.

Big possibilities

LLMs ability to translate a request into something actionable can enhance AWS ChatOps beyond what was previously imaginable. You're not only simplifying the process but democratizing it, making AWS management accessible to every team member, regardless of their technical expertise.

While this example is rife with potential dangers, it's simply an example of how to integrate #generativeai LLMs, ChatOps, and AWS in order to deliver a novel experience for #devops users.

By employing LLMs, we are accelerating #chatops, making it more intuitive, responsive, and accessible. Whether it's a Helpdesk representative or an experienced DevOps engineer, anyone can now manage AWS infrastructure using plain language.