IIB/ACE and DevOps

In our role as developers or software engineers we are finding more and more of the teams we work with a moving into multi-disciplinary?teams and DevOps (SecDevops, agile, scrum, there are many names).

Not to start a flame war on agile vs DevOps,?my take on this is, as developers, we need to be across a number of skills and need to be able to help our teams in a number of different roles?(and not really call it anything).

More developers need to be mindful of testing their code and also the system as a whole. There are fewer silo's of developers vs testers, where you would write some code,?throw it over the fence to the testers and at some point in the future they will send back some bugs.

We are seeing fewer BA's, so developers are going out and talking with the business. Often developers are becoming part of the living memory of the business rules and processes.

So we are expecting more from our developers not just in terms of what roles they play but also what skills and tools they need to use.

Hands up who has learnt how to write Docker, Docker-Compose, Kubernetes deployments, Flux, Istio, OPA, Helm and the 3000 AWS CLI commands in the last few years ??

As well as having to write code in probably more then one languages (Polygot programming), attend stand ups, run sprints, meet the business and do production support ??

All while having a shift left mentality ?

We work mainly in the code quality space and continuous integration space (SonarQube + your choice of a Ci/Cd tool - Jenkins, Bamboo, etc) so our challenges are different from what a lot of development teams?have in that we don't "business users" to work with, we work mainly with developers.

And while we work with teams to help them in their challenges, being that we are not part their organization, we do have another degree of separation that we need to deal with.

We sometimes get asked, can your tool help with how our applications perform ? Yes and No. Very useful response. So yes, we can help to identify things that are bad.

But often times we don't have visibility over an organizations logs, telemetry data and code. Often the same challenges that developers in DevOps teams are trying to overcome.

So we are trying to help by intelligent guessing.

Often teams have enough trouble getting access to logs and this kind of data internally let alone with the right detail to share with 3rd parties.?

But slowly more and developers are being involved with day to day operations, so with access to the operational data and tools such as Kibana, Grafana and Prometheus?teams are getting more tools to be able to be part of the Ops in DevOps.

So anything that we can do to help teams access the data about their applications earlier in the lifecycle is going to make their lives easier.

So a while back we started generating diagrams for IIB code that we scan for teams. These diagrams help us with understanding architecture from a birds eye view.

When we added in code test coverage using tracing, we had to work with some of the lower level debugging logs from an IIB/ACE instance.?

Working with these logs, gives us more data from a operations perspective during?the unit test/integration testing process. This is the same data that we might feed into a Kibana report or Prometheus dashboard around operational performance.

The difference being that with operational monitoring/metrics/telemetry we are looking at what is happening over time, with the unit testing/integration?testing reports we are looking at a point in time snapshot.

When we add in this point in time data with the current visibility over code we can start providing developers better insights to performance earlier during the development process.?

Shifting left the ability to detect and investigate certain of types of performance issues earlier.?

We previously posted about having the ability to visually represent the performance of different flows and nodes in IIB/ACE using that auditing data that the runtime provides.

We have just recently added further more detailed support specifically for mapping nodes. We track in the entry and exit from a mapping node to indicate whether it has been tested or not,?we are using that same trace information to determine how long the mapping node executed for (and how many times).

As we do with statistics for flows and nodes over time, we can use the mapping timings data over time to produce more details graphs of performance to help indicate which mapping nodes are the mostly costly, and determine if there are performance changes to the application over time.

To also highlight mapping performance, we added an additional rule - "This mapping node has the highest average elapsed time from the statistics available (WMB)"

Like the other rules we have for performance tracking (node performance and flow performance), they in themselves shouldn't fail a build. Even a mapping node that follows all?the best practices and performs well will still be marked as the most expensive compared to the others.

Also, with custom metrics:

We can also record performance against flows and nodes, so we have also added to the custom metrics we can record against an individual flow,?so as well as recording a node count and longest path through a flow, we can also record the average CPU usage for a flow found during testing.

So now (hopefully) we can give DevOps engineers using ACE/IIB more tools to help understand their applications without earlier and more seamlessly, in one spot.?

Hopefully making their life easier by having a few less tools that they must learn.

If anyone is interested in a demonstration or has any questions, thoughts or wants to try it internally please reach out to me either on LinkedIn or via our website:

https://bettercodingtools.com

You can also reach me via email at:

[email protected]

Or contact me via the contact page on our website:

www.bettercodingtools.com/contact

Regards

Richard