Friday: An example of AI augmentation analysis & guitars

I see many architectural diagrams but very few example of use, and even less of a process of failure until something successful appears. Many boards are sold a vision without showing adoption of AI is a process of failure until success is achieved, a result of in experience and lack of familiarity both of the AI with the subject matter and with the staff with that specific AI implementation (LLMs from different vendors will do things differently). This is why it's important to quickly iteratively deliver value to the organisation and build on those successes rather than that go big bang.

For this first friday example (I will build out more interesting examples over the coming weeks), I will use a "simple" example - how do you tune a guitar?.

On the surface it's using the tuning pegs right? Only if you're doing it simply.

In this example we're looking at a new build guitar. I know from manual analysis what the errors are but I want to see how the AI (with little prompting) will come up with. This has non-linear problems, it has multiple compounding issues and the final result is to minimise the error.

I will attempt to explain the issues and relate to data science data analysis as we progress. Think of this as the deterministic steps your agent would do to analyse and process. Alternatively with a stochastic agentic agent this is a manual walk through of the process it would do.

Step 1: It really is all about the data

Current AI all starts with good clean data. We're lucky in that we have clean data - or do we?

In playing each fret the strings can bend based on the pressure resulting in a very slightly different sound frequency which may make a musical note higher or lower than is expected.

The following data set is 'cleansed' in that I've tried to minimise the amount of error due to playing the string. In reality this step can be a make or break for the entire process of AI use case and it should therefore be a considerable portion of the use case.

Above a value of zero is a perfect result. Looks like we have some work todo. Positive and negative indicate lower and higher pitch than the musical note we want. However - complication here is that guitars are build imperfect, their temperament defines the positions of the frets along the fretboard, and as a result, the fretboard goes from positive error (flat) to negative error (sharp) with the cross over being planned into the guitar design. Essentially this make a guitar sound a certain way. In our case our cross over is around the 12th fret position.

Manually our data looks like this:

Manually, looking at this I can tell that the guitar has:

• too high nut action which means the gap between the fret and string is too large which causes the high errors to the left.
• neck relief that needs changing (this is the slight bow in the neck that can be adjusted) due to the size if the error across the the diagonal. Correctings this will minimise and flatten the trend of errors.
• 12th fret intonation of one of the strings (blue) is out as it doesn't follow the temperament trend.

So let's try with Claude.ai free plan over a number of days (as we consume quite a few tokens doing this). These tokens will be the cost that your organisation will incur during the build and operation using a third party LLM AI.

Step 2: Claude.ai Input at the prompt

Data shared at the prompt here is, thankfully, not personally identifiable information nor is the latest strategy documentation. When placing data into a prompt due-diligence in data privacy and cybersecurity is required. In the case of many AI free plans or non-enterprise plans, the AI will use that data for learning the model available to all.. which could result in suggesting data from your PII or corporate sensitive strategy docs in responses from your competitors.

The opening prompt I attach the CSV of the data without any documentation and simply issue this:

attached is the cent error (frequency error in 1200th of a fret) for a guitar. analyse the errors, take your time and suggest possible issues with the guitar.

Let me analyze this guitar intonation data carefully. The data shows the cent errors for each string (E, A, D, G, B, E from low to high) across different fret positions, where 0 represents the nut.
Guitar Intonation Error AnalysisClick to open component
After analyzing the data, here are the key issues I've identified:        

So this has (a) deduced the entire CSV data and what that relates to. That in itself is pretty amazing as a pattern recognition.

In an organisation - this enables documentation to be quickly assessed, additionally the AI due to its training understands that I as a human also have a visual condition so it elects to draw a graph to centre it's discussions around:

The same would be true for an agentic AI - it's important to train the AI in understanding the next consumer that takes the information (possibly a tool or service that the agentic AI has been told to use). The key here is not only the understand the technical integration but what does the data mean? Especially if the data is called the same thing but actually is used or interpreted differently between the sources and tools. If I walk about a "Home address" is that the legal address, or just one of many domicile addresses for an individual.

Remember Agentic AI agents can operate by itself, create it's own approach and strategy in using data or tools to solve the problem. It's significant risk if an AI sees a parameter name and then assumes that the same as the sensitive data parameter called something similar or the same name. The AI could then elect to send that PII data to that agent.. when it shouldn't. Guard rails should be in place to prevent that.

So the first assumption it made wasn't not a pleasant one - it decided to use only 20% of the data (the first 5 frets) but it got a pretty good analysis but what sub-sample and how biased/problematic that would be to analysis is something AI developments really need to focus on:

First Fret Issues:
Extremely high positive cent errors in the first fret (ranging from +9 to +28 cents)
The low B string is particularly problematic with +28.58 cents
This suggests the nut might be cut too high, causing sharp notes in the first position <<<***2

Progressive Error Pattern:
The errors generally decrease as you move up the neck
By fret 24, most strings show negative cent errors
This pattern suggests a possible neck relief issue or bridge height problem <<<***3

String-Specific Problems:
The low B string shows the most extreme variations
The high E string tends to have more stable intonation
The B string (second thinnest) shows consistently negative errors in higher positions <<<***1

Specific Areas of Concern:
Large negative errors around frets 20-23, particularly on the thicker strings
Inconsistent error patterns between adjacent frets, suggesting possible fret wear or leveling issues
The nut position shows relatively minor errors compared to the first fret, indicating the main issue isn't the nut slot depth        

It seems Claude form of analysis is done is a specific way. This is the same for each vendor and not all vendors analyse or reason the same way. This presents issues in switching vendors with prompts (this is quite a large undertaking). Additionally in the case of LLMs the learnt model only has a certain applicability time which results in organisations having to refresh models on a regular basis, with the cost of revalidating the business process AI agent (this is where having an framework for parallel operation and testing for revisions should be automated).

In terms of accuracy of analysis - at this stage is appears that it's highlighting obvious variations and starting to focus on reasoning the possible issues, although it's jumping ahead in terms of a working through the process from what can cause the most impact, this is also apparent from the recommendations:

Recommendations:
Set-up Adjustments:
Check and adjust the truss rod as the pattern suggests neck relief issues <****3
Verify bridge saddle heights and positions
Consider having the first few frets checked for proper height and wear

Hardware Inspection:
Inspect the bridge saddles for proper compensation
Check nut slot depths, particularly for the low B string <<<****2
Verify fret leveling, especially in the higher positions

String-Specific Actions:
Pay special attention to the low B string setup as it shows the most extreme variations
Consider adjusting the bridge saddle for the B string (second thinnest) to address the consistent sharp notes <***1

Would you like me to focus on any particular aspect of this analysis or provide more specific recommendations for any of these issues?        

So I've marked three recommendations with <<<**** but it's almost there, the issue is the key problems are lost in noise of details that would (as a process of solving this) come after. The danger here is that it's not targeting a sequence of priorities, partly because a I didn't ask that question, which may cause actions to be less than optimal or even incorrect in the steps of the tooling without further training. It is unclear what the steps are in these recommendations.

This is why it's important that as part of agent development test data and prompting is analysed and tested to ensure it's not randomly going off and using tools (cost) that it doesn't need too.

Additionally vague verbose output (large on detail that's not completely associated) increases the context window use, as most LLMs work by keeping the entire prompt history and run the context window over it.

This causes two things:

• increases cost through the verbose output increases token used (costed)
• increases context window use which lowers the performance of the model.

So Claude has got this right (in a verbose way):

• Nut slot high - makes the string bend more and go higher (sharper note)
• neck relief (truss rod adjustment)
• the intonation of one string is out (which is commented on originally), which is easy to adjust.

We can do better - so I told it to use all the data (because that was what I was expecting initially!):

include all frets

With the complete data, I can now see several additional patterns:

Three Distinct Regions:
Frets 1-7: Large positive errors (especially frets 1-4)
Frets 8-15: Relatively stable with moderate positive errors
Frets 16-24: Trending negative with significant fluctuations

Critical Points:
Fret 1: Extreme sharp errors (+9 to +28 cents)
Fret 20: Notable negative spike, especially B string (-12.08 cents)
Fret 21: Low B string hits -14.71 cents
Fret 23: Low E string reaches -15.19 cents

High-Fret Issues:
Significant deterioration in intonation above fret 16
Most strings trend flat (negative cents)
Erratic behavior in the final frets (21-24)
High E string maintains better stability than others in high positions

String-by-String Analysis:
Low B (B1): Most volatile, with extreme swings (+28.58 to -14.71 cents)
Low E (E2): Similar pattern to B1 but slightly more controlled
A and D: Generally track each other, with D showing more negative tendency
G: Relatively stable in middle frets but poor in high positions
B (B2): Unique pattern with more negative tendency throughout
High E (E3): Most stable overall, especially in higher positions

This suggests:
Structural Issues:
Possible neck bow or back bow affecting first position accuracy
Potential fret wear or leveling issues in the upper frets
Bridge compensation may need significant adjustment

Action Needed:
Complete fret leveling job, especially focusing on frets 16-24
Truss rod adjustment to address the first position issues
Bridge saddle height and compensation adjustments for each string
Possible nut slot depth adjustments, particularly for the low B and E strings

Priority Fixes:
Address the first position issues (likely neck relief)
Level and dress frets, particularly in the upper register
Adjust bridge compensation for each string
Fine-tune nut slots

Would you like me to elaborate on any of these points or provide specific recommendations for addressing particular issues?        

You now note a couple of things - firstly it uses all the data and (surprise!) it notes some additional patterns. Second it seems to be more confident in the issues, actions and priority fixes!

Prompt Templates and prompt guardrails really are important on an operational basis and not just at the build stage. Think of these is precisely the same way as cybersecurity vulnerabilities - they may be your fastest way to resolve immediate issues across large numbers of AI services.

It now notes that the highest frets (the ones closest together) could do with attention (that's true they do).

However now it's priority fixes are incorrect - it's down played the nut slot height (not so good!), but is noise pointing to the neck relief (correct). To make it more concerning - it's more confident that it's correct.

So this indicates yet another issue with prompts in AI - specifically for Agentic AI which can generate or use prompts without oversight (in theory). It is not only the stochastic nature of generative AI but also the reaction of the AI to small changes in data sets, prompts or sequences of prompts can change the resulting action of the Agentic AI.

So I give it a hint/correction:

Consider nut action height

Yes, let's analyze the nut action height's potential influence on these intonation errors.        

The graph on the right shows that the guitar in tune but as soon as you fret the first note it's out of tune (as you'd expect with a high nut slot deforming the string). It's a nice response but it doesn't really show the right reasoning (it's the increasing ramp in the original graph that is indicative of this problem).

Looking at the nut and first fret relationship:
Clear Nut Height Issues:
All strings show a dramatic increase in sharpness at the first fret
The low B string shows the most extreme jump (from -3.93 to +28.58 cents)
Even the high E, with the smallest jump, goes from -0.16 to +9.73 cents

Pattern Analysis:
Thicker strings show larger error increases at the first fret
Nut errors are relatively minor (between -3.93 and 0 cents)
The dramatic increase at fret 1 strongly suggests high nut slots

Specific Concerns:
Low B string: 32.51 cent jump from nut to first fret
Low E string: 25.60 cent jump
A and G strings: ~21 cent jump
High strings show smaller but still significant jumps

Recommendations for Nut Action:
Immediate Actions:
Lower nut slots, starting with the low B string
Target reduction should be approximately:
Low B: ~0.015" reduction
Low E: ~0.012" reduction
A and D: ~0.010" reduction
G and B: ~0.008" reduction
High E: ~0.006" reduction

Technique:
Use graduated nut files to maintain proper string seat
Check action with feeler gauges at first fret
Target first fret clearance:
Low B/E: 0.020"
A/D: 0.018"
G/B: 0.016"
High E: 0.015"

Follow-up:
After lowering nut slots, recheck intonation at frets 1-5
May need minor bridge saddle adjustments after nut work
Consider slight truss rod adjustment after nut modification

This analysis suggests that high nut action is a primary contributor to the intonation issues, particularly in the first position. Would you like specific guidance on the nut slot adjustment process or information about how this relates to the other intonation issues we identified?        

Now we're cooking - it's even nicely provided us with some reductions in the gaps!

However this is not an Agentic AI compatible result - if we tell it to consider a specific aspect and then it decides (confidently again) that this is the primary contributing issue, then it is not understanding the problem the Agentic AI should solve but merely the word salad it is relating to articles it knows.

The key take aways from this example:

• Agents and Agentic AI needs to understand more than word salad relationships, it needs testing to demonstrate that - which is likely to slow adoption heavily.
• Skillsets of standard Prompt Engineering is not Agent/Agentic AI engineering, so don't expect employees to replace development in the short-medium term.
• Agentic and standard Agents are prone to large changes for small inputs, so ensure your risk, cyber and AI extend to cover guard rails and restrictive rule sets that can be pealed back over time.
• Human understanding of the problem is still required for the short-medium term. AI even general 'AGI' is going to see problems from a general and global perspective but not specific for your customer or your organisation. It will improve but business requires differentiation to compete. A single approach for all businesses will lead to zero differentiation so have you addressed this in your strategy?

So this begs the question: Is this really a AI problem to solve?

The short answer is not entirely - there's some simple regression but the complexity of the geometry warps makes the implementation a multi-dimensional simple regression but of a low order to make it simple to tackle in an AI approach.

For those guitar geeks the guitar in question is a 7 string, 28.625" scale length. DIY scratch design and built.

Construction is from 7 different woods. A through neck design of a lamination using Maple, Wenge, Walnut, Purple Heart, Walnut, Wenge, Maple (if you look at the headstock you can see the lamination pattern), the body wings are made from Alder with a thick flame maple carved top. The fretboard is African rosewood (Bubinga).

Hardware: Gotoh fixed bridge, Schaller tuners, Seymour Duncan pickups.

The funky design is to sit on the lap in a classical style without the fretting hand sitting above shoulder height. The neck is wider and thicker than normal to prevent my hands from cramping.