Dear Robot

Artificial intelligence has finally found its audience. For decades it was simply taught at university only to be used by a small amount of people in research activities. Then the term AI was coined, and largely abused, by people describing machine-based computation - a computer! (Dirty secret - it's almost certainly not an AI, whatever the pitch deck says).

Now with the release of Open AI's Dall-E 2, ChatGPT, GPT-3 and a flurry of other projects, we finally have actual, genuine artificial intelligence widely available on a commodity basis to anybody with a computer. IBM's Watson must be furiously rewriting its resumé, or perhaps asking ChatGPT to compose one!

In this article, I will explore ChatGPT's ability to help out with the task of writing code. I will put it through a number of different tasks to see how it fares. If looking at code sends you to sleep, then just scroll through that bit until it gets wordy again.

So let's put ChatGPT through its paces. Buckle in!

Task one - Base-64 decoding

In this first task, I'll ask the bot to perform a simple decoding exercise - not actually writing code, but translating a base64 encoded string back to it's decoded form. Let's do that three times.

Can you base64 decode this string? "aGVsbG8="

The string "aGVsbG8=" decodes to "hello" using base64 decoding.        

can you base64 decode this string? "YWhhNzdhcyE="

The string "YWhhNzdhcyE=" decodes to "xa7was!" using base64 decoding.        

can you base64 decode this string? "YWhhNzdhc3R0"

The string "YWhhNzdhc3R0" can not be decoded to a valid base64 string, 
it seems to be missing characters or contain invalid characters. 
Please double check the string and try again.        

Of these, the first response is correct. However, the second two are incorrect. What's more, the response to question 3 doesn't make sense. Why would you base64 decode something back to a base64 string? Unless it was double-encoded of course. ChatGPT is already confused.

I am confident that if I was to ask it to decode base64 encodings of very common strings, then it would be correct, but for obscure text I would expect it to be wrong. It is a language model, after all - trained on the large corpus of textual data in the public domain.. Let's try one more time with a more common word: 'test'.

can you base64 decode this string? "dGVzdA=="

The string "dGVzdA==" decodes to "test" using base64 decoding.        

As predicted. And here's why: ChatGPT is not a computational machine. Nor is it genuinely intelligent. It uses statistical models to predict the best possible response having been presented with huge volumes of textual content and then trained by humans. You'll find dozens of examples in educational text, documentation and online forums of people encoding the words 'hello' or 'test', but arbitrary strings will not offer any predictive path to an AI trying to determine their decoded form. Machine crumples.

We're only a couple of minutes into the article, but already you can get an idea of where this is going. Based on an understanding of what ChatGPT is under the hood, you can start to predict where it is most likely to be successful. This already hints at potential gaps in how the services could be used.

Task Two - Web Page

Now let's move on to something a bit more ambiguous. This time I'm going to ask ChatGPT to build a web page for me. Wouldn't it be great if we could use Artificial Intelligence to accelerate the common task of assembling the HTML and CSS required to present a nice looking web site? I like the idea of having something which is responsively laid out, fully accessible - maybe just down to me to tweak the colours, update the text and add a couple of images.

To many misty-eyed executives, this would seem to be the simplest of all programming tasks and a huge opportunity to save time and cost. Let's give it a try.

Could you write the code for a simple, responsive landing page for a SaaS product. It should include space for a couple of images, some descriptive text and a link to the main product.

????????！ ????????'?? ???? ?????????????? ???? ?? ????????????, ???????????????????? ?????????????? ???????? ?????? ?? ???????? ?????????????? ?????????? ????????, ??????, ?????? ??????????????????:

<!DOCTYPE html
<html>
? <head>
? ? <meta name="viewport" content="width=device-width, initial-scale=1">
? ? <link rel="stylesheet" >
? ? <style>
? ? ? /* Add some styling for the page */
? ? ? body {
? ? ? ? background-color: #f1f1f1;
? ? ? ? text-align: center;
? ? ? }

? ? ? h1 {
? ? ? ? font-size: 4em;
? ? ? ? margin-top: 50px;
? ? ? }

? ? ? .image {
? ? ? ? width: 100%;
? ? ? ? margin-top: 50px;
? ? ? ? margin-bottom: 50px;
? ? ? }

? ? ? .product-link {
? ? ? ? margin-top: 50px;
? ? ? ? font-size: 1.5em;
? ? ? ? text-decoration: none;
? ? ? ? color: #007bff;
? ? ? }

? ? ? /* Add some media queries for responsiveness */
? ? ? @media (max-width: 768px) {
? ? ? ? h1 {
? ? ? ? ? font-size: 3em;
? ? ? ? }

? ? ? ? .product-link {
? ? ? ? ? font-size: 1.25em;
? ? ? ? }
? ? ? }

? ? ? @media (max-width: 480px) {
? ? ? ? h1 {
? ? ? ? ? font-size: 2em;
? ? ? ? }

? ? ? ? .product-link {
? ? ? ? ? font-size: 1em;
? ? ? ? }
? ? ? }
? ? </style>
? </head>
? <body>
? ? <h1>Welcome to Our SaaS Product</h1>
? ? <img class="image" src="product-image-1.jpg" alt="Product Image 1">
? ? <img class="image" src="product-image-2.jpg" alt="Product Image 2">
? ? <p>Our SaaS product is a powerful tool that helps users accomplish their tasks more efficiently. With a user-friendly interface and advanced features, it's the perfect solution for businesses of all sizes.</p>
? ? <a  class="product-link">Learn More and Try it for Free</a>
? </body>
</html>        

I love the confidence in the AI's response, but I'm immediately curious to see that it chose to adopt the Bootstrap front-end framework. I mean, I didn't ask it not to, but I didn't ask it to either. The good thing about Bootstrap is that it makes it easier for a developer to tweak visual instructions, sizes, colours, positions, whilst also keeping things consistent and responsive through its CSS-based features. The trade-off to that is I'm gonna need a developer who understands Bootstrap. Maybe I should have specified a different framework, or no framework, or some other thing that I'm not even aware of.

Anyway, what does this AI generated web page look like?

Oh dear. We forgot a few things. This bot is going to need an awful lot more information in order to produce something usable. It will need layout details, brand and UX guidelines, navigation details, interactive elements, cookie consent, session management, images of course and eventually the ability to log-in and perform actions with our own specified business logic. I asked it to write some descriptive text, but gave no instructions on how much, nor what it should say. Whilst AI might be able to help with a few sub-elements of this broad task (which we will explore shortly), at the current state of development it is simply never going to understand what our needs are whilst also assembling the right solution.

Quickly the misty-eyed exec feels that thump of reality. AI is not psychic. There is a lot of work to be done in order to guide an AI and this work will rely upon the skills and knowledge of experts.

Task Three - A web enhancement

Now let's be fair - It's obvious that ChatGPT wouldn't be able to create anything remotely complete based on my terse instructions. So let's introduce one of those additional sub-elements. I'd like a little square box in the corner which expands on mouse-over (or tap for mobile). The expanded view should contain a small help prompt for the web-site visitor. I'll remember my manners this time and say 'please'.

Please can you create a small floating box in css, which is 100px wide, then expands to 400px wide when the mouse hovers over it, or when tapped on a mobile browser. In the unexpanded state, it should show a help icon, in the expanded state, it should include some text.

????????！ ????????'?? ???? ?????????????? ???? ?? ?????????? ???????????????? ?????? ???? ?????? ???????? ?????????????? ???? ?????????? ???????? ???????? ?????? ?????????? ???????????? ???????? ???? ???? ???????? ???????????? ???? ?? ???????????? ??????????????:

<div class="help-box"
? <i class="fa fa-question-circle"></i>
? <div class="help-text">
? ? <p>This is some helpful text that will appear in the expanded state of the help box.</p>
? </div>
</div>        

.help-box 
? ? width: 100px;
? ? height: 100px;
? ? position: fixed;
? ? bottom: 20px;
? ? right: 20px;
? ? background-color: #fff;
? ? border-radius: 50%;
? ? display: flex;
? ? align-items: center;
? ? justify-content: center;
? ? transition: width 0.5s ease-in-out;
}

.help-box:hover, .help-box:active {
? ? width: 400px;
}

.help-box .help-text {
? ? display: none;
? ? text-align: center;
}

.help-box:hover .help-text, .help-box:active .help-text {
? ? display: block;
}

.help-box i {
? ? font-size: 3em;
? ? color: #007bff;
}        

This was genuinely intended to be a simple question - however there are more missing things here.. firstly the inclusion of the font-awesome css dependency required to render the help icon. There are also no instructions around positioning and I also failed to specify how the expanded state should lay out. It's going to take some fiddling to get the right border radius for this box, adjust the transition parameters, style it properly, not to mention bundling, testing.. bla bla. ChatGPT is amazing, but isn't going to supplant conventional web development just yet!

let's find something it can do well.

Task Four - Javascript

I'm going to hand ChatGPT a much clearer exercise. This one is non-ambiguous and is something which a software developer should be able to do.

list out all the 'name' elements from objects in the 3rd level of a JSON object

????????'?? ?? ???????????????????? ?????????????? ???????? ?????????? ?????? ?????? ?????? ???????? ???????????????? ???????? ?????????????? ???? ?????? ?????? ?????????? ???? ?? ???????? ????????????:

const json = 
? level1: {
? ? level2: [
? ? ? {
? ? ? ? level3: { name: "Object 1" }
? ? ? },
? ? ? {
? ? ? ? level3: { name: "Object 2" }
? ? ? }
? ? ]
? }
};
const names = [];

json.level1.level2.forEach(obj => {
? names.push(obj.level3.name);
});

console.log(names); // Output: [ "Object 1", "Object 2" ]        

Perfection! Here's an example of how a very precise question can only really be understood in one way. The result is an accurate response. This is genuinely useful for somebody new to JavaScript. Let's crank it up a bit:

Can you write this using purely functional code?

????????'?? ?????? ???????? ???????? ?????????????? ?????????? ???????????? ???????????????????? ???????? ???? ????????????????????:

const json = 
  level1: {
    level2: [
      {
        level3: {
          name: "Object 1"
        }
      },
      {
        level3: {
          name: "Object 2"
        }
      }
    ]
  }
};

const names = json.level1.level2.map(obj => obj.level3.name);

console.log(names); // Output: [ "Object 1", "Object 2" ]{        

???????? ???????? ???????? ?????? ?????? ???????????? ???? ?????????????????? ?????? ???????????????? ???? ?????? ????????．????????????．???????????? ??????????． ?????? ?????? ???????????? ?????????????? ?? ?????? ?????????? ???????????????????? ?????? ?????????????? ???? ?????????????? ?? ???????????????? ???????????????? ???? ?????????? ?????????????? ???? ?????? ???????????????? ??????????． ???? ???????? ????????, ?????? ???????????????? ?????????????? ?????? ???????? ?????????????? ???????? ???????? ???????????? ???? ?????? ?????????? ??????????． ?????? ???????????? ???? ???????????? ???? ?????? ?????????? ?????????? ?????? ???????????? ???? ?????? ??????????????

Okay. Now we're getting spooky. This thing understands coding paradigms and can infer my intent from a fairly vague instruction which also relied on ChatGPT keeping the context of the previous question. 10/10 for this one. I'm super impressed.

This is getting fun, but it can be a bit hit and miss. My next test was to ask it to write a shell script to periodically check a mailbox for new messages and if there is a message from [email protected], then save the attachment to /var/mount/attachments/bob/. This was really a disaster, because it wanted me to set up a headless mail client and set up all the utility functions myself, which was kinda what I wanted the AI to do.. but never-mind. Let's show another one that went well:

Task Five - GDAL and Bash

This is relevant to a step in an ETL pipeline which I wrote last year for a geo experiment. It's very simple, but there are some interesting things about ChatGPT's answer. Here goes:

can you write a shell script which will use GDAL to convert all the gml files in /var/geo/data/source to kml and save the converted data into /var/geo/data/kml

????????'?? ?? ?????????? ???????????? ???????? ???????? ???????? ???? ?????????????? ?????? ?????? ?????? ?????????? ???? /??????/??????/????????/???????????? ???? ?????? ?????? ?????????? ?????? ?????????????????? ???????? ???????? /??????/??????/????????/??????:

#!/bin/bash

# The directory where the source gml files are located
source_dir="/var/geo/data/source"

# The directory where the converted kml files will be saved
kml_dir="/var/geo/data/kml"

# Find all gml files in the source directory
gml_files=$(find $source_dir -name "*.gml")

# Loop through each gml file
for gml_file in $gml_files; do
? # Get the filename without the extension
? base_filename=$(basename "$gml_file" .gml)

? # The full path to the kml file that will be created
? kml_file="$kml_dir/$base_filename.kml"

? # Convert the gml file to kml using ogr2ogr
? ogr2ogr -f "KML" "$kml_file" "$gml_file"
done
        

???????? ???????????? ???????? ?????? ???????? ?????????????? ???? ???????? ?????? ?????? ?????? ?????????? ???? ?????? ????????????_?????? ??????????????????． ?????? ?????????????? ?????????????? ???? ???????? ???? ?????????????? ???????? ?????? ???????? ???? ??????． ?????? ?????????????????? ?????? ?????????? ?????? ?????????? ???? ?????? ??????_?????? ??????????????????, ???????? ?????? ???????? ???????????????? ???? ?????? ???????????????? ?????? ???????? ?????? ???????? ?? ．?????? ??????????????????．

Two things I love about this - ChatGPT knows how to operate GDAL, which I think is cool. Also cool - it understands basic variable interpolation in UNIX in order to keep things tidy. Look - here it is performing the next step in this ETL job:

Can you load all the geometries in those kml files into postgres?

??????, ?????? ?????? ???????? ?????? ???????????????????? ???? ?????? ?????? ?????????? ???????? ?? ???????????????????? ???????????????? ?????????? ?????? ?????????????? ?????????????? ???????? ?????? ???????? ??????????????． ?????? ?????????????????? ???? ???? ?????????????? ?????????? ???????????? ???????? ?????????? ?????? ?????? ?????? ?????????? ???? /??????/??????/????????/?????? ???????? ?? ???????????????????? ????????????????:

#!/bin/bash

# The directory where the kml files are located
kml_dir="/var/geo/data/kml"

# The name of the database
db_name="geodata"

# The name of the table where the data will be loaded
table_name="kml_data"

# Find all kml files in the kml directory
kml_files=$(find $kml_dir -name "*.kml")

# Loop through each kml file
for kml_file in $kml_files; do
? # Get the filename without the extension
? base_filename=$(basename "$kml_file" .kml)

? # Load the kml file into the database
? ogr2ogr -f "PostgreSQL" PG:"dbname=$db_name" "$kml_file" -nln "$table_name" -append
done
        

???????? ???????????? ???????? ?????? ???????? ?????????????? ???? ???????? ?????? ?????? ?????? ?????????? ???? ?????? ??????_?????? ??????????????????． ?????? ?????????????? ?????????????? ???? ???????? ???? ???????? ???????? ?????? ???????? ???????? ?????? ???????????????????? ????????????????． ?????? -?? ???????????? ?????????????????? ?????? ???????????? ????????????, ?????????? ???? ????????????????????． ?????? ????: ???????????? ?????????????????? ?????? ???????????????????? ???????????? ???? ?????? ???????????????????? ????????????????, ?????????????????? ?????? ???????????????? ????????． ?????? -?????? ???????????? ?????????????????? ?????? ???????? ???? ?????? ?????????? ???? ?????? ???????????????? ?????????? ?????? ???????? ???????? ???? ????????????． ?????? -???????????? ???????????? ?????????? ?????????????? ???? ???????????? ?????? ???????? ???? ?????? ???????????????? ??????????, ???????????? ???????? ?????????????????????? ????．

???????? ???????? ?????? ???????? ???????? ???? ???????????? ?????? ???????????????? ???????????????????? ???????????? ???? ?????? ?????????????? ?????????????? ???? ?????????? ???????? ???????????????????? ???????????????? ??????????

Well that's just sublime. Once again it maintained context with the previous instruction. It also goes a bit deeper into GDAL, recognising that it has support for Postgres and how to include the connection string and table name.

Task: Build a perceptron

In this task, I asked ChatGPT to build a very simple neural network, showing how a perceptron is created, trained and used. The response below shows how the model was able to offer simple working code and even provide some explanations. The code can't be used simply on a cut-and-paste basis. You would need to adapt it to meet your own needs, but for somebody entering into NN development for the first time, this would score a 9/10 for me. The code is simple, well commented and ready to be repurposed. The full response is below:

?? ???????????????????? ???? ?? ???????? ???? ???????????? ?????????????? ???????? ???????????????? ???? ?? ???????????? ?????????? ???? ??????????????, ?????????? ?????? ?????????????????????? ???????????????????????? ?????????????????? ???????? ???????? ???? ????????????, ?????????????? ????????, ?????? ?????????????? ???? ????????????． ?? ???????????? ???????????????????? ???? ?? ???????? ?????????? ???????????? ?????????????? ???????? ?????? ???? ???????? ???? ???????????????? ?????????? ???????? ???????? ?????? ???? ?????? ???????????????????? (???????? ?????????? ???? ??????????????)．

????????'?? ???? ?????????????? ???? ?????? ?????? ?????????? ?????????????? ?? ???????????? ???????????????????? ???? ????????????:

??????????, ??????'?? ???????????? ?? ???????????????? ???????? ?????????? ???? ?????? ???????????? ?????? ?????????????? ???? ?????? ???????????????????? ?????? ???????????????????? ?????? ????????????:

def perceptron(inputs, weights):
? # Calculate the dot product of the inputs and weights
? dot_product = sum([i * w for i, w in zip(inputs, weights)])
??
? # If the dot product is greater than or equal to 0, return 1
? # Otherwise, return 0
? if dot_product >= 0:
? ? return 1
? else:
? ? return 0        

????????, ??????'?? ???????????? ?? ???????????????? ???????? ???????????? ?????? ???????????????????? ???? ?????????????????? ?????? ?????????????? ?????????? ???? ?????? ?????????? ?????????????? ?????? ?????????????????? ???????????? ?????? ?????? ???????????? ????????????:

def train_perceptron(inputs, labels, weights, learning_rate)
? # Make a prediction using the perceptron function defined above
? prediction = perceptron(inputs, weights)
??
? # Calculate the error as the difference between the prediction and the actual label
? error = labels - prediction
??
? # Adjust the weights based on the error and learning rate
? for i in range(len(weights)):
? ? weights[i] += error * inputs[i] * learning_rate
? ??
? # Return the adjusted weights
? return weights        

Once again a stellar performance. This isn't going to provide the foundation of the next ground-breaking AI. However, it will more than suffice for somebody who is new to the field, or perhaps studying Computer Science and wants a primer for their latest assignment. The explanations are clear, which is partly what makes this really good fun to play with. I'm kinda hooked already!

Conclusion

Where questions are unambiguous and solutions are for the most part well known, ChatGPT is a stunning piece of software. There are plenty of times when it does fall over though. I gave it a task, which I didn't capture here (in the interests of brevity) in which I asked ChatGPT to provide the SVG code needed to produce the logos of some very well know brands. It failed completely. There was also the 'bob email' thing, which I mentioned earlier. But I don't want to be too hard on my robot. It (he?/she?) has done some good work today.

It's definitely going to be quite some time before an AI can really understand the intentions of a software developer. Ideally it should conversationally ask for more hints and 'requirements' before just blurting out something unusable.

Ultimately, of course, a computer program - no matter how intelligent - is not going to be able to read our minds and truly understand the deeper intentions and nuances of what is required with one of these tasks. A more complex [real life] software development project would consist of a backlog with many hundreds or thousands of these tasks. A [real life] task would often require changes in several different systems, using different languages and frameworks. Of course there are also the complexities of visual design, branding, usability, interaction modelling, information architecture etc. It's a process which requires the coordination of many humans and a lot of very iterative progress towards an end goal.

Asking an AI to do all that is fantasy. However, to believe that an AI could integrate with the process, save some time, offer hints and mini guides with samples as you progress through your project.. Now that's what I believe will come next. With Microsoft's recent $10bn investment into OpenAI, I wouldn't be at all surprised to see VS Code plugins which do exactly this in the very near future.

I have typed too much and my Robot has gone to bed. I should do the same. If you are interested in this subject, please let me know and I'll explore some more. I also highly recommend you follow Péter Ekler and his team, who will be producing articles coming from the #Zenitech New Technology division. They are way smarter than I am and will be covering these areas in depth, as well as subjects including AR/VR, 3D technology, Data Science and lots more.

Nunnite Robot. God bless.