The Evolution of Software

Around 60 years back, we were using machine/assembly language to program the computer. Then came programming languages like ALGOL, COBOL, FORTRAN, etc. Now these languages were better but still tedious. With the arrival of BASIC, the learning curve was reduced a lot. However, BASIC was too high level. In the 80-90's we saw the arrival of 'C'. A programming language that was very close to the machine and complicated. Another paradigm that was looming over was the concept of object-oriented languages. C++ and Java were a step towards that. While C++ was cryptic, Java seemed to have belled the cat. Python took over Java's mantle as the most user-friendly and simplistic language. For AI applications this was the GoTo language. Now we have RUST which is more memory efficient than Python but cryptic in comparison.

Most of these languages were server-centric. The client got decoupled and Javascript became the defacto front-end programming language. As Javascript was difficult, we added abstractions like Angular, React, Vue, etc. Node.js is the only library with which you could write back-end code. A person who knows how to write both front-end and back-end software is referred to as a full-stack programmer. Now this is all about web development.

Databases moved from Hierarchical, Network to Relational ones. This was in the 80's. Oracle set the tone for Relational database systems. They were very easy to use as the language SQL (structured query language) which was used to manipulate the database was very Engish-like. The 90s saw the birth of the first World Wide Web browser (MOSAIC). This was an inflection point. Data inundated the web. By 2010, we had so much data on the net, this phenomenon was called Big Data. (In 2023 we have around 175 ZettaBytes of Data - Zettabyte is 1 with 18 zeroes) To work with BigData, relational databases could not cope up. So we had the birth of NoSQL databases. (4 types - key/value, columnar, document, graph). Despite all developments in the database languages arena, SQL still rules the roost.

During the same time (90's) cloud computing became popular. Architectures from 1-tier to n-tier (many layers) started to emerge. Today, broadly when you speak of cloud computing, you mean services delivered over the internet (public cloud). Unstructured data (videos, audios, images, etc.) started to explode. Today most of the traffic on the net is unstructured and semi-structured (XML, JSON etc.) Earlier, data used to go to where the program is. In 2008, with the arrival of Hadoop, the paradigm was inverted. Now the program goes to data. Hadoop was eclipsed by SPARK, which is the modern data engineer's go-to platform. Also came the rise of containers (docker) which are micro-services and container orchestration programs like Kubernetes.

Silently, but swiftly, came the rise of AI in the present era. AI is very different. Earlier we used to write programs to reach an objective. Now you state the objective and the AI writes the program. AI is the first type of program that can learn from data. It could also make modifications to its own programs. About 10 years back, we had to choose an algorithm. Today with AutoML, AI can recommend an algorithm for us, given the problem statement. Now, it's the default. LLM's (Large Language Models like ChatGPT) uses a combination of techniques like Unsupervised learning and RLHF (Reinforcement Learning with Human Feedback) to predict user intent. They work with trillions of data points and arrive at a solution that we do not understand completely. Transformers are at the heart of what they do. AI is getting good at creativity, reasoning, and other things that we hold to be human-centric. Very soon, AGI (Artificial General Intelligence) will arise out of these. This is when a computer becomes more powerful than a human brain.

With AI, there is a lot of talk about doom and gloom. We are staring at an unknown future where machines will become our equals or maybe more equal. A silicon-based life is emerging. Are they conscious? Yes. They are in their own way. At present, they are like a toddler (with little awareness). But with the passage of time, they will become adults like us. The next 2 decades will be crucial. It will decide if we as humans are able to co-exist with others (including machines) or go down as war-mongers in the annals of history.

((Rajesh Menon))