Perspectives on Building Teams to Make Software Intelligent

Ever since the Chat GPT’s/LLMs success has made it to the headlines; AI/GenAI has ubiquitously been a household name and the most sellable intellectual property in the software world across all industry verticals. This article details a few bottom-up perspectives on building development teams to make software intelligent, while trying to keep the jargon to a minimum.

?Always start with the Mathematical Language

In GenAI the most prevalent use cases are for the LLMs wherein, ‘predicting the next word’ is the most fundamental problem. To solve this in most of the LLMs, the words or parts of the words in the text is mathematically represented as token. Set of tokens make vector from linear algebra. This ‘vector’ is a single column matrix with multiple rows or can be a matrix with a single row and multiple columns.? Like any software program the mathematics here too, take an input, processes it, and generate an output. Putting all these three together as vectors,

• a vector ‘i’ for input
• a vector ‘h’ representing intermediary stages and are called as hidden layers.
• a vector ‘o’ for output

A matrix with vectors ‘i’ and ‘h’ make a very basic neural network, while the matrix having all three of ‘i’, ‘h’ and ‘o’ would make a ‘transformer’ neural network. Adding more ‘h’s would result in a multi-layer neural network. ChatGPT was architected with these transformers. When the idea behind ChatGPT was proposed in Google Research’s landmark paper “Attention Is All You Need” in 2017, the transformer has 6 hidden layers or ‘h’ vectors, and the current version ChatGPT-4 has 96 hidden layers also to mention a tensor is a multi-dimensional matrix from which Google has derived the name TensorFlow. And then there are varieties of neural networks each serving a specific use case such as recurring neural networks (RNN) for text, convolution neural networks (CNN) for images etc.

Today, there are plenty of programming languages with an abundance of APIs. However, most of the available documentation is all about using functions and methods and has barely any details about the underlying linear algebra and related mathematics even as a pseudocode. The prevailing phenomenon in the industry is that the programmers who code with the existing APIs pass as Data Scientists, Machine Learning Engineers in-spite of being oblivious to the underlying linear algebra and related mathematical logic. This might yield revenue for the businesses for a short run while simultaneously sidelining the innovation quotient. It’s important to realize that the mathematics and logic behind the code is what makes the software and services sellable.

Organizations must hire fresh graduates with advanced degrees in Mathematics, Linear Algebra and statistics and start investing in training them with the needed programming skills. Now it's an imperative for the hiring managers to go thru the academic transcripts and check for Linear Algebra and related coursework. Promoting mathematical rigor and in development teams and stepping up the R&D spend on researching new algorithms is the way forward to gain a competitive advantage in this emerging market of AI/Gen AI.

Foster Tool and Technology Agnosticism

Years ago, when Big Data paradigm boomed, the industry was confronted with the limitations of Java in handling large data sets. Addressing this challenge, the industry and community has created a brand-new ecosystem with extensive libraries which were built by leveraging Python/Cython’s flexibility in wrapping independent non-python code into python-importable modules. To cite a few examples; python libraries like NumPy, SciPy wrap C/C++ code while importing their libraries for vectorized operations, and PySpark wraps Scala code in its imports.

Fast forward to 2023, a new programming language ‘Julia’ has emerged as one of the most suited languages for numerical computations such as solving differential equations for simulation and is being sought-after as a backend for Python and R. This is an ever changing landscape, programming skills alone would not cut the bill for the marketability of individual developers and the call for the depth of their understanding in science behind the software is louder than ever.

In the recent years industry has witnessed a steep rise in the success of software when it is closely coupled with hardware, for example Google’s Tensorflow and Meta’s PyTorch heavily relied on Nvidia’s CUDA APIs for achieving performance by GPU acceleration. More recently this segment of the industry has been rallying along with the more recent Nvidia’s Tensor core processors and their TensorRT APIs.

Furthermore, all major public cloud service providers have started to provide tools for machine learning operations (MLOps) which has veiled the mathematical rationale behind the ‘learning’ and ‘training’ aspects of AI/ML applications to the convenience of the developers making the developer community more and more dependent of their ecosystems.

In directly the prevailing phenomenon will make the skill set of workforces be limited to the technology environment they’re exposed to and many a times developers especially in service industry end up being out skilled as the tools and technology keep changing.

Organizations must hone and foster a culture of technology agnosticism in their human capital. Emphasizing and training their development teams with core computer sciences, computational theory, compiler construction, programming language theory etc. to esprit research and development. This would enable them to look outside the box of existing COTS/Tools to come up with home-gown intellectual assets in providing services to the customer and deliver innovation. Now there is a fortiori for service industry to come up with platform independent AI/GenAI solutions and widen their business appeal. The bang to the buck lies in patent sharing with customers, thus creating a mutual competitive advantage thru research and innovation.

Encourage Open-Source Culture and Contributions

Form the purview of organizations, there can’t be a more honest display of capabilities than publishing open-source repositories. Open-source repos would surely serve as utmost transparent marketing collaterals in attracting potential customers.

Form the purview of individual developers an open-source contributions provide a challenging learning experience. These contributions would greatly increase the marketability of the human capital in the service industry. The leadership in service industry should encourage and motivate their development teams to be making more open-source contributions in AI/ML/GenAI areas.

Importantly, currently AI/GenAI is one of the most seriously regulated technology. Over the last few months regulatory authorities of Bharat, USA, EU Parliament, have begun fixing heavy guardrails around AI/GenAI service providers. The most common clauses in these regulations include sharing the data sets, biases, and results of the AI/GenAI services with the local regulatory authorities before going to the market. By and large, contemplating business strategies and taking open-source licensing models into account is an inescapable reality.

Organizations now have need to educate their development teams about doing open-source contributions and maintaining their homegrown open-source repositories. Secondly this calls for increasing the awareness towards regulatory compliance and re-analyzing beyond the standard 3 & 4 clause BSD and MIT licensing models for the success of business.

Futuristic perspectives on the role of AI/ML in Networking and CSPs

While the industry is undergoing its adoption to the cloud nativeness of secure access service edge (SASE) paradigm which has coupled the SD-WAN and security functions together, Programming Protocol-independent Packet Processors (P4) and P4 programming has begun its disruption already. Major chip manufacturers and network OEMs have been investing in P4 research and development. Recent products in this segment include Intel's Tofino Ethernet switch and P4 Studio, Cisco’s Silicon One Q211 processor etc.

Traditional SASE takes away all/most of the functionalities from the network elements such as switches, routers, gateways etc. to the cloud, while the P4 does the exact opposite by bringing evermore functionality to the devices in the data plane. Although the current research and development in P4- neural network switches are still in a nascent stage this trend of deep data plane programming with AI/ML techniques for zero-trust, security etc. will continue to raise. One of the P4 use cases with a very high monetizable potential is P4 driven traffic isolation for archiving network slicing. The steep rise in the industry participation in the P4 Consortium and it’s SwitchML open-source project is a harbinger for a major disruption to the ways in which CSPs and Telcos are currently leveraging the network for new features, improving customer experience, and growing their business in the overall.

While the traditional AI/GenAI use cases such as network operations, security, churn analytics, service delivery, customer facing and interacting etc. in TSPs and CSPs have their place; the P4 powered in-network machine learning and distributed-models for AI/ML applications are bound to play a crucial role is 5GB/6G networks. Service and consulting organizations now have a more than ever need to make thicker industry collaborations to grow their intellectual property portfolios. This is one segment where their capital investments in research and development will surely yield results in the long-term.

Conclusion

The author believes that AI/GenAI is inherently an opportunity to create business opportunities and never seen before use cases. One such example is the NVIDIA’s Quantum Cloud and it’s new open-source CUDA-Q quantum computing platform with over 160+ partners in their ecosystem, wherein NVIDIA’s Hopper GPUs are connected to quantum processors with PICe interfaces. The programming interfaces to this hardware include NVIDIA’s cuQuantum, cuStateVec, cuTensorNet; Google’s Cirq etc. These programming interfaces and SDKs were built on the same mathematical techniques behind GenAI such as LLMs, Linear Algebra etc. to simulate quantum circuits. There will always be a vacuum where the existing software will short of leveraging the latest breakthroughs in hardware. Service organizations need to focus on building and hiring teams with right skillsets and capabilities to come up with malleable and platform independent software solutions to capture these new opportunities.

The unfathomable transmutability of the intellectual property related to GenAI, Linear Algebra and LLMs and new algorithms etc. are screaming loud and clear about co-patenting and IP sharing agreements with the customers as the need of the hour for securing revenue for service businesses in the long run. Business Developers with an unborrowed vision need to metamorphosize into Business Vikings. The GenAI business conquest is a bottom-up process that begins with nurturing the teams with needed skills and rallying them towards accomplishing business objectives. There is money in the market and now is the time to blow the war horns and start raiding the market for business.?