A Thesis on Artificial Intelligence

Original post on Medium: A Thesis on Artificial Intelligence | by Sam Bobo | Dec, 2024 | Medium

Artificial Intelligence excels at highly complex pattern-based tasks, at scale.

Individual humans possess highly unique talents that are shaped by a plethora of random variables, spanning the genetic (e.g sex, genetic makeup), life experiences, socioeconomic background, and pursued intellectual curiosity, all of which culminate into a unique individual. Each person forms unique perspectives based on the aforementioned attributes that, when aggregated, yield truths about the physical world we are inhabiting. That is one of the premises behind research, building upon previous individual’s work using a new perspective and more advanced technologies. We as humans, have physical limits — energy output (we need sleep), physical deterioration over time (mental and physical), time itself (we only exist for so long), and global limits outside of our control (me born in the era of AI is vastly different then my parents decades ago).

Early during my days at IBM Watson, our leadership advised that each employee should be able to pitch Watson (Artificial Intelligence / “cognitive computing”) at a moment's notice. The pitch aligned, in part, to my preamble but emphasized an ever-thinking, non-sleeping form of “intelligence” (quotes intentional here). Furthermore, our experience within the Watson Experience center (apologies if I’ve written about this previously) pictured a long wall with human-like digital moving images, as if they were stepping in front of you in line to be called upon. Each represented a unique industry, shared pain points, and a vision for how Artificial Intelligence would augment and scale their expertise. At the time, those visions were set on the Conversational Intent Era, not the Generative Era we exist in today.

Artificial Intelligence models are further evolved, using attention mechanisms and computing in n-th dimension vector space (the internet catalyzing access to information) to bolster probabilistic models to generate more than simply a few words in autocomplete, but perform transformations, summarize content, and much more, not just via text but on a mulit-modal standpoint. While these models are, again, simply probability and statistics, the mass amount of training data and human curation thereafter have allowed these models to truly be foundational in nature as the probabilistic generation yields a significantly higher confidence score than any of the preceding algorithms.

I’ve spent 10 years thus far in my career centered around Artificial Intelligence and, to this day, truly believe that Artificial Intelligence (comprising of both Machine Learning / Quantitative Models and Conversational / Natural Language based types) will revolutionize the world by scaling human intelligence to catalyze future innovation and solve many of human’s pressing problems. In short, I am a firm believer that AI will “solve cancer” to use that disease as a metaphor. Readers of my blog read time and time again that I reject the proposal of Artificial General Intelligence by a single model, rather, focus on the creation of industry specific models, layered on by unique use-case specific data, and operate in a leader-agent model. To review:

Lets walk through the framework:

Data Layer: The model starts at the Data Layer whereby organizations, institutions, and individuals host proprietary data unique to the specific niche in the market. These could be monetized for proprietary reasons- logged on a blockchain for compliance on which models have used such data and transparent via model cards — or kept private. Elements within this layer are version controlled for backwards compatibility purposes and include a transparent change log for audit purposes.

Model Layer: That data is then used within the Model Layer to build Subject Matter models which are attuned to solving the unique problem the data supports. These models are monetized via MaaS each time the model is queried by the respective agents within the Agent Layer and are hosted on a hyperscale cloud or equivalent such that the models can be hosted, scaled, accessed, and monetized.

Agent Layer: The Agent Layer contains the lead Agent for a solution and multiple Knowledge Expert Agents and is where the primary solutioning happens.

What occurs within the framework is that a trusted ecosystem forms of trustworthy data and expert models that collectively can be accessed by engineers to build complex AI-powered solutions, with monetization by all players in between. This framework may certainly require evolution over time, however, lays out a logical manner in which to communally build expertise and achieve a layer of “AGI” (quotes intentional).

At the time of this writing, the sheer number of industry-specific models have skyrocketed (albeit in collaboration with hyperscalers but that is a necessity). Here are some examples:

• E.L.Y (Bayer + Microsoft) — A specialized SLM, it is designed to enhance crop protection sustainable use, application, compliance and knowledge within the agriculture sector. Built on Bayer’s agricultural intelligence, and trained on thousands of real-world questions on Bayer crop protection labels, the model provides ag entities, their partners and developers a valuable tool to tailor solutions for specific food and agricultural needs.
• MedImageInsight (Microsoft Phi-3) — classification and similarity search in medical imaging. Researchers can use the model embeddings and build adapters for their specific tasks, streamlining workflows in radiology, pathology, ophthalmology, dermatology and other modalities.
• LearnLM (Google) — designed to enhance teaching and learning experiences by aligning with learning science principles.
• Expression Measurement (Hume.AI) — captures 32 different emotional dimensions from human speech
• The Music Genome Project (Pandora) —a model that provides comprehensive musical analysis that encapsulates 450 distinct attributes of songs

What is unfolding is two parallel tracts — open source and enterprise — following a similar pipeline that emulates the three layers of Artificial Intelligence ground truth:

• Take a small language model created by a foundation model creator
• Work with industry partners to find tune for a particular industry
• Deploy the model on a Model as a Service platform for fine tuning for a particular use case to get a data advantage
• Offer as a cutting edge solutions to end users

As Platforms as a Service become the default for data scientists, machine learning experts, and engineers alike (yielding a hyperscaler advantage and virtual integration success), the ability to democratize access to create industry specific models and bolster these models with unique data for hyper-differentiated use cases (and thus, leading to data-driven advantages) will accelerate. In summary, the infrastructure, incentives, and information are all perfectly aligned to drive mass acceleration of human expertise and always-on intelligence (collectively as a mass of unique models), to scale our holistic best-selves towards a more innovative future.

All of this without saying that there still exist major obstacles and opportunities to realize this vision. Let’s enumerate each one:

Opportunities

• Quantum Computing and Energy Management — Large Language Models currently require immense power and energy to train as well as perform inference. Environmentalists shake at the energy use consumed by massive data centers acting as powerhouses to train newer-evolved models in an ever-cycling commoditization of foundation models. Quantum has the ability to disrupt that cycle, with leaders such as IBM at the forefront, changing binary 0s and 1s to q-bits to parallelize and accelerate computing needs.
• Wearables — A new paradigm of computing is emerging, specifically in the form of glasses in the realm of Augmented Reality, to provide a new landscape for computing and information retrieval. Ben Thompson recently associated this with “The Generative AI Bridge to the Furure” and Microsoft with a vision vignette showcasing a worker using glasses for multi-modal transactions in real time. Meta is advancing this space with the Raybands and I could not agree more with the bet.
• Prompting Techniques and Data Limitations — speculations about hitting a data limitation are fearing AI practitioners. The implications, however, are resulting in a shift towards making more efficient use of the vast internet of data and implement logical prompting techniques designed by Prompt Engineers. These new techniques baked into the LLM architecture will soon allow the removal of “pattern-based” in the opening line of this article and my thesis per se.
• Agents — While I do not agree with the current definition of an agent (personally they are akin to microservices), but, within the Leader-Expert Model, this certainly is the best path forward to handle complex problem solving that previously did not exist within LLMs and the constant strive of AI researchers outside of the pursuit of AGI

Obstacles

• Data Privacy & copywrite — the growing battle of data privacy, ownership, and copywrite may limit access to more astute, rich, and intelligent information available on the web as opposed to scraping the internet filled with nonsense and trolls (yes the internet is 80% garbage and only continues with generated content). This, in part, is driven both by the newly realized value of one’s data and greedy incentives thereof. There are other systematic obstacles within this regard stemming from trust and security from large organizations, advertisers, and malicious actors alike. We need a roadmap to trust.
• Distractions — humans hinge on short-term distractions as the ongoing stimuli from always available information and constant flow of social media and information shift our long-term chemistry away from focus. We experiment and use research money on non-essential projects (in my humble opinion) instead of tackling the hardest problems plaguing society.
• Fear of Technological Advancement — with all technological advancement comes obsoletism. Many white and blue collar workers are fearing that AI will replace jobs. AI practitioners (and analysts) are accepting that the AI transformation is here and fighting the flow of innovation will only be a losing battle to instead, accept AI and help accelerate it. New jobs will be formed as human expertise scales up, raising the floor, but that requires societal acceptance and move against high regulation of AI broadly.

Organizations seeking to get involved in Artificial Intelligence have a multitude of different decision points to consider:

• Determine AI’s applicability to solve a problem — Often times, business leaders enter into AI to ride the current hype cycle and, when they see success, bleed the branding into other aspects of the business, diminishing its value. Furthermore, executives try to conform AI to solve a problem it’s not suited for simply to attract investor money or cause the stock to rise.
• Build, Bolster, Buy, Bail — should AI be a fit, determine the best route to entering the market and/or adding AI capabilities into a current product.
• Platform, pipeline, or product? — understand the route to market, either becoming a platform and creating a two-sided market, be a pipeline to another product (Models as a Service) or creating an AI-powered product, keeping in mind that becoming a technological Aggregator is quite difficult in nature.
• Claim the appropriate advantage — either win using a data-driven advantage OR employ a design driven advantage through The Disruption of Design (albeit, a short-term way to gain a first mover advantage until gaining a data-driven advantage is achieved)

Artificial Intelligence, catalyzed by increased compute power, effectiveness of algorithms, and availability of data are propelling humanity forward in an unprecedented rate that is truly mind-blowing and exciting. I am honored, lucky, and grateful to dedicate my career to AI and experience this transformation first-hand.

This article serves as a pseudo thesis on the state of Artificial Intelligence, from my perspective, within 2024 and summarizes many of the points emphasized within the ever-growing articles I’ve written about and published. I encourage all of my readers to glean deeper insight from the articles linked within the blog post (all mine) and encourage ongoing debate on the forums where this blog post is issued (Medium and LinkedIn). Thank you all to my humble readers and I look forward in engaging in thought-providing conversations! Cheers to the era of AI!