6 ways AI can progress in 2023

1) Bigger Windows: Hitting Limits?

One approach is expanding the size of neural network windows, such as GPT-4's current 32k limit.

However, the facts suggest that increasing window size doesn't always lead to improved performance. Consequently, this method may not be viable until new solutions are discovered. Most developers are still prefering a mix of LLM+Vector embeddings.

2) More Modalities

Training large language models (LLMs) on text and images together yields better results than focusing on either modality independently.

Adding other sensory inputs like sound or video could further enhance performance and enable more advanced user experiences.

Imagine an LLM equipped with computer vision that can interact with your desktop screen, understand its contents, and perform actions using mouse and keyboard input - just like a human!

This breakthrough would eliminate the need for direct AI integration into products since any existing tool could benefit from these versatile capabilities.

Basically AI would be your true copilot, even for offline world, since I could see the world via video camera.

We are years away regarding performance requirements to accomplish this, but it works in the "lab", so it most likely will work outside the lab as well.

3) Nested AI Agents

The concept of nested agents involves creating hierarchies within AI systems similar to organizational structures found in businesses - CEO down through management teams and employees. Each agent specializes in specific tasks while delegating others among their peers or subordinates.

People have already employed these autonomous agents with the aim of generating income, yielding some impressive outcomes. With existing evidence supporting this model, I personally anticipate even more progress in this approach within the current year.

4) Deeper/Different Learning Architectures

Another approach involves developing deeper LLMs by going beyond traditional neural networks' standard algorithms. Many believe GPT-4 outperforms its predecessor not because of increased parameters, but due to its more profound architecture and the integration of advanced algorithms within the network itself.

Many intriguing research papers propose groundbreaking architectural ideas in AI. However, many remain theoretical.

In the coming months, we'll see if these concepts can be effectively implemented.

5) Domain-Specific Expertise

Large organizations invest in training LLMs for specific domains like finance or medicine.

While these specialized models can be helpful, they often don't surpass the capabilities of general-purpose LLMs like GPT-4 when it comes to providing accurate results.

As multimodal generic models continue to improve, it's expected that they'll cover a wide range of domain-specific tasks as well.

So the whole field of domain based training isn't more promising than it was a year ago, actually it's less promising today. However new training techniques are being developed and perhaps we will see better results here.

6) Fine-tuning Existing Models

Initially, fine-tuning LLMs was expected to deliver great results for custom needs. However, reality proved different as models lost generic capabilities post-fine-tuning, with vast data requirements for improvements. Currently, prompt engineering outperforms fine-tuning in most cases – an observation shared by us at MarsX.dev and out network.

Follow me on twitter to read more about AI and DevTools: https://twitter.com/johnrushx