Rethinking Container Orchestration: A Proposal for the Future

Containerization and orchestration have revolutionized IT infrastructure, with Kubernetes reigning supreme in the orchestration domain. However, according to a provocative article by Ian Eyberg, CEO of NanoVMs, containers might not be the future of IT infrastructure.

Challenging the Status Quo

Ian Eyberg contends that the container architecture and, by extension, Kubernetes, are inherently flawed. Security has been a constant concern, and the innate design of these systems makes them vulnerable. What's even more alarming is that many organizations adopt Kubernetes not necessarily for its technical or economic benefits but more as a tactic to retain engineers who've built their expertise and personal brand around this technology.

The reality is, many have spent years pushing the narrative that Kubernetes is the solution to all infrastructure problems. But as Eyberg points out, there are alternatives, and containerization might not be the definitive solution we've been led to believe.

MicroVMs and NanoVMs: The Real Future

Eyberg emphasizes that both Google and AWS are acutely aware of the limitations of containers and Kubernetes. Proof of this is their investment in technologies like Firecracker and gVisor, aiming to offer more secure and efficient solutions.

Eyberg's vision is that microVMs, particularly unikernels, will dominate the future of IT infrastructure. These technologies promise greater security, efficiency, and flexibility compared to traditional containers.

Moreover, there are already initiatives treading these waters, merging tech like Web Assembly and Blockchain to craft distributed cloud infrastructures. Projects like Blockless Network, StackOS, and wasmCloud stand as testament to these burgeoning innovations.

The Serverless Potential with MicroVMs and Unikernels

The trajectory Eyberg hints at, coupled with the inherent advantages of unikernels, could supercharge the serverless paradigm. By slashing cold start times down to milliseconds through this orchestration and the minimal overhead of unikernels, we might witness efficiency like never before. Hurdles that today seem insurmountable with current serverless solutions, like prolonged boot times, could be effectively tackled. This wouldn't just boost efficiency but would also pave the way for unparalleled automation and resilience, unlocking fresh possibilities for cloud applications and how they're deployed and scaled.

Toward Autonomous Orchestration

Rather than relying on centralized cluster management, what if agents managed themselves? Imagine a system based on three core principles:

• Decentralized Knowledge: Agents eventually know their peers.
• Dynamic Leadership: Any agent can assume a management role, but only one at a time.
• Persistence and Availability: Using dynamic replicas backed by snapshots for initialization.

The vision is straightforward: agents, instead of being passive microservices, take charge of orchestrating themselves.

Technologies and Design Principles

To realize this vision, various technologies and principles could be employed:

• Micro(nano)-VMs and Unikernels: Reducing overhead and optimizing resources.
• Gossip Protocol and RAFT Consensus: For decentralized coordination and decision-making.
• Distributed Messaging and Immutable Event Databases: Potentially using technologies like blockchain.
• Event Prioritization over RPC: Favoring event-based communication over remote procedure calls.
• Operations and Management Tasks: Handled by specialized micro-VMs, with communications facilitated by a NATS hub.

Hypothetical Development and Involved Technologies

To implement this vision of autonomous orchestration, a structured development plan is essential, considering the right technologies at each stage. Below is a hypothetical roadmap based on planned progress:

• Initialization with Minimum Replicas: The system would start with a bootstrap establishing a minimum number of replicas to ensure redundancy and availability from the get-go.
• Member List Configuration: Using the gossip protocol, agents could discover and communicate with each other in a decentralized manner.
• Leader Selection using RAFT: Leader election would be based on clear rules, such as preferring agents with more available resources. If a RAFT consensus agent leaves the cluster, another from the remaining agents would be chosen to ensure leadership continuity.
• Integration with NATS: An embedded NATS cluster would be created to facilitate agent communication. In later stages, these NATS nodes might be separated from RAFT nodes for greater modularity.
• Data Management and Replication: The leader would maintain an up-to-date management database and be responsible for replicating information to the rest of the RAFT consensus agents.
• Micro-VM Deployment in the Cloud: A dedicated tool streamlines the auto-deployment of micro-VMs on a Cloud Platform—ideally, a decentralized one—enhancing the system's scalability and adaptability.
• Specialized Micro-VMs: For specific tasks, like curating and monitoring, specialized micro-VMs would be launched that could manage themselves autonomously.
• Distributing Changes to All Nodes: Through mechanisms like NATS publish/subscribe, all nodes would be ensured to be informed and synchronized with the system's latest changes.

Building this hypothetical system would require integrating multiple technologies, including decentralized communication protocols, lightweight containerization solutions, and cloud management tools. While this development poses significant challenges, it also offers an opportunity to rethink and enhance our current container orchestration practices.