? Introducing AIKit: seamlessly fine-tune, build and deploy open-source LLMs as secure containers!

AIKit is designed to be a comprehensive solution for developers looking to fine-tune, build and deploy large language models (LLMs) with ease. With AIKit, you dive into a world of possibilities without the complexities!

???Key Offerings of AIKit:

1. Inference:?AIKit brings extensive inference capabilities across various formats. Its compatibility with the OpenAI API through a drop-in replacement REST API means you can seamlessly integrate any OpenAI API compatible client to interact with open-source LLMs.
2. Fine Tuning:?AIKit offers a fine-tuning process with an extensible interface, with a fast, memory-efficient, and straightforward fine-tuning experience.

?? Why AIKit?

• ?? For inference, operates without the need for GPUs, or Internet - Docker is all you need!
• ?? Create containers with minimal image sizes, enhancing security with no vulnerabilities and a smaller attack surface, thanks to a custom distroless-based image!
• ?? Offers robust fine-tune support that's fast and memory efficient!
• ?? Utilizes an easy-to-use declarative configuration for both inference and fine-tuning.
• ? Fully compatible with OpenAI API, allowing use with any compatible client.
• ?? Supports multi-modal models and image generation with Stable Diffusion.
• ?? Compatible with a variety of models like GGUF, GPTQ, EXL2, GGML, Mamba, and more.
• ?? Ready for Kubernetes deployment.
• ?? Capable of supporting multiple models within a single image.
• ??? Facilitates GPU-accelerated inferencing with NVIDIA GPUs.
• ?? Ensures security with signed images with sigstore cosign.
• ?? Support for non-proprietary, self-hosted or local container registries to store model images.

??? Getting Started is a Breeze!?

Kick off with AIKit on your local machine sans a GPU with a simple Docker command and see the magic happen!

docker run -d --rm -p 8080:8080 ghcr.io/sozercan/llama2:7b        

curl https://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
    "model": "llama-2-7b-chat",
    "messages": [{"role": "user", "content": "explain kubernetes in a sentence"}]
  }'        

?? Hit the ground running with pre-made model containers

From Llama 2 to Gemma, from Mixtral to Phi 2, AIKit offers pre-made models for both CPU and NVIDIA CUDA environments, ready to use out-of-the-box!

?? What's Next??

For deeper dives into fine-tuning models, creating your own images, and much more, head to the AIKit website. Embark on your AI journey with AIKit today and unlock the full potential of Large Language Models with minimal fuss!