AI for Developers: Tabnine and GitHub Copilot

As Copilot is becoming generally available, this might be a good time to write a comprehensive comparison between the two leading AI assistants for software development Tabnine and Copilot by Microsoft.? Details here are from our CEO and Founder Dror:

Usually, I suggest that my team start with the user value and experience, but for this specific comparison, it’s essential to start from the technology, as many of the product differences stem from the differences in approach, architecture, and technology choices.?Microsoft and OpenAI view AI for software development almost as just another use case for GPT-3, the behemoth language model. Code is text, so they took their language model, fine-tuned it on code, and called the gargantuan 12-billion parameter AI model they got Codex. ?

Copilot’s architecture is monolithic:?“one model to rule them all.” It is also completely centralized - only Microsoft can train the model, and only Microsoft can host the model due to the enormous amount of computing resources required for training and inference. ?

Tabnine, after comprehensively evaluating models of different sizes, favors individualized language models working in concert. Why? Because code prediction is, in fact, a set of distinct sub-problems which doesn't lend itself to the monolithic model approach.?For instance: generating the full code of a function in Python based on name and generating the suffix of a line of code in Rust are two problems Tabnine solves well, but the AI model that best fits every such task is different.?We found that a combination of specialized models dramatically increases the precision and length of suggestions for our 1M+ users.

A big advantage of Tabnine’s approach is that it can use the right tool for any code prediction task, and for most purposes, our smaller models give great predictions quickly and efficiently. Better yet, most of our models can be run with inexpensive hardware.

Now that we understand the principal difference between Microsoft’s huge monolith and Tabnine’s multitude of smaller models, we can explore the differences between the products:

First, kind of code suggestions. Copilot queries the model relatively infrequently and suggests a snippet or a full line of code. Copilot does not suggest code in the middle of the line, as its AI model is not best suited for this purpose.?Similarly, Tabnine Pro also suggests full snippets or lines of code, but since Tabnine also uses smaller and highly efficient AI models, it queries the model while typing. As a user, it means the AI flows with you, even when you deviate from the code it originally suggested

The result is that the frequency of use - and the number of code suggestions accepted - is much higher when using Tabnine. An astounding number of users accept more than 100 suggestions daily.

Second, ability to train the model. Copilot uses one universal AI model, which means that every user is getting the same generic assistance based on an “average of GitHub”, regardless of the project they're working on.?Tabnine can train a private AI model on the specific code from customers’ GitLab/GitHub/BitBucket repositories and thus adjust the suggestions to the project-specific code and infrastructure.?Training on customer code is possible because Tabnine is modular, enabling the creation of private customized copies. Tabnine "democratizes" AI model creation, making it easy for teams to train their own specific AI models, dramatically improving value for their organization.

Third, Code security and privacy. There are a few aspects of this. Users cannot train or run the Copilot model. The single model is always hosted by Microsoft. Every Copilot user is sending their code to Microsoft; not some of the code, and not obfuscated - all of it.?With Tabnine, users can choose where to run the model: on the Tabnine cloud, locally on the developer machine, or on a self-hosted server (with Tabnine Enterprise). This is possible because Tabnine has AI models that can run efficiently with moderate hardware requirements.?This means that, in contrast to Copilot, developers can use Tabnine inside their firewall without sending any code to the internet.?In addition, Tabnine makes a firm and unambiguous commitment that no code the user writes is used to train our model. We don’t send to our servers any information about the code that the user writes and the suggestions they’re receiving or accepting.

Philosophically, Copilot is more of a walled garden where Microsoft controls everything. Copilot users are somewhat subjects in Microsoft’s kingdom. Tabnine’s customers can train the AI models, run them, configure the suggestions, and be in control of their AI. ?

In sum: both products are great; you’re welcome to try (Tabnine Pro) and see which one you prefer. for professional programmers, Tabnine offers in-flow completions, the ability to adapt the AI to their code, and superior code privacy and security.

Also, here's a detailed comparison table of Tabnine vs Copilot https://tabnine.com/tabnine-vs-github-copilot