STOIC, a Modern Data Workbench

Ever since we started STOIC, we have introduced it as a next-generation spreadsheet, or a big data spreadsheet. This triggered a lot of productive discussions, but it also inspired some resistance or opposition: can STOIC really replace conventional spreadsheets? Is it really a spreadsheet, or is it something different? And should we still call it a spreadsheet?

Today, I believe that we should graduate from the spreadsheet terminology, and call our product what it really is: a modern data workbench. Is it a spreadsheet? Yes, absolutely! But calling it a spreadsheet only tells one small part of the story. STOIC is more than that. At its core, STOIC integrates into a single user experience most of the functionality offered by the following tools:

• A data preparation tool
• A data warehouse
• A data analytics tool
• A data visualization tool
• A programming notebook
• A report generation tool
• A presentation tool

That is quite a lot of functionality to pack into a single user experience, and what makes STOIC work is that we did not start from six or seven individual components which we tried to stitch together. This would have created a monster that only Victor Frankenstein could be proud of. STOIC is not a toolbox. It is a workbench.

At its core, STOIC is made of just three distinct components:

• Sheets (think conventional spreadsheets)
• Tables (think database tables)
• Notebooks (think crossover between word processor, presentation tool, and code editor)

Why three and not two or four? Because math! Let me explain:

It is all about dimensionality.

On one end of the dimensionality spectrum, you have spreadsheets, which are fundamentally two-dimensional objects. They give you columns and rows, which can be scrolled horizontally and vertically. Columns and rows play a largely interchangeable role, and this is what gives the spreadsheet its unmatched flexibility.

On the other end of the spectrum, you have word processors or presentation tools, which are fundamentally one-dimensional objects. You have a linear narrative, which can be scrolled vertically on a word processor, or slide by slide on a presentation tool.

This difference in dimensionality is why a spreadsheet and a word processor work in fundamentally different ways, and why you cannot simply merge them into a single tool. Unfortunately, these are not sufficient. Once your data becomes larger, it does not fit within a conventional spreadsheet anymore. Instead, you want to upgrade to a database, which tables look two-dimensional, but not in the same way as spreadsheets are two-dimensional.

In a relational database table, columns and rows do not play interchangeable roles: if you are a statistician, you would describe the columns as your variables, and the rows as your observations. And if you put your data modeler shoes on, you know that the values of a column in a relational database must share the same datatype (e.g. integer or string).

There is no clear quantification for the dimensionality of a database table. For the sake of argumentation, we will just accept that it is greater than one and lower than two. Somehow, it falls in between the dimensionality of a word processor (one), and the dimensionality of a spreadsheet (two). And that is why you need at least three distinct components.

But do you need more? Well, at STOIC, we believe that less is more, and that three is plenty already. Therefore, we have worked very hard to ensure that you can do pretty much everything you need with just three: sheets, tables, and notebooks. How did we manage to do that? First, we looked at all the tools that are one-dimensional, and we merged them into a single user experience: the notebook. These included:

• The code editor for developing custom functions and models
• The word processor for generating reports
• The presentation tool for designing slides

You can use our notebook for doing all three.

Second, we made our database table editor powerful-enough that it could be used to support a wide range of activities, including data preparation, data analysis, and data visualization. But this required that we step into the third dimension, by going from single tables to stacks made of multiple tables (what we like to call layers). This made it a lot easier to implement complex data preparation pipelines without having to resort to confusing flowchart tools, and design multi-layer charts and maps once the data has been properly sliced and diced.

Third, we used the conventional spreadsheet metaphor as a two-dimensional canvas that can be used to create complex arrangements that really could not fit within a linear structure. This turned out to be critical for supporting the creation of interactive dashboards as well as content-rich presentation slides.

Last but not least, we made sure that every component could be embedded within any other component, extending the re-entrant dependency graph that made the original spreadsheet so very effective: you update a value somewhere, and every other value that depends on it through simple formulas gets refreshed automatically. This means that you can:

• Embed a table within a sheet range.
• Embed a notebook within a sheet range.
• Embed a sheet within a notebook block.
• Embed a table within a notebook block.
• Embed a notebook block within a table cell (usually through column-level formulas).
• Embed a sheet range as data source within a table.

This flexibility is what makes STOIC work, for a wide range of use cases. Granted, it is more complex than a conventional spreadsheet, but it is also a lot more powerful, and we made sure to embrace conventional metaphors and design patterns whenever possible.

In short, we built a modern data workbench.

Please contact me if you would like to join our early access program.