My Zero-to-One Journey

I’m flying home from an amazing week at Distributech in Orlando, FL, where we announced the 1.0 of GridOS Data Fabric, our agile data infrastructure which deploys on top of our GridOS Foundation Kubernetes stack to bring the benefits of a data mesh architecture to the grid industry. This represents the culmination of a 4 year journey that I have been on since I joined GE grid software in late 2019. Looking out the window of my plane, I can’t help think about the path that led me to begin that journey—and how it prepared me to walk it.

My career has been a long and winding road, and one that doesn’t fit the mold for most software engineers. There were times in the past where I felt self-conscious about that. My degree is in philosophy, and I have worked in a number of fields/roles before settling my career on building data management products. At a certain point in my career (especially during my time as a startup co-founder/CEO) my perspective changed, and I realized it was actually a feature, not a bug. It might not be evident by looking at my LinkedIn profile, so I want to use my GridOS Data Fabric journey to illustrate how a non-traditional path can prepare us for unique challenges in ways that are hard to duplicate via a more traditional route.

I created the first cornerstone of GridOS Data Fabric, Anybase, back in 2020. I drew on the knowledge of databases gained while working full-time in the data management technologies field starting back in 2014 when I joined Couchbase as an engineer. I spent those years since reading academic/industry research papers, for which a philosophy degree provides good training. The Relational Calculus, which Anybase’s SQL engine implements, even derives in part from the work of Bertrand Russell, which I studied in my upper division philosophy classes (applied philosophy!).

I created the next cornerstone, Organizer, in 2021, which adopts message processing paradigm based on the Enterprise Integration Patterns—a book I own and got to know well coming up as a software architect. I was also once long a Cisco-Certified Network Administrator, so I knew the power and flexibility of routing and data flows paradigms more broadly. I found I could also apply some of the concepts from general packet networking to messaging pipelines and data flows.

GE Vernova software manages around half of humanity’s electricity at some point or another. Billions of lives depend on our software to power all aspects of modern life, from education and entertainment to banking and broadband. We know the importance of reliability, and have long included data quality checks in our SCADA and modeling applications. However, the decarbonized grid requires a deeper relationship with data than ever before. Our industry is dealing not just with more volume of data, but with more variety moving at a higher velocity. That is already a tough enough challenge for managing operational data and analytics. However, there is one more big challenge we must meet: the veracity of data.

Veracity, the degree to which our data truthfully represents the state of electrons and the devices that control them, is more important than ever just by virtue of rapidly expanding complexity and the automation needed to orchestrate the grid. Add to that the fact that the technologists at utilities and operators face growing pressure to iterate faster and more frequently in order for their organizations to adapt to a rapidly changing market and climate. The fragility of today’s data environment makes it seem impossible to build and deploy new decision support and integration projects while improving reliability. It can be achieved, but only by delivering self-service access to data, automated enforcement of governance policies, and by broadening the traditional view of grid data quality.

I made sure GridOS Data Fabric delivers new data quality management and governance tools to meet these needs. I drew heavily on my own experience as an analyst for the Florida Department of Juvenile Justice’s Bureau of Data and Research. I was tasked with defining the data governance policies and enforcement mechanisms for the statewide Juvenile Justice Information System used by most of its 10,000 employees. This was a data “governance” project in the fullest meaning of the word. The project was initiated at the direction of the Florida Legislature after its Auditor General identified a number of systemic data quality problems. The lessons I learned 20+ years ago turned out to be more common than I realized, and the accumulated knowledge and best practices are refined, elaborated on, and built into the structure of GridOS Data Fabric itself.

GridOS Data Fabric isn’t just my journey. It’s a journey that’s been shared alongside many others. Software is a collaborative medium, just like film and television—something I also spent a couple of years doing. Like a film, Data Fabric is more than the sum of its individual contributions. There is a strong social component to making good software and communication and camaraderie are the glue that binds both good film crews and software teams together.

And like a film, to make any software work you have to anticipate the effects it will have on those using it. During my undergrad, I came within one class of completing a minor in anthropology. While I don't typically find any daily uses for paleo-archeology and population genetics knowledge, I do find the ethnography skills I was taught to be incredibly useful in software engineering. They are handy for understanding how the shape of a solution and the assumptions it encodes play a large role in adoption. I’m not alone in thinking this.

I hope this post helps to give confidence to others like me who have taken a non-traditional path to an engineering career or are new to power systems like. I often tell people: tech is 5% hardware problems, 5% software problems, and 90% “wetware” problems. Having broad experience and a broad liberal arts education both provide an excellent foundation for solving all three kinds. Software engineering also needs non-traditionalists to bring in new ideas which help to keep our field creative and dynamic.

Two more things: I feel profound gratitude to the many people who have contributed to GridOS Data Fabric and helped it become a reality. While there are too many to list here, there are some without whom GridOS Data Fabric would never have happened: Michael Hart , Eric Fleuret , Jeremy Osterhoudt , Matt Hempleman , Andrew Gillies , and especially Andy Rector for championing our team and mission.

I’m very excited to finally be able to talk about what I’ve been working on for the past 4 years. The database and distributed systems communities are so active and vibrant, and it’s been tough not being able to share with you all. We have done innovative work and have a lot to talk about. Stay tuned for upcoming talks, papers, and events!