What role will data standards organizations play in a world powered by artificial intelligence?

Note: A version of this story was first published on June 18, 2019, at FreightWaves.

There was significant excitement in the container shipping industry in April. The reason for this excitement is the formation and public launch of the Digital Container Shipping Association (DCSA). DCSA is a standards organization that is being established to create common information technology standards, paving the way for interoperability in the container shipping industry. That got me thinking; In an age when artificial intelligence could probably be applied to solving some of these problems, what role should a data standards organization play?

What is a standards organization?

A standards organization is an organization whose primary purpose is to develop technical standards that address a problem common to the organization’s prospective members — in most cases prospective members will be companies within a specific industry. Container shipping, for example. Once the standards have been developed, the standards organization encourages their adoption by the industry as a whole. Initial announcements indicated that DCSA was formed by MSC, Maersk, Hapag-Lloyd, Ocean Network Express, and CMA CGM.

Why does the container shipping industry need a standards organization?

Maritime freight shipping insiders complain that there are still too many manual processes reliant on paper and pen. Moreover, a lack of common data standards means that it is very difficult to conduct business across all shipping carriers. Banking transactions between customers of different banks around the world are relatively straightforward because the global banking industry has adopted Society for Worldwide Interbank Financial Telecommunication (SWIFT) standards to enable transactions between banks on behalf of bank customers. The shipping industry has no SWIFT equivalent.

The downside of not having industry wide standards is that it makes it difficult for customers to easily do business with one another if doing so requires the exchange of data and information. It can also impede innovation across the industry since there’s never a complete picture about the issues confronting the industry.

That state of affairs is no longer acceptable to the customers that the shipping industry serves.

Well then, what’s the problem?

I was a data analyst during the early part of my career. A major part of my responsibilities involved scrubbing data to make it useful for my own analysis. During my time at UBS, one of the databases I relied on had 150,000 rows and 40+ columns, and I experienced, first-hand, what Hugo Browne-Anderson described in the HBR article What Data Scientists Really Do, According to 35 Data Scientists: “It has been a common trope that 80% of a data scientist’s valuable time is spent simply finding, cleaning, and organizing data, leaving only 20% to actually perform analysis.”

We can safely assume that data standards organizations seek to materially reduce that 80% drag. At the same time there are many startups today that are building platforms to solve this problem, without necessarily relying on standards organizations. So the question is: In the future, will there be a need for data scientists to spend any time at all on cleaning and preparing data? The answer, from some startup founders is: No, and data standards will not be necessary.

The platforms being built are aptly described in The Business of Platforms by Michael Cusumano, Annabelle Gawer, and David Yoffie: Systems that “bring together individuals and organizations so they can innovate or interact in ways not otherwise possible, with the potential for nonlinear increases in utility and value.” [1]

These platforms would mainly enable innovation, but they’d facilitate transactions between individuals and organizations as well. The real power of such a platform will be around the capabilities it provides for participants in specific industries to more easily collaborate with one another.

Such a platform could automate up to 99% of the manual data janitorial work required of data scientists today by;

• Enabling direct and indirect collaboration between different categories of platform participants through the use of application programming interfaces,
• Predicting and automatically executing the thousands of steps required to read, parse, understand, cleanse, and transform data so that it can be used by artificial intelligence systems,
• Retaining full transparency with the ability for human experts to override the system at any step, and
• Automatically managing the orchestration and synchronization of all dependent transformations downstream.

Industry groups are simply ill-equipped to solve this problem at scale.

Industry consortia, like DCSA, are formed with good intentions. However, such good intentions overlook the stark reality that such groups are generally ill-equipped to solve the problems they most desperately need to solve in order to successfully accomplish their purpose and mission. Here are just a few of the problems such groups face.

• First, getting buy-in from a representative portion of the industry’s players is difficult.
• Second, decision-making is painfully slow. I experienced this first-hand during my days at UBS and Lehman Brothers. To get around it, I often by-passed the industry group’s modalities and executed a process that I felt suited my team’s specific objectives.
• Third, commitment is very difficult to secure. This is true both at the outset, when an individual company is being persuaded to join the group, and, on an on-going basis. For example, whenever a new CEO, CTO, or CIO is appointed, a company’s participation in the industry group faces the risk of being called into question.
• Fourth, because companies in the same industry nonetheless pursue different strategies and priorities, there are always questions about what is being standardized and why those specific standards have been chosen by the group.
• Fifth, often such industry groups lack the required technical expertise to actualize the group’s stated purpose and mission.

What should such industry groups focus on?

One of the things that has struck me about the shipping industry is the lack of trust that exists amongst companies in the industry. I have been even more surprised when people who have spent many more years studying the industry have pointed out to me that this acute distrust goes all the way down to the level of individuals in the industry. To paraphrase the words of one person I spoke with about a year ago; Someone stole someone else’s girlfriend, or won a promotion the other did not get while they were in the management training program at Maersk, and since then they have been sworn, life-long enemies. The trouble is that no one else knows this history, and so their actions now that they are c-level executives at these big shipping companies do not make much sense to people on the outside.

Industry groups and standards organizations need to spend most of their time and energy building trust.

I described this as the Culture Question when I spoke with Eric Johnson, Senior Technology Editor at the Journal of Commerce, for his article JOC Blockchain Explainer: Demystifying the buzz for the shipping industry. The culture question is about trust, ethics, and how the players within an industry decide to engage with one another; Are there areas of close cooperation, distinct from areas of intense competition, or is everything out-and-out cut-throat competition?

By comparison the Tech Question is easy to solve. The builders of new innovations for supply chain logistics, supply chain finance, and supply chain management include within their ranks researchers, academics, engineers, technologists, inventors, and entrepreneurs capable of solving the Tech Question.

The only people with the capacity to solve the Culture Question are the organizations and people who seek to buy new innovations for supply chain. That is where standards organizations can be most effective; building trust between people and organizations that have historically had no reason to trust, or cooperate with one another, and establishing areas where the industry agrees cooperation will create disproportionate value for the industry’s customers and suppliers.

The alternative? Unpleasant competition and potential disruption imposed by digitally savvy attackers, who also have the financial wherewithal to impose pain on industry incumbents.

[1] Cusumano, Michael A., The Business of Platforms (p. 13). HarperBusiness. Kindle Edition.