Self-Healing Data Project 2 - Exploring Alternatives

This is the second installment of my brief "ride-along" series where I'll be walking you through the innovation process from idea to finished product.

In the first installment of this series, we worked through the vision. We talked about why it was important to separate the phases of exploration and exploitation, or generation and cultivation, or ideation and critique. We're still in the ideation phase, generating ideas, so we will continue to suspend our judgment and our disbelief as we work through what the path to our vision may look like.

In summary, the vision so far is that we want data that heals itself, that leaves a record of what it healed and why, that communicates with stakeholders, and that looks upstream and downstream for causes and effects. If you just had any flavor of the thought "that sounds like a tall order", this is your reminder to save the critique for a later phase. Do not allow your natural inclination to self-reject to lower your efficiency in this process.

You may also at this step be tempted to explore how the dream can be accomplished. This is the wrong phase for that. We cannot know what implementation details are necessary or sensemaking until we know specifically what to do. In order to avoid diving into the "how" prematurely, we want to stick to exploring the "what". Working backward from that description of the idea, what needs to happen in order to realize it?

• Data has to be checked
• Data has to be corrected
• Communication has to be sent
• There has to be a standard for what data should look like
• Logs have to be maintained
• Logs have to be interpretable

Each of these points has subpoints, things that must also be true for these things to be realized, but the deeper we dive, the closer we get to the how. This is the point in the process where it is appropriate for us to think "All of this sounds great but how are we doing to do it?" Or, rather, it is now appropriate to ask "What are all of the ways that this could be done?" In order for us to get the most that we can out of this exercise, we need to be generative and expansive before we start limiting and focusing in. I don't want to know how data is checked. I want to know ALL of the ways that data can be checked so that I give myself as much room as I possibly can have to navigate the space to a high quality solution. The scenario happens to have some natural constraints but you'll be surprised by how far we can stretch those.

So, what are all of the ways that data can be checked?

• A person can read print-outs line by line
• A person can read print-outs column by column
• Software can check data using checksum
• Software can check data using conditionals or asserts
• Software can check data using dictionary lookups
• Software can check via clustering
• Software can check via tree traversal
• A slime mold colony can be grown specifically to validate data sets which will be represented as piles of sugar and pools of alcohol respectively
• Hardware working with computer vision that manually verifies paper printouts
• An aggregate of the same number of people as there are lines of data, each with one line that is theirs for the duration of the exercise, self-sorting into categories.
• Scent-tagged sheep, using pairings of matched data and scents, can be herdered into groups by sheep-dogs.
• TSP algorithm can find the longest path between plotted data points for attribute pairs
• Mice can run mazes designed based on commonly occurring patterns in the data, such that the mazes are more frustrating the more different the production is from the sample data and blood cortisol levels can be measured upon completion.

You get the picture. You may review some of those options and think "ridiculous". Slime mold solves the Travelling Salesman Problem better than most algorithms do and they've existed for 600 million years. Most people do not think far enough outside of the box to come across unusual solutions to problems like that (where biomimicry could have saved us decades of research). Generate first, critique second.

For each "what" item, we generate a list of "what are all the ways this can be accomplished". Let's compare what that looks like to the standard workflow

.

In the standard workflow, because we coordinate on a single solution up front, by auto-rejecting everything that sounds "unreasonable", it puts us on a fixed track towards implementing that single "reasonable" solution. In generating an abundance of options based on a detailed ideal vision, we give ourselves the ability to navigate that space. This is why it's a mistake to begin the 'cutting' process too soon here. We don't even know enough to evaluate what to cut yet and we're also cutting ourselves off from tangential innovation.

Next we'll work on how to narrow the configuration space we've just created by generating this plurality of options.