MINING THE GAP (in support of decision making)

Recently, a live experiment was conducted to measure bias in a group of geoscientists to establish the breadth and depth of opinion in support of decision making. A group of 49 oil & gas sector geoscientists (mostly basin modellers) consented to participate in the live, interactive dataroom simulation at the Geological Society to assess their bias for a frontier opportunity, and to gauge their attitudes to risk.

We scarcely need reminding that exploring for oil and gas is fundamentally an evidence-based science.??Basin modellers create models that convolve data and geological learning to make predictions about the presence/absence, and the nature of petroleum systems.??We demand a great deal from basin models to predict what a petroleum system is capable of delivering (its endowment potential), whilst also revealing some of the main levers to polarise risk, which in turn supports decision makers to drive data acquisition to further reduce risk.??But it is often?uncertainty?that drives basin modelling, which is there because, for all the evidence, there are gaps in our knowledge as there are gaps in our?perceptions.???The complexity is compounded by the fact that all basins are unique, which means that any basin model will always be wrong because (amongst other things) we can’t accurately characterise what we don’t know.??Uniqueness is the enemy of predictability, so we have to consciously build compromise into models (normally around resolution), in the hope of capturing a plausible range of what we know we don’t know.??The greater the uncertainty, the larger the content of the basin model that is strongly influenced by the content knowledge (the experience) of the basin modeller.??

We pride ourselves on our objectivity when it comes to prediction.??Unfortunately, what we do know is more influential than what we don’t know and that, in itself, is a form of?bias, which can lead to overconfidence.??In effect, we use the models to convince ourselves of the predictability of events, which are in fact unpredictable, implicitly denying the reality of uncertainty.??That doesn’t invalidate the model but it should make us wary of the influence that our motives may have exerted in making the model.

The live results of the simulation were fed back to participants during the session, revealing two different modes of thinking. Firstly, participants were asked to self-report their opinion on the prospectivity of the opportunity; their intuitive gut-feel on a scale of 1-5. The simulation secondly deconstructed each opinion by examining what the data were actually telling them about the opportunity as individuals, expressed on the same scale of relative attractiveness for the opportunity. By comparing participants’ opinions with what the data were actually telling them, a measurable internal conflict (a perceptual gap) emerged, revealing a clear swing from positive to negative with increasing experience, which the authors argue is both measurable and essential to identify in support of decision making.

The results demonstrated that experience really does make a difference for basin modellers, but in a manner that came as a surprise to the authors*. That conclusion was only achievable because the session provided sufficient numbers of individuals whose expertise was specifically targeted at a known flaw in the data.???How you blend perspectives (whose opinion you elicit) to contribute to decision making is shaped by the business question driving the elicitation.??The experiment confirms the existence of a perceptual gap between participants intuitive gut-feeling for an opportunity and what the data are actually telling them – a gap which is measurable as a vector.??This vector has practical application in support of decision making but with appropriate verification, can it also illuminate the human condition???

Good science is occasionally experiencing the triumph of the unexpected but you also have to be able to spot it; the experiment continues.

*for a full account of what the data tell us about basin modellers, please read the "results" article: www.k2vltd.com/article-25