Transformation Framework Part II: Value of Qual & Quant Data

The study of transformation, leadership, and management has historically, if not necessarily, been a social science. Sources written and conclusions made about each have been dominated by qualitative arguments whose explanatory power is determined by comparison to an ideal, fusion of various theories, and case studies. This chapter discusses the relative explanatory power of quantitative vice qualitative studies in order to provide a general illustration of the increased specificity and explanatory power offered by quantitative techniques. Though the underlying sources used in this thesis do not all use the most rigorous methods highlighted in this chapter, the value of quantitative methods will become apparent and enable the use of quantitative terminology as we discuss and describe the methodologies and validity of each of our sources. Discussing these concepts separately also avoids interrupting later discussions.

Business texts represent the bulk of material regarding transformation, leadership, and management. Most best-selling titles, as well as a cacophony of lesser known sources, are characterized by qualitative techniques, or worse, colloquialisms, and conventional wisdom. Most abduct a story, anecdote, or partial set of data followed by an inference, or series of inferences, to argue particular factors seen in one organization represent a linkage to, or a central driver of, any an organization’s success. However, even the most reasonable arguments offer no guarantee an apt analogy has been drawn or take a deliberate approach to argue alignment to a conventional conception of historical analogy. This is not to suggest many do not offer useful illustrations, homilies, or fables regarding seemingly true lessons. The best do. Typical business texts merely do not guarantee particular factors, behaviors, or characteristics necessarily compel a particular outcome, let alone high-performance in all cases.

Better sources do use a structured approach. They specify a central question, idea, or theory, followed by deliberate research design, data collection, analysis, and conclusions as justification of their findings. Common instruments include interviewing, field research, secondary sources, historiographies, oral history, and biographies, all of which help gather more comprehensive data, enable deeper analysis, and draw more precise and reserved conclusions. Social science refers to these techniques as triangulation.[1] Still better triangulation uses observations, data, and trends that are analyzed, and conclusions arrived at, from a number of different perspectives, necessarily offering greater certainty that a conclusion is correct.[2] Triangulation is similar to what the IC might refer to as multiple independent confirmation followed by an analysis of competing hypotheses.?Such triangulation in business texts is rare.

The best being said, even the best qualitative argument has limitations. Qualitative research tends to describe the unquantifiable and “seeks answers to questions by examining various social settings and the individuals who inhabit these settings.”[3] It explains “how humans arrange themselves and their settings…make[ing] sense of their surroundings through symbols, rituals, social structures [and] social roles” as a “means of accessing unquantifiable facts about the actual people researchers observe.”[4] Therefore, if an outcome or phenomenon is quantifiable, a qualitative approach would not be preferable.

This takes us to a discussion of the explanatory power of quantitative data. Just as triangulation increases the quality and certainty of qualitative arguments and conclusions, the right techniques applied against a sufficiently large set of quantitative data provides the opportunity to offer conclusions that are better still. “The methods used by social scientists fall on a continuum from totally uncontrolled (and perhaps uncontrollable) techniques arising in natural settings to totally controlled techniques of observation.”[5] A phenomenon whose inputs and outputs are measurable make it, if only in relative terms, worthy of controlled techniques. As such a quantitative approach is capable of yielding more precise analyses and conclusions. This is a critical distinction between qualitative (uncontrolled) and quantitative (controlled) data, and is the cornerstone of this thesis. Transformation, leadership, management, and human capital have long been thought to be necessarily qualitative matters. However, that is no longer the case.

First, the outcomes of commercial business are inherently quantifiable. Second, we find that we can meet the necessary data set size criteria via profitability and related metrics in the commercial space. With the advent of modern computing power it much easier to compare volumes of data to determine which factors (education, training, tenure, etc.) actually drive enterprise-level outcomes (revenue, productivity, profitability, satisfaction). Statements like “I think the best leadership profile is”, “I think the best management practice is”, or “I think transformation is driven by”, are no longer necessary. Based on quantitative and statistical research we can disambiguate, to a level of mathematical certainty, what the quantifiable answer to each of those questions are. Even in the event we cannot quantify the drivers in and of themselves, we will still make better inferences since we know the underlying explanatory power and degree of certainty of the few factors of concern.

Third, a quantitative approach directly pertains to our inherently quantitative definition of transformation. As such, the sources underpinning the remainder of this thesis use quantitative methodologies with similar performance-based transformation definitions. The goal of both commercial and government organizations is to increase their performance such that their products and services are delivered in the most efficient manner customers repeatedly return with confidence and delight. At least it should be. Suffice to say that is the outcome produced by the best commercial, government and non-profit organizations.

Before offering details regarding our sources’ specific methodologies, we will briefly describe the quantitative terms and techniques that offer progressively larger explanatory power.?Relevant terms include concept, clarified concept, conceptual definition, operational definition, significance, correlation, regression, validity, reliability, and test-retest. Our first four terms are relatively straight forward and contribute to a quantitative study by defining what we are attempting to measure with progressive specificity. A concept is “an idea or mental construct that represents phenomenon in the real world.”[6] It is commonly referred to in a single word that is commonly used but difficult to define (e.g. “threat,” “terrorism,” “transformation”).[7] A clarified concept provides an initial sense of the characteristics corresponding to a concept as well as those that do not (e.g. “average,” “inflection point,” “great”). A conceptual definition then communicates at least three things: “the variation within a characteristic, the subjects or groups of subjects to which the concept applies, [and] how the characteristic is to be measured.” (e.g. long-term financial performance, revenue growth vis-à-vis competitors).[8] Finally, an operational definition “describes explicitly how the concept is to be measured empirically.”[9] Our operational definition for transformation has already been offered: an enterprise that, having long performed average to the general market, exhibited a performance inflection, such that its long-term performance was dramatically better than the general market, its industry, and direct competitors then sustained for fifteen years or more.

After a large set of quantitative data is complied, a number of calculations can be used to analyze averages, means, and distributions. We will not discuss these techniques in any great detail. There are simply too many such techniques, depending on type of distribution of data within a data set, for a brief discussion. In short, these techniques provide the ability to identify trends within the data set as a whole. The most relevant technique for our purposes is statistical significance. Statistical significance is used early in an inferential statistics study to determine “whether an observed relationship between an independent and dependent variable really exists in the population or whether it could have happened by chance.”[10] Showing the findings in a data set are statistically significant ensures we can, mathematically speaking, trust the results.?For example, a data set shown to have statistical significance with a confidence interval of .05 proves there less than five chances in one hundred where an existing relationship between a factor and an outcome either did not actually exist, or that no relationship was inferred when one in fact exists.[11]

After ensuring the level of confidence of our data set, we measure the association between independent variables (inputs, factors) and a dependent variable (output, result being measured).?This allows us to begin drawing conclusions about how, and how well, the variables explain each other. A strong measure of association along with strong significance is the recipe for the highest explanatory power between independent and dependent variables. Again, there are many ways of measuring association based on the type of distribution in a given data set. We focus on correlation and regression for two reasons. First, they clearly illustrate the progressive value of quantitative data. Second, they represent the central technique used by two of our primary sources.

Correlation is an association “that gauges the direction and strength of a relationship between variables.”[12] For example, snowfall and the winter season are highly correlated. This provides some insight, but only to the extent it compels other questions. Intuitively we understand the winter season does not itself cause snowfall.?Other conditions are also necessary.?To that end, regression improves the specificity of our analysis and determinations. Regression isolates “the effect of one independent variable on the dependent variable, while controlling for the effects of all other variables.”[13] For example, though snowfall and winter are highly correlated, regression would show how more specific factors (month, date, temperature, relative humidity, dew point, and barometric pressure) do, or do not, contribute to snowfall.

More valuable still is the fact that regression produces a “coefficient that estimates the size of the effect of the independent variable on the dependent variable.”[14] In that sense regression indicates how much the independent variables contribute to determining the dependent variable (absolute), and how much each individual independent variable contributes to determining the dependent variable compared to other independent variables (relative). More specifically, the coefficient belonging to each independent variable enable computing the R-squared statistic for a regression equation, which quantifies the extent to which the independent variables (month, date, temperature, relative humidity, dew point, and barometric pressure) explain the dependent variable (cumulative).

We can also explain R-squared in a more conventional, less colloquial way with a hypothetical mathematical equation. A coefficient represents “the exact nature of the relationship between an independent variable and a dependent variable” and “reports the amount of change in the dependent variable that is associated with one-unit change in the independent variable.”[15] Note the following regression equation where Y represents a dependent variable, X, Z, V, and W represent independent variables, and a, b, c, and d, represent coefficients. If the coefficients’ values were .5, .2, .1, and .05 respectively we would clearly see the relative value of each independent variable (X, Z, V, W) in predicting the independent variable (Y). For example, coefficient b characterizes the?

Y = a(X) + b(Z) +c(V) + d(W) + e

?Figure 1. Hypothetical Regression Equation

Source: Author Analysis.

?explanatory power of factor X. Similarly, because of the way a regression calculation would have placed the factors based on their coefficients, we would also implicitly see which factor (X, in this case) had the largest explanatory power, and how the remaining factors (Z,V,W) make a similar but smaller contribution. R-squared measures “the completeness of a relationship” between a set of independent variables and the dependent variables. Using the same example, the cumulative explanatory power of X, Z, V, and W is determined by the square of the sum of the independent variables’ coefficients. In this case our regression would explain about 72 percent ((.5+.2+.1+.05)^2) of the variation of the dependent variable. The higher a regression equation’s R-squared statistic, the better it predicts an outcome and the more powerful the independent variables, together, are for explaining the dependent variable.

To that end, multiple regression is a means of continuously adding and subtracting independent variables to a regression equation in order to determine if doing so increases our ability to explain the dependent variable. To the extent adding independent variables to the regression equation increases the equations R-squared statistic, adding that variable to the equation adds value to the explanatory power of a quantitative analysis. Similarly, if a newly added independent variable (e.g. S, T, U, etc.) showed covariance with a preexisting independent variable (X, Z, V, W) and, as such, increased the coefficient of a preexisting coefficient (b, c, d, e), this would also increase the explanatory power of our equation and quantitative analysis. Due to the complexity of the calculations, and the number of permutations required to generate results, regression and multiple regression are both typically executed with computers.

Finally, validity, reliability, and test-retest calculations provide further assurance quantitative findings are trustworthy. Validity of a measurement represents, “the extent to which it records the true value of the characteristic the researcher intends to measure.”[16] Reliability of a measurement is “the extent to which it is a constant measure of concept.”[17] Test and retest methods involve an “intuitive approach to assessing reliability: apply the measure once, and then apply it again to the same group of subjects.”[18] If a measurement is reliable a corresponding test produces similar results each time it is run.

SO WHAT? WHY SHOULD WE CARE?

Having talked in some detail about quantitative techniques it is important to put them in perspective again. In general, quantitative methods reduce the amount of subjective decisions made within a researcher’s methodology.[19] The fewer subjective decisions, the more certain and reliable our results are. Qualitative data is useful in matters where substantial philosophical doubt exists, or where the factors being analyzed simply cannot be quantified. However, to the extent these conditions do not hold, as in judging the performance of a large commercial or government organization, quantitative data is, in relative terms, a fundamentally better method for the dismissal of myth, providing well-founded conclusions, and building a framework regarding the true drivers of transformation. Focusing on sources that use these or similar techniques enables the creation of a framework that is both more reliable and repeatable than conventional wisdom, anecdotal sources, and case studies. As we explore the methodologies of our underlying sources we see the level of certainty we can have about transformation and human capital best practices and their impact on long-term performance at the enterprise, organizational, and individual levels.

[1] Bruce L. Berg, Qualitative Research Methods for the Social Sciences: Third Edition (Boston: Allyn and Bacon, 1998), 4. Cited hereafter as Berg.

[2] Berg, 5.

[3] Berg, 7.

[4] Berg, 7.

[5] Berg, 7.

?????????????????[6] Phillip H. Pollack III, The Essentials of Political Analysis: Second Edition (Washington, D.C.: Congressional Quarterly Press, 2005), 7.

[7] Pollack, 8.

[8] Pollack, 11.

[9] Pollack, 13.

[10] Pollack, 131.

[11] Pollack, 133.

[12] Pollack, 154.

[13] Pollack, 168.

[14] Pollack, 154.

[15] Pollack, 155.

[16] Pollack, 16.

[17] Pollack, 15.

[18] Pollack, 17.

[19] Berg, 217.