The next page of the PhD – the examiner feedback is in…

The examiner’s feedback is in for my PhD thesis. I figure this is as good of an opportunity as any for an update as my two… I mean three… um, four… OK, five-year journey nears its conclusion.

The examination process

For those not familiar with the process, PhDs are not so much ‘graded’ as they are reviewed by a number of external examiners who then provide feedback and recommendations for the student to pass or amend their thesis. The review is ‘blind’ in that the PhD candidate does not know the reviewers and the reviewers do not know the candidate. Some schools have an additional session to ‘defend’ the thesis in front of a panel of experts, but this was not the case in my situation.

By the time the thesis gets to the examiner, it has gone through multiple reviews with your supervisors and you have spent countless hours writing and rewriting between 80,000 to 150,000 words. Core chapters become the appendix, later relegated to a past revision as you ‘murder your darlings‘ – remove sections that you love but do not support the main premise of your research. Some candidates publish in journals as part of the process to get additional feedback along the way. I recommend this approach based on my experience of having not published. That said, I was not clear on my position until close to the very end and am only now feel I am in a position to publish.

This explanation is to set the stage for my growing anticipation from the time I last posted about my PhD in October 2020, submitted my thesis for review in December, and then receiving the examiners’ feedback last week. I had all intentions of maintaining a more public diary of the process. For most of the past several months, however, there would have been little in the way of a practical update.

To recap those just tuning in, the journey has included:

• the development of the question,
• the launch of the research tour,
• an update four weeks into the tour,
• reflections after the tour,
• thoughts as I analysed the research data, and
• reflections as I wrapped up the main thesis.

There is likely one more post after this in the series once I graduate. I share my results here for transparency, to update any who might still be following along, and for others who might learn for their own candidacy journey.

The feedback

A ‘hefty’ thesis

I first pitched my PhD idea as a driving tour around Australia visiting every innovation ecosystem. This was overly ambitious to the point of unrealistic and I was thankfully encouraged to limit the scope to Queensland. That said, I am grateful to have visited a large number of regions in each state and territory over the past five years.

Both examiners commented on the scope and scale of the theses, which reflects my own initial thoughts that a PhD was reflective of a life’s work. And yet as I neared the end of my journey, my supervisors drummed it into me that a PhD is only the beginning of a longer trek into continuous personal, professional, and academic discovery.

My thesis is an interrogation into each aspect of three constructs – uncovering everything I could know on the nature of the innovation ecosystem, what we know about innovation hubs, and what is meant by community resilience. The more I learned, the more nuance I discovered. Each line of questioning would uncover entire new lines of reasoning or bodies of research, like pulling a thread on a never-ending tapestry of collected knowledge. Even in my examiner’s feedback, they identified entire lines of reasoning and foundational authors I had not come across.

The thesis was a constant tension between putting in too much and needing each link in what I believe to be an important causal chain between innovation activity and community impact. I wavered between the two perspectives, but appreciate the examiners’ feedback as validation that I landed at an acceptable point somewhere in the middle.

Examiner 1: This is an impressively ambitious thesis that seeks to answer questions on the relationships between innovation hubs, innovation ecosystems and community resilience. These are timely questions given the recent theoretical turns towards the notion of ecosystems in the entrepreneurship and innovation literature and enthusiasm to fund interventions in these ecosystems by policy makers. In exploring these themes, the thesis marshals an impressive amount of data, comprised of 147 interviews across 16 regions of QLD.

Examiner 2: This thesis is unusual, unusually good, in its ambition, the amount of effort and attention-to-detail in data collection and analysis, and its sheer ‘size’…not in pages, but in scholarly heft. There are a huge amount of ‘chunks’ of knowledge, either organized in new ways (the work on typologies here is ‘dull’ work but extremely helpful), presented in new ways, or genuinely new in and of themselves, far in advance of most modern PhD theses. So it’s clear even from a cursory reading that this thesis is absolutely worthy of a doctoral award.

Overall, superb work here, and indicative of a researcher who was prepared to go the extra (many) miles. The innovation mapping process is of sufficient detail and rigour to be a worthwhile academic output in its own right, and it is worth saying that this thesis, in terms of ‘heft’ is more than the usual ‘size’ of thesis, particularly in the 21st century. This thesis represents an example that could have been drawn from the 1980s when theses were expected to offer a substantial advance on knowledge. The thesis also offers something of a ‘manual’ for future researchers in the field, and I hope the thesis is rapidly published in forms that make it highly accessible to scholars.

The theoretical position

I have come to refer to my theoretical position as my theory ‘stack’. This is inspired by my software agency background where software is build on a ‘technology stack’ of various database, presentation, and processing platforms. For my thesis, my ‘stack’ integrated theory and method, using systems theory, actor network theory, critical realism, appreciative inquiry, and social network analysis.

Each of these lines of reasoning and methods is appropriate for situations of uncertainty and disruption, but there was limited documentation I came across on the integration of all of these for research. I struggled early with structured models of ecosystem maps that did not seem to match reality. I am grateful to a mentor early on who shared actor network theory with me that legitimised an approach that treated everyone and everything as a unique actor, while critical realism aligned with my postmodern leanings and allowed for multiple, equal realities of what the data indicated, what could be observed, and people’s lived experiences.

As noted by the examiners, this will take further refinement and needs to be tested against current approaches by leaders in the field.

Examiner 2: The conceptual model doesn’t make the mistake of packing too much in, or piling on the complexity that is (genuinely) present in the contexts it describes. It takes the key elements and puts them together in a logical way that will be quite testable and usable by scholars in future.

Examiner 1: First, the work is theoretically very ambitious in its attempt to integrate a number of theoretical lineages within a broader systems framework. To some extent this is welcome (if brave) because the notion of an ecosystem when applied to social phenomena does invite some theoretical complexity in engaging with it. So while I enjoyed some of the conceptual work displayed in trying to integrate systems theory, critical realism, actor network theory, appreciative inquiry (and even briefly role theory), this approach may well raise some eyebrows (ie cause problems) among reviewers of top journals.

Introducing a novel theoretical framework while also trying to present original empirical work, sustain an argument that warrants a significant conceptual contribution to a targeted area of theory is challenging indeed. In the context of the thesis, it meant significant space was devoted to explaining how these frameworks fit together. With hindsight, a straighter line could have been drawn between your conceptual framework, your empirical material and the novel insights of your findings. After reading the thesis closely, it was not clear why adopting the approaches outlined by leading ecosystems researchers such as Ben Spigel and Erik Stam would have led to inferior or less significant insights.

Are we explaining what or why? And can we establish cause?

The core of my thesis is in identifying what, if any, contribution the innovation ecosystem, and specifically the innovation hub, has on indicators of community resilience. I take on board the feedback about the challenges with inferring cause from the interviews as well as not identifying underlying causes. In my defence, I reflect on the personal and national context in which I started my research.

I began my PhD when I was managing what was possibly the first dedicated innovation hub in Australia that was fully owned and operated by a local government council. Over 70 additional innovation hubs would be created in Australia over the course of my research. This is more than double the number that started in the five years prior to my research. The ecosystem was emerging and there were many unanswered questions both for me personally as well as for the broader community.

My main question was whether an innovation hub had a contribution towards community resilience, whether that contribution was enabling or inhibiting, and what was the nature of that contribution. I ended up identifying over 60 roles in the ecosystem that contributed to over 80 community resilience indicators through over 120 enabling and inhibiting contributing factors. Further, the emphasis on the innovation hub resulted in correlating 21 functions of the innovation hub to contributing factors.

As soon as you see the information presented, the next logical question is “but why?” In what situations does the innovation hub have an influence? Does the hub business model matter? What about other roles in the ecosystem? And what is the influence of culture and funding?

These are all important questions for follow-up research now that a baseline foundation has been established. The interview data also needs correlation with additional data sources and interviews in additional regions. I take the examiners’ feedback on board and look forward to expanding the model. As one examiner aptly noted: “I think the next steps here is to work out a more targeted contribution for the insights contained in this literature – how do the insights developed through your empirical work either challenge or add something new to the core ideas outlined across this work?“

Examiner 2: I would say, however, that there is one important limitation missing here. Apart from the ‘natural’ limitations (and strengths) of a qualitative approach, there is no mention here of the cross-sectional nature of the study. Any attempt by the candidate (and there are a number of them) to infer causality are made on poor foundations. The approach relies on participant recollection and perception, with all the sense-making flaws of complex reality, and the systematic biases (e.g. in recall) that that approach ‘naturally’ has. There are times when the candidate brushes over this aspect of his approach and uses language that suggests causation has been more or less established.

Examiner 1: At times, such presentations run the risk of seeming to argue that ‘everything is related to everything else’, and that each actor/role can either be enabling or inhibiting depending on context and other factors. This is, to be fair, one of the more vexing challenges of examining complex social phenomena (I believe there’s an old joke about the so called ‘first law of sociology: sometimes they do, sometimes they don’t). However, I am inclined to agree with Stam’s (2015) ‘sympathetic critique’ of ecosystem approaches: (and note that you cite this paper in your thesis): “the approach as yet only provides a long laundry lists of relevant factors without a clear reasoning of cause and effect. These factors do provide some focus, but they offer no consistent explanation of their coherence or their interdependent effects on entrepreneurship” (Stam, 2015: 1764).

In other words, we already know that the various ‘factors and actors’ that comprise ecosystems can be related to each other in all sorts of different configurations and nonlinear ways, but I believe to really make an important contribution in this field requires presenting plausible accounts of relations of strength or significance between some factors/actors over others, or a richer account of a particular context that can help illuminate why this specific innovation hub or ecosystem has this particular relationship with community resilience (eg as a case study or ethnography that adequately paints the historical/cultural/social/economic factors of a particular place). For all its theoretical breadth and empirical scope, the thesis often presents as somewhere in between these two poles– too abstracted to offer a rich sense of a particular hub or place (eg a multi-year case study), and yet too cognisant of local contingencies to advance generalisable claims about the relations between key variables that might hold across contexts.

Next steps – publications, policy, and practical

I am soon to be in that state of purgatory that some PhD candidates find themselves between thesis completion and journal publication. I know I am sitting on a large body of work with several publications that will naturally emerge. Both examiners commented on the need to publish, which add a prompt for me to action.

The publication process will also come with refinement of the application for policy makers and practitioners. Turning the body of work into actionable chunks will be part of the next process. It will also emerge naturally as I test the results on industry peers who work in the field.

Examiner 1: If you haven’t already begun this process, I would encourage you to work some of the key insights into publications for some of the leading scholarly journals that discuss these questions, such as Small Business Economics, Technovation, Research Policy, Strategic Entrepreneurship. Entrepreneurship Theory and Practice. Journal of Business Venturing. Technological Forecasting and Social Change. Journal of Business Research.

Examiner 2: Indeed it is a surprise and something of a disappointment to note that during the production of the thesis, there is no evidence that the candidate published from his thesis. It’s not a major issue, but hopefully this is addressed shortly.

The journal writing process is new to me and has challenged my previous habits of regularly sharing through blog posts and social media. I struggle with the balance of crafting knowledge into a format for a peer-reviewed journal and releasing information for immediate feedback and value. I also feel the tension between writing up the results and being in the field applying the knowledge. This will be my next journey over the course of the next 12 months, to develop the pipeline of papers, get the thesis into a publishable book format, and apply the lessons in community.

I have had several people who are keen to read the results, and I appreciate their interest. I look forward to sharing more broadly as the results are published.

Thank you and give back

The examiners’ feedback applies to a large extent to my supervisors and the community who contributed to the work through their interviews, passion, honesty, and hospitality. I am grateful for their patience as I get to this stage and look forward to sharing more throughout the year.

I am eternally grateful to my supervisors Retha Wiesner, Char-lee McLennan, and Aastha Malhotra, along with Ben Lyons for the support. Also the team at the University of Southern Queensland Rural Economies Centre of Excellence, and QUT's Australian Centre for Entrepreneurship Research.

That gratitude translates to giving back and collaboration with others on the journey. A PhD is a challenging endeavour. I would not be here if not for the help of those who have gone before me. If you are somewhere on the journey and are studying in a related field, I am always happy to share whatever might add value to your path.

Thank you for reading this far and being a part of the journey. I look forward to learning, sharing, and contributing with you where we might work together on impact.