Sustained Overtime: Skeptics and Bad Arguments
Science and data vs debate tactics
TL;DR - Skeptics of research on the consequences of sustained overtime generally rely on debate tactics and not countervailing research. They are not arguing to determine what’s true, but arguing to win . Let’s deconstruct an instance of this behavior.
INTRO
I recently wrote a survey article describing over 100 years of data showing that sustained overtime is often a mistake for your team. The article evoked a comment that seemed innocuous on the surface but could also be interpreted as several fallacies of argument: anecdote, appeal to authority, and an implicit ideology.
The comment:
In an interview with John Carmack, he said he worked for 60 hours a week (12 hours a day) for years and he felt pretty good about it. So I guess some people might be outliers.
I did say in the article that there are probably individual outliers, much like the very small number of people (1%-5%) who can function on 6 hours of sleep or less . So at first read it seems like the commenter was agreeing with me.
But then I realized that the commenter may not have read the full article and was more likely trying to use John Carmack, a famous person, as a counterexample and appeal to authority .
As an honest scientist I feel bound to seriously consider criticism, so I investigated.
I’m not trying to be mean or unfair to Carmack, but that investigation put some of his public statements under a microscope.
WHO SAID WHAT
I don’t know which interview the commenter refers to but I found a likely candidate interview on YouTube from August, 2022.
More importantly, my search revealed a 2016 online discussion of the topic that’s more interesting because it involves a number of participants besides John Carmack.
That online conversation is the focus of our analysis.
If you like original sources, read the complete, original comment thread from 2016 in which Carmack responded to someone sharing a link in Hacker News to an article in the Guardian, “Why working fewer hours would make us more productive ”.
Then next thing you should do is read the exceedingly thorough and polite rebuttal to John Carmack’s commentary by Glyph Lefkowitz .
DEEPER ANALYSIS
There is a lot going on in that thread and in Lefkowitz’s rebuttal. The short version is that Carmack took the role of skeptic to research on overtime versus a number of other people.
My own investigations previously revealed a pattern: skeptics don’t usually offer counter evidence, by which I mean research results. Instead they use rhetoric and debate tactics to discredit results. For example, implying that we just need a better work environment, better management or supervision, better logistics, or better self awareness of our own capabilities.
Carmack’s position followed this pattern, so let’s take a look.
Debate Tactics
I’ll start by saying that I have a great respect for John Carmack’s accomplishments. His games were foundational for where the game industry is today. He’s a smart person, and I am not looking for a fight.
But his arguments in that thread were largely debate tactics that amount to arguing to win a debate .
When I started going through the text and making a list of debate methods I realized that there were a whole lot of them. I’m lazy, so I cheated and fed Carmack’s argument text to ChatGPT (GPT?4o, I think; could be 3.5) with the following prompt:
ChatGPT identified a list of logical fallacies and quoted text fragments for reference. You can do the same (GPT-4o is free to use) so I won’t copy/paste the whole thing.
It identified the following fallacies that are arguments in the absence of evidence:
领英推荐
You can get definitions for each from a number of sources, but Wikipedia is a go-to for me because it’s easy. Here is that link .
As a check, I performed the same exercise with Google Gemini , which is also free. Gemini identified a list of ten tactics that were a subset of the ones ChatGPT listed, albeit using some different labels. Gemini used Analogy in place of Anecdote and Red Herring in place of Hypothetical Scenario.
ChatGPT also pointed out an Appeal to Authority, but I exclude that one because it was for discussing references to relevant research. ChatGPT is not perfect, which is why I always double check its work.
Maybe it would do a better job if I paid for premium service. I may be lazy, but at least I’m cheap. ??
Being Fair
All writing, including science writing, uses persuasive language that can fall into classes of debate tactic. After all, there is an exceedingly long list of methods of argument. But we also need to clarify that some tactics are fact-based and logical, while others are meant to sway people in the absence of facts and logic.
To be fair, I fed my own article on the consequences of overtime into ChatGPT using the same prompt. ChatGPT identified 10 debate tactics but only one that might be considered persuasion in the absence of evidence: Appeal to Common Sense. This is where I proposed fatigue as a source of reduced productivity.
What matters is not whether tactics are used, but whether they are used in a fair and above-board way to support a logical argument with defensible data.
OUTRO
I think it’s true that people who are passionate and engaged tend to work longer and harder. It is also possible that they can achieve more than someone who is not passionate and driven.
But the cultural forces promoting sustained overtime for everyone don’t rely on data and research to support the practice. They rely on rhetoric, aka debate tactics.
Perhaps more importantly, and an issue that also motivated this article, if you hold up icons of our industry like John Carmack and tell people “he said he worked for 60 hours a week (12 hours a day) for years and he felt pretty good about it”, you will absolutely encourage new folks in the industry to do the same, probably to their detriment.
I have never met John Carmack, and perhaps his position on this topic has changed. But his opinions about sustained overtime shouldn’t be ignored because the magnitude of his social influence can serve as a bad example for others.
REFERENCES
?? Hacker News forum at YCombinator with John Carmack’s comments about overtime
?? Lex Fridman podcast on YouTube: Day in the life of John Carmack
?? Blog post by Glyph Lefkowitz in response to Carmack’s comments
?? Robin’s Substack: Sustained Overtime is Often a Mistake
?? Robin’s Substack: Arguing to Win & One-Way Doors
?? The Guardian: Why working fewer hours would make us more productive
?? Google’s LLM: Google Gemini
?? OpenAI’s LLM: ChatGPT
?? Scientific American: Why Do Some People Need Less Sleep? It’s in Their DNA (2019)
?? Wikipedia article describing appeal to authority
?? Wikipedia article on Social influence
?? Wikipedia article on Logical fallacies