Response to McKinsey: Measuring developer productivity? Episode #62
Stephan ?? Schmidt
CTO ⊕ AI Coach ? #1 book "Amazing CTO" ? Podcast Guest ? Keynote Speaker ? ex-eBay ? ex-ImmoScout ? Helping CTOs with everything - just ask!
By Stephan Schmidt
Happy ?? Tuesday,
This week’s insights
Good reading, have a nice Tuesday ?? and a great week,
Stephan #CTO Coach and ex-CTO
?? If You Only Read One Thing
The McKinsey report about developer productivity I’ve linked to last time made some waves on the internet. Kent Beck, who thought about developer productivity before most current developers were born, has something to add. And there are many great points about productivity in his article. Go read it up, so you have some ammunition when being called out. The core argument against the McKinsey view is about “Input < Output < Outcome < Impact”. I use that as a categorizer of goals all the time myself and you should too. Input (what he calls effort) is the worst, only use it if you have nothing else to measure success with. Lines of code is an input metric. Features are an outcome metric. Impact is the best category. Aim here. And McKinsey is mostly about “Input” metrics, whereas SPACE and DORA are Outcome and Impact metrics. And input is easier to measure, and managers don’t know what those numbers mean, so of course they will tend towards input metrics.?MUST READ
?? AI
Ask the same questions to many LLMs. Not only is this very interesting to understand differences between models, but also between sizes of the same model, like Falcon 40B vs Falcon 7B. I did like “Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? Let’s think step by step.” Especially “So the number of sisters Sally has is 6.” HA take that logic!—funniest is?“erm.. 3”?- but there quite are a lot funny ones for 9, 12, 24. But there is the right answer in the results (1) somewhere.
A GPT-4 Capability Forecasting Challenge (12 minute read)
It is difficult to know what at this point in time, AI can do and can’t do. This small game challenges you to forecast what tasks GPT-4 can succeed with.?It helped me understand that the power of GPT-4 is nuanced. Must read
Apple is reportedly spending ‘millions of dollars a day’ training AI (3 minute read)
“Those involved in its development tell The Information that Apple’s most advanced LLM, known internally as Ajax GPT, has been trained on “more than 200 billion parameters” and is more powerful than OpenAI’s GPT-3.5”?200 billion is a large model, larger than the 180B Falcon model.
Do people move to the app? Do people use IDE integrations? Do people use the GPT API? Or did the novelty wear off? The article bases its conclusion on data from Similarweb, a company that estimates web traffic.?“Worldwide desktop and mobile website visits to the ChatGPT website decreased by 3.2% to 1.43 billion in August, following approximately 10% drops from each of the previous two months.”
It is a little slow on tokens/sec, and quantized down, but seems to work nicely. It does this by using the CPU and RAM it seems. Not clear if inference (not training) will be a CPU or GPU task in the future. Or if GPUs get massively more RAM, because LLM models get bigger by the week.
Startups move aside, here comes a company with deep pockets. Is copyright the factor which decides about SaaS AI success??“As customers ask whether they can use Microsoft’s Copilot services and the output they generate without worrying about copyright claims, we are providing a straightforward answer: yes, you can, and if you are challenged on copyright grounds, we will assume responsibility for the potential legal risks involved.”
领英推荐
https://blogs.microsoft.com/on-the-issues/2023/09/07/copilot-copyright-commitment-ai-legal-concerns/
Stories I’ve Enjoyed This Week
I’d argue most of the talk in the article—“Force yourself to write down a single clear goal which is stretching but achievable in five years.”?is about visions not strategy, but the points are valid. I talk to startup CEOs and CTOs, and they have no strategy. Therefore, features are all over the place, work is chaos, and success seems random. “Why startups do need strategy” gives some good step by step advice on how tyo achieve a strategy - and as CTO you should have a tech strategy too, where is tech in 5 years?
Similarweb says GPT traffic drops (see other article), The Verge sees the end of Google. Google dominated the way we saw the internet - everything through the Google lens. Then through a social media lens. Is AI next? But more to you the CTO: What if SEO traffic of your company drops 90%. Can it survive? What is plan B? Not your problem, but what if marketing comes screaming?
Most CTOs I know neglect working with their peers on the management team.?“Working as a “first team” with your fellow leaders is a mindset and operational mode in which you prioritize supporting your peers instead of your direct reports.”?First team looks extreme when looking at it on a management level, but there are good insights for CTOs. First, you’re in a management team, you’re no longer just a techie. The VP of Marketing and the CFO are your peers. Second, create a management team of your direct reports. And do “first team” with them.?This was the most successful thing to make my CTO jobs easier, happier and more successful.
“The UK’s worst air-traffic outage in a decade was caused by an anomaly in the airspace manager’s software system, which confused two geographical checkpoints separated by some 4,000 nautical miles. [..] The glitch triggered a shutdown of the software system run by NATS for safety reasons, according to a preliminary report”?Again a crisis happens because two problems interact. And how does your system work with bad input? And do you Fuzzing to find bad input? No? Start.
I’m a fan of being a servant leader—not. Well again I am. Can you make up your mind, please? What is a servant leader? “Servant leadership is a leadership style that empowers employees based on the premise that if employees are satisfied and cared for, they will be more motivated to give their best.” Agreed. But the terminology is manager. I do agree as a manager people on your team are your biggest lever for success. And if you care for them (and respect them, and you’re loyal to them), they will give their best. But we need to stop throwing around manager and leader interchangeably. A leader is someone who leads. Doh. A leader points out where to go, and then leads people there. But yes, managers should be servant managers.
Engineers neglected security for too long. This will become cumbersome: “at its essence it is no small thing that the White House proposes: legislation instructing people on how to code their software and requiring adherence to those instructions.”*
Largest car manufacturer in the world. Doh.?“The system malfunction was caused by the unavailability of some multiple servers that process parts orders. As for the circumstances, regular maintenance work was performed on August 27, the day before the malfunction occurred. During the maintenance procedure, data that had accumulated in the database was deleted and organized, and an error occurred due to insufficient disk space, causing the system to stop. Since these servers were running on the same system, a similar failure occurred in the backup function, and a switchover could not be made.”?Tell this your CEO so she is mentally prepared when you make the same mistake?I know, you won’t, until you do.
We assume emotions are universal. They are not. Emotions are those for which we have word. Germans have different emotions than people in the US-perceived ones. With more international teams due to remote work, keep this in mind. People on your team might have emotions you don’t have. And you have some, they can’t express.
?