At the All-In Summit, investor Bill Gurley placed three historical cases side by side — not to make a point about technology, but to ask a harder question about institutions.
The Appendix F Gap — Public Record
1 in 100
Estimated catastrophic failure rate per flight — working engineers
1 in 100,000
Estimated catastrophic failure rate per flight — management
3 orders of magnitude
Gap between the two estimates — inside one organisation, about one vehicle. Source: Appendix F, Report of the Presidential Commission on the Space Shuttle Challenger Accident.
In a talk titled Bill Gurley: Searching for Feynman, delivered at the All-In Summit, venture investor Bill Gurley arranged three cases — Richard Feynman and the Challenger Commission, the Boeing 737 MAX MCAS system, and artificial intelligence — not as a tour of technological failure, but as three different answers to a single structural question. The talk is available on YouTube.
On the 737 MAX, Gurley said: “So they wrote a piece of software, secret software called MCAS, and they put it in there without telling anybody.” That is his narration from the stage, attributed to him throughout; this piece asserts nothing in its own voice about the cause of any accident, characterises no company’s conduct, and describes no regulatory or legal finding.
On Feynman and Challenger he said: “He staged a stunt on live television. He found some material that was similar to the O-ring and in front of everybody, he dipped it in his cold ice water… and cracked it to show the failure.” And on AI he offered a personal probability comparison: “So, forgive me if I’m unafraid of what some AI guy’s chatbot is going to do to humanity when you can’t tell me what this thing already did. My P(second COVID) is way higher than my pdoom.” That final remark is his stated personal view; this piece takes no position on the origins of COVID-19, names no hypothesis, and neither endorses nor refutes his probability comparison.
The key insight: Gurley’s three cases are not about technology. They are three different answers to one question — when something goes wrong inside a complex system, is anyone inside it both able and willing to establish why? The Feynman demonstration is what “yes” looks like when it costs the person something.
The information needed to correct the estimate was already inside the building. It was not reaching the people who set the number.
The Structural Read
Gurley does not quote Appendix F to the Report of the Presidential Commission on the Space Shuttle Challenger Accident — but the document is the sharpest version of his point, and it is worth supplying. Feynman’s appendix discusses estimates of catastrophic failure per flight ranging from roughly 1 in 100 among working engineers to roughly 1 in 100,000 from management, which Feynman described as evidence of a lack of communication between management and working engineers. Neither figure is offered here as the correct probability. The structural observation is about where the disagreement sat.
It was not between the organisation and an outside critic — the familiar shape of a safety argument. It was between two parts of the same organisation. The information needed to correct the official number was already inside the building. It was not reaching the people who set it. That is an observation about the location of the information, not a claim about anyone’s competence or motives.
The MCAS passage, handled with the restraint it deserves, is about disclosure rather than software. In Gurley’s account, a control behaviour was added and, in his words, put in “without telling anybody.” The general observation available without adjudicating anything further is narrow and worth stating precisely: a behaviour that has not been disclosed cannot be anticipated by the person who must respond to it. That is a property of the disclosure, not of the code. It generalises well past aviation and past any particular system.
Verification Is an Institutional Property, Not a Technical One
The Permission Layer framework is conventionally applied to who is allowed to inspect a system — regulatory bodies, audit rights, licensing conditions. Gurley’s argument cuts across that layer at a different angle. An entitlement to look is an institutional arrangement. The willingness to look adversarially, and to be unpopular for what you find, is a disposition. The second does not follow from the first. Feynman’s ice-water demonstration was not a new finding about O-ring physics. Its value was that it made an existing finding impossible to smooth over — in public, in front of people who would have preferred it smoothed. That is the disposition, not the arrangement.
Applied to the current debate about who may inspect AI systems — a debate Gurley is not addressing — the structural reading is this: much of the argument concerns entitlement. Who holds the right to audit, how often, at what layer of the stack. Gurley’s contribution is orthogonal and sharper. A right of inspection is worth nothing unless somebody actually exercises it, adversarially, and accepts the social cost of the finding. No connection is claimed here between his talk and any specific regulation, proposal, company, or incident, and none is named. This piece takes no position on whether AI risk is overstated or understated.
Three Implications
IMPLICATION 1 — The Internal Disagreement Is the Diagnostic
The Appendix F gap — three orders of magnitude, one organisation, one vehicle — does not describe a failure of external oversight. It describes a failure of internal information flow. The correcting data existed. It did not reach the decision-makers. For any complex system governed by layered institutions, the more useful diagnostic question is not “does a regulator have access?” but “does the number being used at the top of the organisation reflect what the people closest to the system actually believe?” When those diverge significantly, the arrangement already exists; the information is simply not moving through it.
IMPLICATION 2 — Disclosure Is Prior to Anticipation
Gurley’s MCAS observation, as attributed, reduces to a point about the sequence of knowledge. A behaviour that has not been disclosed cannot be anticipated by the person required to respond to it. This is a property of the disclosure gap, not of the behaviour itself. It generalises directly to any system — software, financial, biological — where the operator and the respondent are different parties. The structural question for any governance framework is therefore not only “can we audit this?” but “has the existence and nature of this behaviour been communicated to the people who will be first to encounter its consequences?”
IMPLICATION 3 — The Right to Look and the Willingness to Look Are Separate Problems
The Feynman demonstration, in Gurley’s telling, was not significant because Feynman had been granted access. He had access. Its significance was that he used it in a way that made the finding undeniable and public, at personal cost. The institutional design problem this surfaces is that verification regimes — audit rights, inspection frameworks, disclosure obligations — solve the access problem. They do not automatically produce the disposition to exercise that access against the resistance of the institution being examined. Building the right does not build the person willing to use it. That is a different design challenge, and it sits at the level of incentives and culture rather than rule-writing.
Business Engineer Framework
The Permission Layer — Access Rights vs. Verification Disposition
The Permission Layer maps who controls which AI capabilities — regulators, platforms, governments. Gurley’s argument exposes the layer beneath it: the institutional disposition to exercise that control adversarially when the finding is inconvenient. The Business Engineer Map of AI places this tension across the full stack, from infrastructure to governance. Understanding where the disposition problem sits — and why it is separate from the access problem — is the analytical starting point.
Gurley’s three cases converge on a single structural point: verification is not a property of the technology being examined, nor of the formal arrangement that permits examination — it is a property of the institution that must actually exercise it, against resistance, in public, at cost. Appendix F puts the sharpest number on the underlying failure: three orders of magnitude of disagreement, inside one organisation, about one vehicle, with the correcting information already present and not moving upward. The question that generalises from all three cases to any complex system — including AI — is not whether the right to look exists. It is whether anyone is both positioned and willing to make the finding impossible to smooth over.
This is businessanalysis only. It is not investment advice, not legal advice, and not public-health or safety-engineering advice. No view is expressed on any security, and no recommendation is made.
The account of the Boeing 737 MAX above is Bill Gurley’s narration from a stage, quoted and attributed to him. Nothing above asserts in this publication’s own voice what caused any accident, characterises any company’s conduct, describes any regulatory or legal finding, or names anyone affected. Mr Gurley’s remark comparing probabilities is his stated personal view. Nothing above takes any position on the origins of COVID-19, names any hypothesis, cites any study or agency, or says whether any explanation is established, contested or absent, and nothing above endorses or refutes his comparison. The referent of “some AI guy” is unspecified, and no person or company is named above as its subject. The probability estimates cited are the figures discussed in Richard Feynman’s Appendix F to the Report of the Presidential Commission on the Space Shuttle Challenger Accident. Neither is presented above as the correct probability of failure. Nothing above takes a position on whether AI risk is overstated or understated, claims any connection between this talk and any regulation, proposal, company or incident, or claims that any institution is or is not capable of investigating itself. Nothing is predicted. This is businessanalysis. It is not investment advice, not legal advice, and not public-health or safety-engineering advice; no view is expressed on any security, and no recommendation is made.
Gennaro is the creator of FourWeekMBA, which reached about four million business people, comprising C-level executives, investors, analysts, product managers, and aspiring digital entrepreneurs in 2022 alone | He is also Director of Sales for a high-tech scaleup in the AI Industry | In 2012, Gennaro earned an International MBA with emphasis on Corporate Finance and Business Strategy.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
