Goodnight Wiki / Software Engineering

Software Engineering

If the Programming Languages section asks how we design the tools, this section asks what happens when we use them — at scale, under pressure, over decades, maintained by strangers. The picture isn't pretty, but it's honest: software engineering is the art of building systems beyond our ability to verify, and the history of the field is a history of strategies for coping with that gap.

The Correctness Spectrum

Software Correctness At Scale frames the problem: 100 million lines of code in cars, 10 million failure paths in Toyota's onboard computer, 911 systems brought down by a single counter overflow. The shuttle software group represents one extreme — 420,000 lines, one error per version, 40,000 pages of specs, $35 million per year — proving that near-perfect software is achievable at a cost nobody else will pay. Paradox Of Automation shows what happens at the other extreme: Air France 447's pilots couldn't fly their own plane because the automation that protected them from small errors created vulnerability to catastrophic ones.

Liveness In Programming and Immediate Feedback argue that the fundamental problem is the gap between code and behavior — we work blind, maintaining mental models of invisible state, and the tools for closing that gap (live programming, Dynamicland, end-user programming) remain prototypes rather than products.

Infrastructure and Maintenance

Software As Infrastructure is the section's most politically charged article. COBOL was designed to be readable and has run credit card transactions for sixty years — and was scapegoated during COVID because a language readable by managers threatens programmers whose status depends on code being hard to read. The Bezos mandate (all teams must expose service interfaces) created AWS. The self-automating workers who hide their scripts because the employer would capture the gains connect directly to The Luddite Question.

Software Teams As Learning Systems reframes the unit of software creation from the individual developer to the learning system of people, code, and tools — the Camerata pattern where "great teams make great people," not the reverse. ABI Stability reveals how deployment models constrain language design: Swift threads the needle of dynamic linking through runtime indirection, while Rust embraces monomorphization and accepts that dynamic linking isn't its problem.

Data, Search, and Protocols

Log Structured Data proposes turning the database inside-out: the transaction log as source of truth, everything else as derived views. Discord's journey from MongoDB to Cassandra to ScyllaDB illustrates the practical challenges. Search And Indexing covers the tricks for avoiding reading everything: trigram indexes, ripgrep's SIMD-optimized literal extraction, the OEIS as search engine for mathematics, Bayesian spelling correction in 36 lines of Python.

Network Protocols In Practice is networking as practitioners encounter it: NAT traversal as the real problem, Tailscale's architecture for making WireGuard work through corporate firewalls, the dialup modem handshake as audible network engineering, and Bell Labs' lost audio compression killed by a lawyer who didn't know who the Residents were. Data Format Design catalogs the pathologies: YAML's Norway problem, the DOM's conflated layout properties, the social graph that is neither social nor a graph.

Trust and Security

Supply Chain Trust maps the chain of strangers between you and every program you run: event-stream's compromised maintainer, Merkle tree logs, reproducible builds, and the vertiginous question of whether hardware can ever be trusted. Zero Click Exploits is the apex predator: FORCEDENTRY building a Turing-complete virtual computer inside a JBIG2 image codec, the PGP problem (technically unbroken, practically unusable), and mercurial CPU cores producing silent wrong results at hyperscale.

Bootstrapping And Trusting Trust is the section's most philosophical article: how do you get from a 280-byte hex monitor to a running Linux system? From vinyl bootloaders to Actually Portable Executables to QEMU's tiny code generator, each project strips away another assumption about what "running a program" requires.

Cryptography As Capability shows the social side: 18th-century secret societies using elaborate ciphers to incubate democratic governance, the Copiale manuscript's Russian-doll encryption, and Shamir secret sharing nearly destroying PayPal because getpass() silently truncated on Solaris.

Rust In Practice gives the practitioner's verdict: the refactoring superpower is real, the syntax tax is real, the supply chain anxiety is real, and the complexity doesn't go away — it gets redistributed into dense syntax and fighting the borrow checker on legitimate patterns.

What Connects Outward

The SWE section bridges to Systems That Eat Themselves through Gall's laws about complex system failure. To Legibility And State Power through the parallel between High Modernist urban planning and "legacy" system replacement. To Compiler Correctness and the PL section through the verification/testing spectrum. And to every article about coordination failure — because software engineering is, at its core, the problem of coordinating human understanding across teams, years, and millions of lines of code that nobody fully comprehends.

Open in stacked reader →