Max Headroom 
UNCLASSIFIED
My father spent his career as an accountant for a major public utility. He didn’t talk about work much; when he engaged in shop talk, it was generally with other public utility accountants, and incomprehensible to those who weren’t. But I remember one story from work, and that story is relevant to our current engagement with AI.
He told me one evening about a problem at work. This was the late 1960s or early 1970s, and computers were relatively new. The operations division (the one that sends out trucks to fix things on poles) had acquired a number of “computerized” systems for analyzing engines—no doubt an early version of what your auto repair shop uses all the time. (And no doubt much larger and more expensive.) There was a question of how to account for these machines: Are they computing equipment? Or are they truck maintenance equipment? And it had turned into a kind of turf war between the operations people and the people we’d now call IT. (My father’s job was less about adding up long columns of figures than about making rulings on accounting policy issues like this; I used to call it “philosophy of accounting,” with my tongue not entirely in my cheek.)
My immediate thought was that this was a simple problem. The operations people probably want this to be considered computer equipment to keep it off their budget; nobody wants to overspend their budget. And the computing people probably don’t want all this extra equipment dumped onto their budget. It turned out that was exactly wrong. Politics is all about control, and the computer group wanted control of these strange machines with new capabilities. Did operations know how to maintain them? In the late ’60s, it’s likely that these machines were relatively fragile and contained components like vacuum tubes. Likewise, the operations group really didn’t want the computer group controlling how many of these machines they could buy and where to place them; the computer people would probably find something more fun to do with their money, like leasing a bigger mainframe, and leaving operations without the new technology. In the 1970s, computers were for getting the bills out, not mobilizing trucks to fix downed lines.
I don’t know how my father’s problem was resolved, but I do know how that relates to AI. We’ve all seen that AI is good at a lot of things—writing software, writing poems, doing research—we all know the stories. Human language may yet become a very-high-level, the highest-possible-level, programming language—the abstraction to end all abstractions. It may allow us to reach the holy grail: telling computers what we want them to do, not how (step-by-step) to do it. But there’s another part of enterprise programming, and that’s deciding what we want computers to do. That involves taking into account business practices, which are rarely as uniform as we’d like to think; hundreds of cross-cutting and possibly contradictory regulations; company culture; and even office politics. The best software in the world won’t be used, or will be used badly, if it doesn’t fit into its environment.
Politics? Yes, and that’s where my father’s story is important. The conflict between operations and computing was politics: power and control in the context of the dizzying regulations and standards that govern accounting at a public utility. One group stood to gain control; the other stood to lose it; and the regulators were standing by to make sure everything was done properly. It’s naive of software developers to think that’s somehow changed in the past 50 or 60 years, that somehow there’s a “right” solution that doesn’t take into account politics, cultural factors, regulation, and more.
Let’s look (briefly) at another situation. When I learned about domain-driven design (DDD), I was shocked to hear that a company could easily have a dozen or more different definitions of a “sale.” Sale? That’s simple. But to an accountant, it means entries in a ledger; to the warehouse, it means moving items from stock onto a truck, arranging for delivery, and recording the change in stocking levels; to sales, a “sale” means a certain kind of event that might even be hypothetical: something with a 75% chance of happening. Is it the programmer’s job to rationalize this, to say “let’s be adults, ‘sale’ can only mean one thing”? No, it isn’t. It is a software architect’s job to understand all the facets of a “sale” and find the best way (or, in Neal Ford’s words, the “least worst way”) to satisfy the customer. Who is using the software, how are they using it, and how are they expecting it to behave?
Powerful as AI is, thought like this is beyond its capabilities. It might be possible with more “embodied” AI: AI that was capable of sensing and tracking its surroundings, AI that was capable of interviewing people, deciding who to interview, parsing the office politics and culture, and managing the conflicts and ambiguities. It’s clear that, at the level of code generation, AI is much more capable of dealing with ambiguity and incomplete instructions than earlier tools. You can tell C++ “Just write me a simple parser for this document type, I don’t care how you do it.” But it’s not yet capable of working with the ambiguity that’s part of any human office. It isn’t capable of making a reasoned decision about whether these new devices are computers or truck maintenance equipment.
How long will it be before AI can make decisions like those? How long before it can reason about fundamentally ambiguous situations and come up with the “least worst” solution? We will see.
AI & ML, Commentary
Radar
