New Tricks for an Old Z-Machine, Part 1: Digging the Trenches

One of the most oddly inspiring stories I know of in all computing history is that of the resurrection and re-purposing of the Z-Machine, Infocom’s virtual machine of the 1980s, to serve a whole new community of interactive-fiction enthusiasts in the 1990s and well beyond. Even as the simple 8-bit computers for which it had originally been designed became obsolete, and then became veritable antiques, the Z-Machine just kept soldiering on, continuing to act as the delivery system for hundreds of brand new games that post-dated the company that had created it by years and eventually decades. The community of hobbyist practitioners who spawned the Interactive Fiction Renaissance of the mid-1990s made the Z-Machine one of their technological bedrocks for reasons more sentimental than practical: most of them worshiped Infocom, and loved the way that distributing their games via Infocom’s venerable virtual machine made them feel like the anointed heirs to that legacy. The Z-Machine was reborn, in other words, largely out of nostalgia. Very soon, though, the hobbyists’ restless creativity pushed and twisted the Z-Machine, and the genre of games it hosted, in all sorts of ways of which even Infocom at their most experimental could never have dreamed. Thus a regressive became a progressive impulse.

In the end, then, a design which Joel Berez and Marc Blank first sketched out hurriedly at their kitchen tables in 1979, in response to the urgently immediate problem of how to move their DEC PDP-10 game of Zork out of the MIT computer lab and onto microcomputers, didn’t fall out of general use as a delivery medium for new games until after 2010. And even today it still remains in active use as a legacy technology, the delivery medium for half or more of the best text adventures in the historical canon. In terms of the sheer number of platforms on which it runs, it must have a strong claim to being the most successful virtual machine in history; it runs on everything from e-readers to game consoles, from mobile phones to mainframes, from personal computers to electronic personal assistants. (To paraphrase an old joke, it really wouldn’t surprise me to learn that someone is running it on her toaster…) Its longevity is both a tribute to the fundamental soundness of its original design and to the enduring hold which Infocom’s pioneering interactive fiction of the 1980s has had upon more recent practitioners of the form. Like so many technology stories, in other words, the story of the Z-Machine is really about people.

One of the more ironic aspects of the Z-Machine story is the fact that it was never designed to be promulgated in this way. It was never intended to be a community software project; it was no Linux, no Mozilla, no Java. The ideological framework that would lead to such projects didn’t even exist apart from a handful of closeted university campuses at the time Berez and Blank were drawing it up. The Z-Machine was a closed, proprietary technology, closely guarded by Infocom during their heyday as one of their greatest competitive advantages over their rivals.

The first order of business for anyone outside of Infocom who wished to do anything with it, then, was to figure it out — because Infocom certainly wasn’t telling. This first article in a series of three is the story of those first intrepid Z-Machine archaeologists, who came to it knowing nothing and began, bit by bit, to puzzle it out. Little did they know that they were laying the foundation of an artistic movement. Graham Nelson, the most important single technical and creative architect of the Interactive Fiction Renaissance of the 1990s (and thus the eventual subject of my second and third articles), said it most cogently: “If I have hacked deeper than them, it is because I stand in their trenches.”

---

Although the Z-Machine was decidedly not intended as a community project, Infocom in their heyday made no particular attempt to hide the abstract fact that they were the proud possessors of some unusual technology. The early- and mid-1980s, Infocom’s commercial peak, was still the Wild West era of personal computing in the United States, with dozens of incompatible models jockeying for space on store shelves. Almost every published profile of Infocom — and there were many of them — made mention of the unique technology which somehow allowed them to write a game on a big DEC PDP-10 of the sort usually found only in universities and research laboratories, then move it onto as many as 25 normally incompatible microcomputers all at once. This was, perhaps even more so than their superb parser and general commitment to good writing and design, their secret weapon, allowing them to make games for the whole of the market, including parts of it that were served by virtually no other publishers.

So, even if highfalutin phrases like “virtual machine” weren’t yet tripping off the tongue of the average bedroom hacker, it wasn’t hard to divine what Infocom must be doing in the broad strokes. The specifics, however, were another matter. For, while Infocom didn’t hide the existence of a Z-Machine in the abstract, they had no vested interest in advertising how it worked.

The very first outsiders to begin to explore the vagaries of the Z-Machine actually had no real awareness of doing so. They were simply trying to devise ways of copying Infocom’s games — most charitably, so that they could make personal backups of them; most likely, so that they could trade them with their friends. They published their findings in organs like The Computist, an underground magazine for Apple II owners which focused mainly on defeating copy protection, hacking games, and otherwise doing things that the software publishers would prefer you didn’t. By 1984, you could learn how Infocom’s (unimpressive) copy-protection scheme worked from the magazine; by 1986, you could type in a program listing from it that would dump most of the text in a game for cheating purposes.

But plumbing the depths of a virtual machine whose very existence was only implicit was hard work, especially when one was forced to carry it out on such a basic computer as the Apple II. People tended to really dive in only when they had some compelling, practical reason. Thus users of the Apple II and other popular, well-supported platforms mostly contented themselves with fairly shallow explorations such as those just described. Users of some other platforms, however, weren’t fortunate enough to enjoy the ongoing support of the company that had made their computer and a large quantity of software on the shelves at their local computer store; they had a stronger motivation for going deeper.

Over the course of the 1980s, the American computing scene became steadily more monolithic, as an industry that had once boasted dozens of incompatible systems collapsed toward the uniformity that would mark most of the 1990s, when MS-DOS, Microsoft Windows, and (to rather a lesser extent) the Apple Macintosh would be the only viable options for anyone wishing to run the latest shrink-wrapped commercial software. This gradual change was reflected in Infocom’s product catalog. After peaking at 25 or so machines in 1984, they released their final few games in 1988 and 1989 on just four of them. The realities of the market by then were such that it just didn’t make sense to support more platforms than that.

But technical transitions like these always come with their fair share of friction. In this case, plenty of people who had been unlucky or unwise enough to purchase one of the orphaned machines were left to consider their options. Some of them gave up on computing altogether, while others sucked it up and bought another model. But some of these folks either couldn’t afford to buy something else, or had fallen hopelessly in love with their first computer, or were just too stubborn to give it up. This state of affairs led directly to the world’s first full-fledged Z-Machine interpreter to be born outside of Infocom.

The orphaned machine at the heart of this story is the Texas Instruments 99/4A, a sturdy, thoughtfully designed little computer in many respects which enjoyed a spectacular Christmas of 1982, only to be buried by Jack Tramiel under an avalanche of Commodore VIC/20s and 64s the following year. On October 28, 1983, Texas Instruments announced they were pulling out of the home-computer market entirely, thus marking the end of one of the more frantic boom-and-bust cycles in computing history. It left in its wake hundreds of thousands of people with 99/4As on their desks or in their closets — both those who had bought the machine when it was still a going proposition and many more who snatched up some of the unsold inventory which Texas Instruments dumped onto the market afterward, at street prices of $50 or less. The number of active 99/4A users would inevitably decrease sharply as time went on, but some clung to their machines like the first loves they often were, for all of the reasons cited above.

This little 99/4A fraternity would prove sufficiently loyal to the platform to support an under-the-radar commercial- software ecosystem of their own into the 1990s. For many users, the platform was appealing not least in that it never lost the homegrown charm of the very earliest days of personal computing, when every user was a programmer to one degree or another, when the magazines were full of do-it-yourself hardware projects and type-in program listings, and when one kid working from his bedroom could change the accepted best practices of everyone else almost overnight. The Z-Machine interpreter that interests us today was a reflection of this can-do spirit.

Ironically, Infocom didn’t begin to support the 99/4A until early 1984, when it was already an orphan. At that point, though, they made all of their existing games available, and went on to make versions of new games for the machine through The Hitchhiker’s Guide to the Galaxy in September of 1984. In all, they released no less than fourteen old and new games for the 99/4A during 1984.

But continuing to support any given machine was a more complicated proposition than simply continuing to use an already-extant interpreter. The Z-Machine in reality was more of a moving target than the abstract idea behind it might suggest. Infocom’s games got steadily bigger and richer as time went on, with more text, better parsers, and more ambitious world models. The original Z-Machine, as designed in 1979, had a theoretical maximum story-file size of 128 K, but the practical limitations of the machines running the interpreters kept the early games from reaching anything close to this size. (The original Zork, for example, Infocom’s very first game, was just 77 K.) As story files pushed ever closer to their theoretical maximum size in the years that followed, they began to exceed the practical limitations of some existing interpreters. When that happened, Infocom had to decide whether reworking the interpreter to support a larger story file was possible at all, and, if so, whether it was worth the effort in light of a platform’s sales figures. Following Hitchhiker’s (story-file size: 110 K), their fourteenth game, but before Suspect (story-file size: 120 K), their fifteenth, Infocom judged the answer to one or both of those questions to be no in the case of the 99/4A.

As one might expect, this decision left a number of 99/4A users sorely disappointed. Among them was Barry Boone, a clever young man just out of high school who was already one of the leading lights of 99/4A hackery. Having read enough about Infocom to understand that their game format must be in some sense portable, he started doggedly digging into the details of its implementation. Soon he was able to make a clear delineation between the interpreter running natively on his machine and the story file it executed — a delineation the Apple II crowd writing for The Computist had yet to manage. And then he uncovered the big secret: that the interpreter packaged with one game could actually run the story file from another — even if said story file originated on a platform other than the 99/4A! Boone:

Having worked out the file format, I wrote a program to crunch the non-TI files and build the TI files. The resulting files appeared to work, but I quickly discovered [a] problem. If I converted an older game that already existed in TI format, everything worked perfectly. But with the newer games, there was a big problem.

The problem was that the interpreter software written for the TI had a number of bugs, many of which did not show up with the original set of games, but became all too apparent with the newer ones and made them unplayable. So I began a process of reverse-engineering the Z-Code interpreter for the TI. Once I reached a point of having recreated the source code, I began working on making the code more efficient, and fixing numerous bugs in the implementation. The largest bug I encountered was a vocabulary-table bug. Basically, the original TI interpreter would hit an overflow bug if the vocabulary table was too large, and the binary-search algorithm would start searching the wrong area of memory to look up words. This had the effect of making the last portion of the vocabulary inaccessible, and made the game impossible to play.

I also added a number of enhancements that allowed the games to load about twenty times faster, and modifications to play the games on TI systems equipped with 80-column displays. Finally, I had to make a second variation of the interpreter so that persons who had an extra 8 K of RAM (known as a Super Cart, or Super Space module) could play some of the games that required a larger memory footprint than 24 K of memory buffer. These games included Leather Goddesses.

Boone estimates that he finished his interpreter around 1986, whereupon he promptly began sharing it with his network of friends and fellow 99/4A enthusiasts, who used it to play many of the newer Infocom story files, transferring them from disks for other platforms. Boone was stymied only by the games from Infocom’s Interactive Fiction Plus line, such as A Mind Forever Voyaging and Trinity. Those games used an expanded version of the Z-Machine, known internally as version 4 — the mature version of the original virtual machine was version 3 — which expanded the available memory space to 256 K, far beyond what the 99/4A could possibly manage. Even without them, however, Boone gave himself and his mates ten new Infocom games to play — i.e., all of those released for the 128 K Z-Machine between October of 1984 and July of 1987, when this original incarnation of the virtual machine made its last bow in Infocom packaging.

But even that wasn’t quite the end of the story. An obscure footnote to Infocom’s history took shape in late 1988 or early 1989, when Chris Bobbitt, founder of a company called Asgard, the 99/4A software publisher that most resembled a real business as opposed to a hobbyist project, had the idea of contacting Infocom themselves to ask permission to market the newer games, running under Boone’s interpreter, as legitimate commercial products. Although Bobbitt doubtless didn’t realize it at the time, Infocom was by then on the verge of being shut down, and Mediagenic, their less-than-doting parent company, were also beginning to feel the financial stresses that would force them into bankruptcy in 1990. They saw Bobbitt’s proposal as a handy way to clear their warehouse of old stock and make some desperately needed cash. Jim Fetzner, who worked with Asgard at the time, remembers how the deal went down:

[Bobbitt] contacted Infocom to ask for permission to release the later Infocom releases, and was given permission to do so on one condition: that the packaging and disks had to be originals for other systems, relabeled (the packaging) and reformatted (the disks) for use with the TI. Infocom scoured their warehouse and sent Chris two very large boxes of the titles he was asking to reproduce—and noted on the invoice that these boxes included every single copy of the relevant titles that Infocom still had in their possession. Some of the titles were relatively plentiful, but others were included in much lower numbers. The boxes only contained four copies of Leather Goddesses of Phobos, for example. All other titles had at least ten copies each, and some had a lot more. He was permitted to buy more copies from remainders in the retail channels, though, so it is possible there are more properly badged Asgard copies of the titles that were harder to find. All of the stock he received from Infocom was gone in a matter of months.

These games, which Bobbitt bought for $5 apiece and sold on for several times that, thereby became the last new Infocom games ever sold in their original packaging — out-of-print games from a dead company sold to owners of an orphaned computer.

Well before Asgard entered the scene, however, another, more structured and sustainable project had led to a Z-Machine interpreter much more amenable to being ported and built upon than Boone’s incarnation of same for an idiosyncratic, bare-bones, orphaned platform. Not long after Boone first started sharing his 99/4A interpreter with friends, a few students at the University of Sydney in far-off Australia started disassembling another of Infocom’s own interpreters — in this case one for Zilog Z80-based computers running the operating system CP/M. The group included in their ranks David Beazley, George Janczuk, Peter Lisle, Russell Hoare, and Chris Tham. They gave themselves the rather grandiose name of the InfoTaskforce, but they initially regarded the project, said Janczuk to me recently, strictly as “a form of mental calisthenics”: “This was never meant to be a public exercise.”

Still, the group had several advantages which Boone had lacked — in addition, that is, to the advantage of sheer numbers. Boone had been a bedroom hacker working on fairly primitive hardware, where cryptic assembly language, highly specific to the computer on which it was running, was the only viable option. The InfoTaskforce, on the other hand, had more advanced hardware at their disposal, and were steeped in the culture of institutional hacking, where portable C was the most popular programming language and software was typically distributed as source code, ready to be analyzed, ported, and expanded upon by people other than its creators, quite possibly working on platforms of which said creators had never dreamed. And then, thanks to their university, the InfoTaskforce was connected to the Internet, long before most people had even heard of such a thing; this gave them a way to share their work quickly with others across a wide, international swath of computing. The contrast with the segregated ghetto that was the world of the 99/4A is telling.

David Beazley, who did almost all of the actual coding for the InfoTaskforce interpreter — the others had their hands full enough with reverse-engineering the Z-Machine architecture — did so in C on a first-generation Apple Macintosh. On May 25, 1987, he used this machine to compile the first truly portable Z-Machine interpreter in history. Within a week, he and his mates had also gotten it compiled and running on an MS-DOS machine and a big DEC VAX. (Ironically, the latter was the successor to the PDP-10 line so famously employed by Infocom themselves; thus one might say that the Z-Machine had already come full-circle.)

As Janczuk remembers it, the first version of the interpreter to reach the Internet actually did so accidentally. He gave it to a friend of his at university, who, as so many friends have done over the years, uploaded it without permission on June 2, 1987. There followed an immediate outpouring of interest from all over the world, which greatly surprised the interpreter’s own creators. It prompted them to release an official version 1.0, capable of playing any story file for the standard — i.e., 128 K — Z-Machine on August 1, 1987. Already by this time, the Commodore Amiga personal computer and several more big machines had been added to the list of confirmed-compatible host platforms. It was a milestone day in the history of interactive fiction; Infocom’s games had been freed from the tyranny of the hardware for which they’d originally shipped. And they could remain free of the vicissitudes and fashions of hardware forevermore, as long as there was an enterprising hacker ready to tweak an existing interpreter’s source code to suit the latest gadget to come down the pipe. (So far, there has been no shortage of such hackers…)

With their university days coming to an end, the InfoTaskforce boys worked on their interpreter only in fits and starts over the years that followed. Not until 1990 did they finish adding support for the Interactive Fiction Plus line; not until 1992, in a final burst of activity, did they add support for Infocom’s last few text-only games, which ran under what was known internally as the version 5 Z-Machine. This last release of the InfoTaskforce interpreter actually attracted a bit of scoffing for its inefficiency, and for generally lagging behind what other hackers had done by that point in other interpreters.

In reality, information and inspiration rather than the software itself were the most important legacies of the InfoTaskForce interpreter. Beazley’s C source told you almost all of what you really needed to know about the Z-Machine, so long as you were sufficiently motivated to dig out the information you needed; doing so was certainly a fair sight more pleasant than poring over eye-watering printouts of cryptic disassembled Z80 machine language, as Beazley and his pals had been forced to do before coming up with it. The InfoTaskforce interpreter thus became the gateway through which the Z-Machine burst into the public domain, even as Infocom was soon to collapse and abandon their virtual machine. This was a role which Boone’s interpreter, for all its naïve brilliance, just wasn’t equipped to play, for all of the reasons we’ve already explored.

An enterprising American hacker named Mark Howell did perhaps the most to build upon the foundation of the InfoTaskforce interpreter during the half-decade after its initial appearance. His own interpreter bore the name of ZIP (for “Z-Machine Implementation Program”), a name it shared with the popular compression format, to enormous confusion all the way around — although, to be fair, this was also the name by which Infocom knew their own interpreters. ZIP was faster and less buggy than the InfoTaskforce interpreter, and for this reason it soon surpassed its older sibling in popularity. But Howell also delved further into the architecture of the Z-Machine than anyone before him, analyzing its design like a computer scientist might rather than as a hacker simply trying to write a quick-and-dirty clone of Infocom’s existing interpreters. When he came up for air, he uploaded his set of “ZTools” — programs for probing story files in all sorts of ways, including a disassembler for the actual code they contained. These tools did much to set the stage for the next phase of the Z-Machine’s resurrection and liberation.

In 1992, another building block fell into place when Activision shipped their Lost Treasures of Infocom collection to unexpected success. It and its sequel collected all of the Infocom games together in one place at a reasonable price, stored as neatly discrete story files ready to be fed into either the original Infocom interpreters included on the disks or an alternative of one’s choice. Lost Treasures shipped only in versions for MS-DOS, the Apple Macintosh, and the Commodore Amiga — the last three commercially viable personal-computing platforms left in North America by that time (and the Amiga wouldn’t enjoy that status much longer). But users of orphaned and non-North American platforms were soon passing around the tip that, if you could just get the story files off of the original Lost Treasures disks, they could be run on their own platforms as well with one of the interpreters that had by now spread far and wide. For example, our old friends at The Computist, still carrying the 8-bit torch in these twilight days of the Apple II, published instructions on how to do just that — a fitting end point to their earliest explorations of the Infocom format.

Across the Atlantic, meanwhile, the magazine Acorn User published a similar article for users of the Acorn Archimedes, a machine that was virtually unknown outside of Britain, a few parts of mainland Europe, and Australasia. (“It’s hard to conceive of videogame nostalgia,” they wrote of the Lost Treasures collections, “but this is as close as it gets.” Little did they know…) It so happened that an Oxford doctoral candidate in mathematics named Graham Nelson was a stalwart Acorn loyalist and a regular reader of that magazine. By the time the article in question appeared, the window opened by the InfoTaskforce interpreter and all the software that had followed it, combined with the Lost Treasures collections, had already led him to begin sliding the next couple of building blocks of the Interactive Fiction Renaissance into place.

(Sources: The Computist 5, 7, 34, 41, 47, 57, 58, 63, and 86; Acorn User of July 1993; Asgard Software’s newsletters from 1989 and 1990. Online sources include Barry Boone’s memories of writing his Z-Machine interpreter at The Museum of Computer Adventure Game History and his bio for the TI99ers Hall of Fame. The original source for the InfoTaskforce interpreter can be found in various file archives. My huge thanks go to Barry Boone, Jim Fetzner, and George Janczuk for talking to me about their pioneering early work in Z-Machine archaeology.)