Middleware’s Message: the Financial Technics of Codata

Abstract

In this paper, I will argue for the relevance of certain distinctive features of messaging systems, namely those in which data (a) can be sent and received asynchronously, (b) can be sent to multiple simultaneous recipients and (c) is received as a “potentially infinite” flow of unpredictable events. I will describe the social technology of the stock ticker, a telegraphic device introduced at the New York Stock Exchange in the 1860s, with reference to early twentieth century philosophers of synchronous experience (Bergson), simultaneous sign interpretations (Mead and Peirce), and flows of discrete events (Bachelard). Then, I will show how the ticker’s data flows developed into the 1990s-era technologies of message queues and message brokers, which distinguished themselves through their asynchronous implementation of ticker-like message feeds sent between otherwise incompatible computers and terminals. These latter systems’ characteristic “publish/subscribe” communication pattern was one in which conceptually centralized (if logically distributed) flows of messages would be “published,” and for which “subscribers” would be spontaneously notified when events of interest occurred. This paradigm—common to the so-called “message-oriented middleware” systems of the late 1990s—would re-emerge in different asynchronous distributed system contexts over the following decades, from “push media” to Twitter to the Internet of Things.

1 Introduction

The stock ticker was a device introduced in the early 1860s by Edward A. Calahan, an employee at the American Telegraph Company, and installed on the floor at the New York Stock Exchange (NYSE) and a handful of brokerage offices in 1867. The resultant system was “both network and machine” (Moore 2016)—electrically linked to a keyboard near the trading floor, each ticker device would print, on a continuous roll of thin tape, the name of a security and its corresponding price quote (Preda 2006).¹ In more recent years, the ticker’s materiality has been rediscovered as a site of theoretical interest by the economic sociologists Alex Preda and Karin Knorr Cetina, the latter of whose work involved ethnographic observation and interviews of early twenty-first century foreign exchange traders. What are the philosophical and/or technological connections between the nineteenth century stock ticker and the phenomenologically immersive real-time representations of markets on computer screens—what Knorr Cetina named “scopic systems”—of modern finance? How can we isolate their distinctive qualities in the history of technology? And how did these technologies relate to other types of distributed messaging systems—so-called middleware—which pervade everyday digital life today?

In this paper, my goal is to demonstrate the philosophical, historical, and technological connections between the ticker and contemporary middleware, by bringing together prior work in economic sociology and the history of computing; the early twentieth century philosophies of Bergson, Bachelard, Peirce, and Mead; and formal definitions of “streaming” data types from computer science. First, I will distinguish the stock ticker’s ontology/phenomenology from the contemporaneous circuit-based telecommunication device of the telegraph, and how its distinctive synchronous, broadcast, and potentially infinite qualities can justify our specific epistemological interest; and I will discuss how arguments and concepts from the aforementioned philosophers (Bergson, Bachelard, Peirce, and Mead) can provide a deeper understanding of the ticker’s distinctive features. Finally, I will show how challenges of incompatible data feeds in the late twentieth century financial industry—along with developments in distributed systems research—led to the message broker, an asynchronous and distributed ticker-like system which today underpins a wide variety of event-centric applications in finance, enterprise software, social media platforms, and more.

2 The Ticker in Relation to the Telegraph

In the decades preceding the introduction of the stock ticker on Wall Street, the use of telegraphy had expanded rapidly, with Europe and North America being connected by a transatlantic cable just a year before in 1866 (transatlantic attempts began in 1858 but failed shortly thereafter or were not reliable). The innovative coding scheme devised by Morse was little more than a mapping of the English alphabet to a variable-length on-off serial code which was affected by the operation of a key (a manually operated electrical switch) which, by opening and closing a lengthy copper wire circuit, would trigger a sounder at one or more distant stations. This type of modulation (or technique for varying a physically contiguous connection to communicate information) of the first electromagnetic telegraphs is known as on-off keying.²

The close relationship between stock exchange trading of the nineteenth century and the early expansion of telegraphy (and other expedient relays, including pigeon post³) is well-documented: one important early telegraph line (which initially competed with an existing optical semaphore relay) was between the Sandy Hook lighthouse in New Jersey and the Merchants Exchange on Wall Street (a progenitor of the New York Stock Exchange); and the first link on the east coast of the USA, in 1846, connected Philadelphia’s Merchants’ Exchange (predecessor of the Philadelphia Stock Exchange) to traders on Wall Street. Almost immediately, newspapers began criticizing “speculators” who used the communication technology for “inside” trading (Du Boff 1980). Telegraphy was also an important factor (along with railroads and the storage facilities of Cronon (1992)) in the development of commodity exchanges, by introducing so-called “to arrive” contracts which specified the date of delivery (Chandler 1977, 211).⁴ Telegraphy was also widely used for a longer distance coordination by expanding businesses; private individual communication, by comparison, represented a minority of telegraph use (DuBoff 1982; Tarr et al. 1987).

As the networks of telegraph offices expanded, telegrams could be relayed from one of a “spoke” of local telegraph offices to each other or to a central telegraph office, and in turn to another locale’s office, at which point the message would be transcribed on paper and taken on foot by a messenger to the recipient (Standage 1998). By contrast, Calahan combined his novel “printing telegraph”—the name given to all receiving devices which could automatically convert telegraphic communications to materialized symbolic text—with the logic of an earlier system set up by Samuel Spahr Laws of the nearby Gold Exchange, in which an operator on the trading floor could transmit gold quotations via a specialized keyboard to real-time “gold indicators” (instruments with dials) via a wire circuit (Hochfelder 2012).

As such, the information on an early stock ticker was be transmitted in a relatively circumscribed geographical region from a manually operated keyboard on the trading floor linked to a so-called “loop” circuit with multiple “listening” ticker devices. Reporters in the “Board-Room” would transmit quotes by telegraph to two ticker firms: one, the New York Quotation Company, would relay quotes by ticker to over 1100 member brokerages; the other, the Gold and Stock, would distribute them to clients outside the exchange—including the “bucket shops,” locations where unregulated side betting would occur (Hochfelder 2006). The ultimate communication of quotes to traders thus was a broadcast form of communication, in which all ticker devices on the loop would receive the same signal at the same time. But it was also a form of communication which unfolded as an unpredictable flow of messages—correlating with the volume of market activity, the ticker would produce a highly variable quantity of quotation data over the course of the trading day, its tape unspooling into waste bins and, often, onto brokerage office floors.

The aforementioned method of relaying telegrams between telegraph offices in a larger network became consistently known as a store-and-forward system; it thus produced what I call message asynchrony—the physical detachment of the act of sending and receiving of information. The ticker was, in contrast, more like a telephone call, which links sender and receiver in a single physically continuous (or indexical) circuit. While I will not discuss early telephone systems here for reasons of space, it should be noted that the kind of circuit switching (or line switching) used to indexically connect telephone users (as in the use of telephone switchboards and their operators) produces a communicative synchrony which can be overtly contrasted with the asynchronous message switching of telegraph offices. Significantly, the intensity of technical development and innovation in telephone line switching in the early twentieth century (Fagen 1975) was, in part, due to the fact that the number of possible (indexical) one-to-one connections increases approximately as the square of the total number of subscribers in a network (Mueller 1989); by contrast, the store-and-forward telegrams of Western Union’s network could be given various levels of priority and delivery time in high-traffic situations without immediately necessitating extra lines. This advantageous quality of message switching would find a recurrence in the 1960s development of packet switching, inspired in part by existing message switching architectures such as those of Western Union’s (Campbell-Kelly 1988; Abbate 2000); in the next section, I will propose a general typology of messaging features which places the distinctive technical qualities highlighted by the advent of the ticker—(1) the asynchrony of data arrival, (2) the synchrony of transmission, (3) its broadcast to large numbers of recipients, and (4) its unpredictable flow—in relation to later developments in data communications.

3 A Typology of Messaging

The modern Western concept of the “message” as a spatial carrying of communication has its origins in Greek gods like Hermes (and Roman analogue Mercury), who played the role of Zeus’ courier—and, in the Bible, angels (ἄγγελοι or angeloi)—who were messengers between gods, or between gods and men; the image of Hermes/Mercury frequently appeared in the imagery of various newspapers and early Western post offices, from Danish periodicals (e.g., the Altonaischer Mercurius in 1698) to the seals (and later, stamps) of the US Post Office from 1782, with the “Mercury” still a common title for American dailies today (DeBlois et al. 2012).⁵ In a postal system, sealed letters can be delivered over long distances and relayed across road networks; this type of communication emphasizes the discrete, the symbolic, and the simplex (or uniplex—each message travels in one direction at a time). This is in contrast to, for example, a face-to-face, in-person conversation, where communication is continuous, immediately both indexical (i.e., contiguous in physical/material reality) and symbolic (e.g., the use of lexical codes), and, as telecommunications engineers would put it, “full duplex”—both participants can communicate at the same time, although turn-taking is a universal pattern (Sacks et al. 1974).⁶

Already, these categories provide a more fine-grained contrast than existing social-scientific typologies of communication techniques, which often merely distinguish between “point-to-point” communications and “broadcast” communications (as in DiMaggio et al. 2001); this particular distinction, in my view, is only one distinguishing feature among many.⁷ Because part of my argument is to more accurately distinguish message-centric systems from other forms of communication, in Table 1 I have enumerated a variety of communication types and technologies to show in which dimensions they differ from each other. In this paper, I will focus on three such dimensions, which act as distinctive features—in the sense of (Jakobson et al. 1961)—which, when combined, will ultimately isolate the particular modality of modern-day message broker (also known as message-oriented middleware) systems:

• Transmission: Synchronous vs. Asynchronous. These terms generally refer to whether or not the sending and receiving participants of a given communicative act send and receive a message “at the same time” (e.g., as approximately takes place during a telephone conversation) or whether the act of sending and receiving are more temporally segregated (as in “leaving a message” on an answering machine to be heard later).⁸

  There are, however, two somewhat distinct types of synchrony/asynchrony; at the contact level and the message level (where the terms “contact” and “message” are used in the sense of Jakobson (1960)). Contact asynchrony occurs when, for example, a communication arrives as an interruption or surprise (a telephone rings; a telegraph sounder buzzes; an email notification appears); whereas, contact synchrony occurs when the sender and receiver are more-or-less permanently locked in a request–reply situation.⁹ For linguists like Jakobson, the contact quality of communication is associated with the phatic function of communication—i.e., the semiotic significance (or lack thereof) of the presence (or co-presence) of addresser and addressee.

  Message asynchrony, by contrast, corresponds to relayed communication—as in a “relay race,” but also in the sense of electrical relays (devices that, when activated, make or break a connection from one circuit to another)—in which the act of sending or receiving information is detached in time (as in an answering machine, where a message is “left” for later retrieval). And message synchrony, again, refers to our intuitive sense of a phone conversation, where speech is heard near instantaneously by the other party. It is this latter opposition which is of somewhat more importance, and so when I use “synchrony” or “asynchrony” alone in this paper, I will by default mean message synchrony and message asynchrony.¹⁰

• Unicast/Multicast/Broadcast. This distinction refers to the number of simultaneous addressees of the communication in the network; one-to-one communication is unicast, one-to-many is multicast—a term from early literature in computer networking and distributed systems (Rowe and Birman 1979; Shoch et al. 1982)—and one-to-all communication, in which all senders receive the same messages (whether in a synchronous or asynchronous manner), is denoted as broadcast.

• Finite/Potentially Infinite Communications (i.e., data/codata).¹¹ Is the communication finite, in the sense that we can expect it to have a maximum length, or is it more-or-less “potentially infinite,” a Heraclitean “stream” of data? More specifically, does the receiver not know in advance how much information they are going to get? I use the term codata as shorthand for the concept of “potentially infinite data,” in order to denote symbolic communications which arrive more in the form of a flow of unpredictable quantity than not. The word “codata” is drawn from late twentieth century research in programming languages and category theory (Hagino 1987; Turner 1995), in which it was used to circumscribe a special type of object which can produce a potentially infinite stream of data.¹² It is not necessarily important whether this flow is “continuous” or “discontinuous,” only that the stream of information is unpredictable in volume and unfurling in time, as opposed to having a known size and an inert manifestation (in the case of a computer, being held on a disk and/or in memory).

Table 1 Typology of telecommunications modes across various distinctive features

In the next section, I will show how this set of distinctive features of modern messaging can help us understand Preda and Knorr Cetina’s interest in—and interpretation of—both the nineteenth century ticker and the financial data systems of twenty-first century traders.

4 The Economic Sociology and Philosophy of the Ticker

Some of the distinctive properties of the ticker device, observed from an economic–sociological and philosophical perspective, were described by the sociologists Alex Preda and Karin Knorr Cetina in the mid-2000s. This work began with field research by Knorr Cetina and Urs Bruegger on the trading floors of three investment banks in Zurich, where they focused on currency traders, their “face-to-screen” orientations (Knorr Cetina and Bruegger 2002a), and the “postsocial” qualities of the traders’ relationships to markets as presented on trading screens (Knorr Cetina and Bruegger 2002b).

Knorr Cetina and Bruegger’s approach to these markets, focusing on a “reflexive, temporal form of coordination” (Knorr Cetina and Bruegger 2002a), was explicitly conceived as complementary to studies of social networks which they deemed insufficient for explaining the relationships of market participants in an elaborate communicative screen space. The next year, Knorr Cetina published another essay which emphasized the currency traders’ world as a flow architecture, drawing on Heraclitean metaphor to describe the technics of empirical market reality (Knorr Cetina 2003); but this was rhetorically opposed not to an material ideology of stasis but to an increasingly high-status subfield of social network analysis. Knorr Cetina’s opposition between network and flow is interesting for the purposes of this paper because the story of message brokers (and the history of distributed computing systems more generally), as we shall see, is one which must quite overtly bring together network and flow. It is less that Knorr Cetina is wrong to oppose them, and more that the concept of “network” to which she addresses her critique is a static one devised by social scientists (at the time, and to some extent still today, consisting merely of “nodes” and “edges,” and thus eschewing complex technosocial relationships and processual change).

Preda, instead of examining the high-tech world of modern-day trading, looked to the nineteenth century and found himself studying something oddly relevant (both historically and ontologically) to Knorr Cetina’s “flow architecture.” In his study of the history of the stock ticker (via archives in New York, Philadelphia, and London), Preda proposed to see the ticker technology as what he called a “standardizer”(which inscribes traces of trading activity in a standardized textual, printed format) and also as a “generator” of an unpredictable flow of values that moves faster or slower along with the trading activity of the moment. Preda also argued that “technology is social action” and thus (citing twentieth century Viennese sociologist/philosopher Alfred Schutz) “generates time structures.” He drew two relevant distinctions in the following short passage:

For instance, a technology that produces data sporadically and at irregular intervals differs from one producing data continuously and at regular intervals. Data perceived as representing past transactions differ from data representing current transactions (Preda 2006, 757).

This first distinction (in the first sentence), I argue, closely corresponds to my concept of (contact) asynchrony vs. (contact) synchrony (the sporadic appearance of stock quotes); and the second distinction (in the second sentence), to data vs. codata—i.e., between transactions-as-stored-record vs. transactions-in-the-moment. Preda is here drawing from Schutz’ distinction between performed action—“action as performed act, as the thing done” and working action—“action as an ongoing process” (Schutz 1962, 214)—which he uses to contrast “arrangements such as a table or list that refers to the past” (i.e., data) against the supposedly volitional aspect of “data presented as a continuous flow” (i.e., codata) (Preda 2006, 757). The stock ticker is thus an intriguing device in its mix of contact asynchrony, message synchrony, and codata: it is an unpredictable flow of discrete interruptions, which nevertheless is closely temporally linked to the site of transaction. This close linkage was also only possible to the extent that the circuit could be physically extended in space; in Paris, by contrast, brokerage houses were scattered all over the city, and therefore, we should not be surprised that the stock ticker did not catch on with remotely the same virulence as in America.¹³

Preda shows that in the early decades of the ticker, brokerage firms still used postal letters to communicate to clients, and convincingly argues that the ticker was initially “not wanted for efficient, accurate and broad diffusion of price data” but instead was desired “because it helped reinforce social status and a monopoly over authoritative price data” (Preda 2006, 765). This was in part due to the separation between the higher status Regular Board, which traded in periodic call auctions, and the Open Board, which traded continuously. But in November 1870, the Regular Board merged with the Open Board, trading in the same room; and the difficulty of call-auction markets to coexist with competing continuous markets would recur as a theme in the history of financial exchanges in the twenty-first century.¹⁴

5 Bergson and Bachelard: Synchronous Duration vs. the Asynchronous Event

What philosophical and social theories, then, are necessary to help bridge the nineteenth century synchronous world of Preda’s ticker with the twenty-first century infrastructure of Knorr Cetina’s flow architectures, in which traders differentially subscribed to a massive variety of streams of market and news events? In this section, I will explain (1) how an early debate surrounding Einstein’s special relativity between the French philosophers Henri Bergson and Gaston Bachelard prefigured the notion of a duality between (message) synchrony and (contact) asynchrony, and thus provides an intellectual basis for later paradigmatic developments in data communications; and (2) how the philosophical work of the sociologist George H. Mead can help us understand our other categories—those of multicast and codata (i.e., digital communications with multiple recipients, unpredictably flowing in the moment)—as a specifically and intrinsically social phenomena in a way that conceptually unicast and/or static data communications/formations are not.

Technoscience in North America and Europe in the late nineteenth century was immersed in the problem of clock synchronization, driven in part by the expansion of railway networks (Galison 2003, 40). Various inventors from the 1830s onwards responded to the challenge of synchronizing clocks by devising systems (with varying degrees of success), typically of a “master–slave” orientation, which would coordinate a primary clock with secondary clocks via electromagnetic signals. The intensity and diversity of these projects, from office buildings to military communications, certainly indicates a demand for temporal consensus in many aspects of bureaucratic conquest; synchronized clocks, from this perspective, represent a most literal version of Bruno Latour’s “immutable mobiles,” those various forms of inscription apparatus (including clocks, but also maps and records) which facilitate administration at a distance (Latour 1986). To use my terminology, such systems intended not necessarily to send arbitrary messages between a master clock and slave clocks but to merely preserve a reliably periodic contact synchrony (or isochrony), so that each could tick in relative unison.¹⁵

The problem of clock synchronization eventually led directly to Einstein’s proposal of special relativity (Einstein 1905), which in a few short years after its publication was widely seen as an exemplar of modern intellectual thought, one which should be accounted for not just in physics but in philosophy and elsewhere (Galison 2003, 24–25). Einstein’s 1905 proposal—that, as a consequence of the upper limit to the speed of light (denoted as c), time was only meaningful with respect to a reference frame—was prefigured by Poincaré, who in his 1898 essay “The Measure of Time,” noted the seeming arbitrariness of measuring lengths of time and/or determining simultaneity for distant events (Galison 2003, 32–36). Poincaré’s perspective was in opposition to the then-popular philosophy of Henri Bergson, for whom the true conception of simultaneity—as well as the flow of immediate experience he called duration—was something intuitive as opposed to something that could or should be formalized geometrically (Canales 2015, 83–84).

The work of Poincaré and Einstein would lead to the concept of space–time as promulgated by Minkowski, who stated that, in the wake of Einstein, “henceforth space by itself and time by itself, are doomed to fade away into mere shadows, and only a kind of union of the two will preserve independence” (Minkowski 2012 [1909]). In Minkowski’s famous diagrams (Galison 1979), the paths of objects moving at different speeds could be plotted on the same two-dimensional graph—with distance on one axis and time on the other—with a (linear) skewing transformation depending on each object’s relative speed (and thus demonstrating how “simultaneity” is in the eye of the beholder).¹⁶ This conceptual approach was extremely problematic for Bergson, who had long resisted the “spatialization” (and thus quantification) of time in all of its forms. As Bergson describes in his first work (1899’s Essai sur les données immédiates de la conscience, retitled in English as Time and Free Will), consciousness should be characterized by a temporal “qualitative multiplicity” which he calls “duration” (durée), and this is to be contrasted with our perception of space, which is quantitative and measurable, and whose metaphors dominated then-popular ways of thinking—including (especially) those of time and consciousness, which Bergson aimed to revise (Bergson 1910). Bergson wanted to rediscover a fundamental distinction between space and time, which he blamed Kant for subtly conflating. Later, in Creative Evolution (1907), Bergson used the metaphor of the cinematographic film (which at that time would have recorded around 15 frames per second) to show how time, in the view of the physicists, was discretized and turned into uniform points on a single spatialized line. For Bergson, “the contrivance of the cinematograph” is identical to that of popular knowledge about time (Bergson 1944 [1907]).

How would Bergson, then, understand the stock ticker? For it is certainly experienced as a kind of flow, but one overtly made up of distinct, unpredictable symbolic events (namely, the reported stock quotations and/or execution of trades on the trading floor, as announced by the trading floor keyboard). The brokers who came under its spell indeed could seem to be immersed in a spiritual hypnosis; but it is one formed from nondeterministic interruption and not pure duration. The ticker, which does not print characters in a continuously periodic motion, does not spatialize time in the manner critiqued by Bergson; but is also quite literally spatializes time by converting the market’s activities into a linear tape. (Moreover, as we will argue below with the help of Mead, the multicast quality of the ticker extends beyond any individual’s subjective experience, and is thus instead profoundly social.) Beyond its synchronic presentism, the ticker manages to throw a wrench into Bergsonian thought in every other one of its aspects.

In contrast to Bergson—who was raised in London and Paris and entered the École Normale Supérieure at the age of 19—Gaston Bachelard worked for the Postes, Télégraphes et Téléphones (PTT—the administrative unit which later become France Telecom in the 1980s) for some years after his secondary education. Transferred to Paris, Bachelard took night classes and acquired a license in mathematics in his twenties, and he had applied to be a professional telegraph engineer shortly before World War I broke out (Chaplin 2007). Entering the Sorbonne in his late 30s after multiple years in the trenches, his advisors Abel Rey and Léon Brunschvicg were both opposed to Bergson’s philosophies (Chimisso 2001). One can get an initial sense of Bachelard’s intervention towards Bergson through the titles of his early books; with The Intuition of the Instant he is problematizing Bergson’s claim that it is only duration, and not instantaneity, that one can experience and understand through an introspective intuition; and with The Dialectic of Duration he wants to put into dialogue with Bergson’s “qualitative multiplicity” that which Bergson was resolutely opposed: the quantitative multiplicity of discontinuities, “lacunae,” and events.¹⁷ Bachelard’s critique of Bergsonian duration was born from the observation that the subjective experience of temporal phenomena can vary, and that the conception of a single dureé (duration) was insufficient. He argued, instead for a duality of duration and event (which I would characterize as related to our dichotomy of contact synchrony and contact asynchrony):

Botanists who limited their science to saying that all flowers fade would just be doing the same thing as some philosophers who underpin their theories by repeating that all things pass away and that time flies. We very soon saw that between this passing of things and the abstract passing of time there is no synchronism, and that temporal phenomena must each be studied according to its appropriate rhythm and from a particular point of view. When we examined this phenomenology in its contexture… we saw that it always comprises a duality of events and intervals. In short, when we looked at it in the detail of its flow, we always saw a precise, concrete duration that teemed with lacunae (Bachelard 2000, xii) [emphasis added].

And Bachelard denies that Bergson’s homogeneous thought is possible without the possibility of discontinuous redirection or interruption:

Bergson takes psychological intuition to be a priori a continuous thread, imposing an essential unity on experience as though experience could never be contradictory or dramatic… Even in the most homogenous order of thought, you cannot go from one essence to another by continuous thought (Bachelard 2000, 42).

With Bachelard, we can understand that a sense of duration can arise from a quantitative multiplicity of instants and discontinuities; and as I will show later, it is this scaffolding on which much of today’s digital, networked world—and the visual contiguity of Knorr Cetina’s scopic systems—relies. By contrast, Bergson represents a philosophy of temporal experience which is so aggressively qualitative and continuous (and, thereby, indexical and synchronous) that it unfortunately resists application to the practical technics of messaging technologies: i.e., of the asynchronous arrival of symbolic (and switched/routed) data, which nevertheless can be differentially experienced or reasoned about as a kind of continuous flow. Message-based communication networks, as I develop them here, thus create for their users a different kind of Bergsonian duration, which may (in a context of high message volume and high reliability) nevertheless still be phenomenologically experienced as Bergsonian duration: consider the mid-1990s description of a distinctly synchronous (call-and-response)-like activity as “surfing the Web”—which is in fact composed of the disorderly arrival of discrete messages (in the form of HTML documents and raster images). This is to say that sufficiently fast-moving switched and/or routed messaging (e.g., the Transmission Control Protocol (TCP) layer of the Internet) can provide a phenomenological illusion of continuous synchrony (i.e., of a web browser downloading data “at the same time” as a corresponding remote uploading server application), despite that “continuity” being ontologically composed of many, many switched and/or routed messages (i.e., TCP packets).

5.1 Mead, Peirce, and Multicast/Broadcast Codata

As Abbott (2001, p. 23) points out, however, Bergson’s theory is wholly asocial, and Bachelard—despite his influence from Pierre Janet, a French psychologist who shared Maurice Halbwachs’ perspective that memory is ontologically social—does not significantly improve on this state of affairs. (Where Bachelard is ultimately a social philosopher, it is in his recognition of the dyadic relation of teacher and student in his theories of scientific knowledge.) To move to a theory of telecommunications which can be ontologically social, we must instead consider the philosopher and sociologist George Herbert Mead, who in his posthumous work Philosophy of the Present declared at the outset that “[t]he world is a world of events” (G.H. Mead 1932, 1).

Mead’s present-centric perspective has its origins in Bergson’s Time and Free Will—but Mead (correctly, in my view) rejects the Bergsonian notion that because all is continuous (if heterogeneous) flux (duration/durée), we must necessarily privilege an introspective/psychological perspective (Emirbayer and Mische 1998; Joas 1997).¹⁸ Mead goes even further than Bergson, in the claim that the present is all there really is; for Mead, the past “...is expressed in irrevocability” (G.H. Mead 1932, 2). In Mead’s presentism, the occurrence of emergent events is how we know time (Abbott 2001, 227); there is no past or future in and of themselves, only past and future as they relate to the passage of emergent events in the present (Adam 1994, 39). He writes:

The social character of the universe… we find in the situation in which the novel event is in both the old order and the new which its advent heralds. Sociality is the capacity of being several things at once [emphasis added] (G.H. Mead 1932, 49).

This is a radical definition of sociality, and allows us to make claims about the relative sociality of communication technologies based on their synchronic/asynchronic qualities and (especially) their unicast/multicast qualities. One can give Mead’s definition of sociality a Peircean reading, in which the interpretation processes core to Peirce’s semiotics—which can occur in a mix of iconic, indexical, or symbolic modes—are precisely those which demarcate a past from a future, by projecting and/or refracting signs from the present into the future.¹⁹ In this hybrid Mead/Peirce view, sociality is a function of the interpretants (i.e., interpretation processes), as signs flow through the present; and the more possible interpretants there are for a given sign object, the more social the sign. (Pragmatically, the variety of possible meanings would be constrained by social and linguistic norms, expressed in other modalities, such as in the stylistic “objectivity” of a newspaper article.)

Peirce was never strict about the mental character of his semiotic interpretants; his theory of responses to signs could, without serious modification, be migrated to a sociotechnical attitude which takes humans as ontologically technical beings. Moreover, interpretants are themselves sign-generating processes, of which further interpretants can take as their sign. One might argue, then, that the stock ticker is a sociotechnical device/network which, in its transformation of the sender’s interpretations of trading floor activity (transducing from indexical electrical modulation to typed symbols), itself facilitates further pragmatic interpretations for a multiplicity of human tape-reading recipients—thus creating a potentially more uniform world of signs for brokerage houses and bucket shops (and today, for cable news viewers and Yahoo! Finance users). It is this continuous multiplicity of simultaneous possible interpretations that constitutes the market “lifeworld” for Knorr Cetina. However, via Mead, we can also see the ticker as intensely social in that its spatially replicated utterances can simultaneously have a wide variety of meanings to its different addressees: the updated price of a stock can mean riches for one broker and ruin for another, and relative indifference in so many more. The ticker was thus both a device and network for making markets “more social” in Mead’s sense.²⁰ From this perspective, we can also observe a distinction between the more “antisocial” valence of the archive (i.e., static data) in comparison to these multicast/broadcast flows of data-in-motion (i.e., codata).²¹

To reach the next two sections, I will leap over a century of telecommunication developments and increased market participation, as well as the “paperwork crisis” of the 1960s which spurred the acquisition of back-office computing technologies (Kennedy 2017). We arrive instead at a different crisis—one of a heterogeneity of incompatible, competing data services, terminals, and nascent desktop computers at the dawn of the era of automated trading, and show how the development of message queues and message brokers changed Wall Street, distributed systems, and the twenty-first century web.

5.2 The Asynchronous Multicast of the Message Broker: Teknekron and the Information Bus

By the 1980s, Wall Street trading rooms were characterized by miles of cables, proprietary trading systems, and a variety of market data sources with heterogeneous displays cluttered on desks. The cables connecting these dozens of technologically incompatible information sources were braided and wound through offices and under carpets, and large electric fans attempted to cool down the masses of electronics (Thornton 2000; Ranadive 1999).²² In the place of the humble ticker, traders subscribed to services from a handful of companies producing incompatible market data formats—the ubiquitous Quotron, as well as Reuters,²³ ADP, Telerate, and Knight-Ridder, with hundreds of thousands of terminals in use worldwide (US Congress Office of Technology Assessment 1990, 133). Beyond these devices, brokerages used crude video switches to limit the number of necessary monitors, with a cost per trading desk of about US$30,000 (Roman 1987). While these firms would gradually move into digital switching technology—with Reuters taking the lead in London (Blackford 1988)—customers still complained that the digital information provided was not easily integrable with their own computer environments.

In 1985, Vivek Ranadivé, an MIT student originally from Bombay, raised US$250,000 from the pioneering Berkeley-area startup incubator Teknekron to start his own company to address these issues of market data incompatibility in the financial industry; the incubator retained majority ownership and the resulting company was called Teknekron Software Systems.²⁴ Ranadive was not the only one attempting to solve these problems in the late 1980s; a consultant for Goldman Sachs, Sam Somech, developed a product (then known as “Distributed Message Queuing” or DMQ) for the Bank of New York to receive clearing and settlement instructions on DEC VAXs from brokerage firms which kept track of positions on IBM mainframes (Sell-Side Technology 1988). Another company in New York (for which Somech would later work), Systems Strategies Inc., offered similar products (Network World 1987). These expensive commercial products differed from the then-popular distributed communication paradigms in computing research; namely, the dominant Remote Procedure Call (RPC) developed at Xerox PARC (Birrell and Nelson 1984)—in which communication between processes in a distributed system occurs in analogy to invoking a procedure in a programming language like COBOL or Pascal—was a synchronous protocol (where senders had to wait for receivers, and vice versa), as opposed to the asynchronous or “store-and-forward” protocols of Teknekron and their competitors.

Teknekron Software’s system integration tools were first selected for use by Merrill Lynch for their equity trading support unit in November 1988. That department’s VP told Wall Street Computer Review that “Teknekron is giving us the glue to put it all together… The value [is] the merging of externally available data along with internally available data” (Schmerken 1989). Their goal at the time was to provide “customized presentation of multiple market information services as well as applications (deal entry, position keeping etc.) and analytics that require real-time market data”—in effect, the first full-fledged computing workstation for Wall Street traders (Kondo and Chithelen 1988).

In 1992 at the mixed academic/industry conference USENIX, Dale Skeen, a Teknekron Software employee and former IBM researcher, presented some of the details of what they had implemented. His paper, entitled “An Information Bus Architecture for Large-Scale, Decision-Support Environments” (Skeen 1992), explained his use of a “publish/subscribe” paradigm (derived from distributed systems work by Cheriton and Zwaenepoel (1985)) and a “software bus” architecture (see Fig. 1) which used the analogy of a hardware bus—the name for the common communication framework for various components of hardware in a computer—to describe the system to which messages where published and disseminated.²⁵ Skeen describes the asynchronous, multicast communication model between publishers and subscribers as “subject-based addressing”; the idea being that a workstation could, e.g., notify this “Information Bus” of its subscription to the subject labeled “Eq.ibm.trade” in order to be asynchronously notified of all trades of IBM stocks, or to the subject labeled “Com.gold.news.reuters” to be asynchronously notified of all news items from the Reuters feed about gold commodities futures.

Fig. 1

Illustration of the “information bus” architecture (Skeen 1992; © 1992 Dale Skeen. Reprinted by permission)

Skeen’s patent applications for the reliable publish/subscribe multicast architecture of the “TIB” (short for “The Information Bus”) reveal a set of contemporaneous systems which, by the early 1990s, had independently developed message-broker-like features outside of the financial industry. These include technologies like Usenet newsgroups; the Zephyr messaging system (Dellafera et al. 1988); and research on windowed development environments such as the FIELD system (Reiss 1990). In each case, reading and writing messages were organized in the form of a (sometimes-replicated) publish/subscribe architecture, in an effort to decouple publishers who may not be wholly aware of all of their subscribers, and vice versa.

In addition, Skeen is clearly attuned to the codata aspect of data feeds from the information bus, drawing direct analogies between the potential composition of “information pipelines” and the style of unicast input/output redirection in Unix operating systems (originally designed by Doug McIlroy) known as “pipes,” in which—just as on a stock ticker—user or system processes read files (or the output of other processes) without knowing their finite or infinite extent—i.e., by treating data as a flow which may or may not terminate (Farrow 2016). As implemented by Dennis Richie and Ken Thompson at Bell Labs, Unix provided pipes from the outset (Ritchie and Thompson 1974); this explicitly processual and flow-oriented ontology differentiated Unix from its mainframe contemporaries (Mélès 2013). It seems plausible that the early systems integrators like Skeen who strove to implement message passing between mainframes and Unix systems (among others), found themselves adopting the same infrastructural metaphor as one of their target platforms. The difference between Unix pipes and the Information Bus, as Skeen points out, is that the former is unicast and the latter is multicast; a given “pipe” of information can have 0, 1, or multiple subscribers (Skeen 1992, 196).

The Information Bus (or “TIB”) of Teknekron Software Systems (later renamed TIBCO after its acquisition by Reuters in 1996) had comparable functionality to existing projects in computer science research, such as Ken Birman’s ISIS system (Birman and Cooper 1990), which shared the goal of reliable multicast distribution of asynchronous messages. But as is shown here, this work was pitched towards the financial industry from the beginning, and best serves to illustrate the techniques which would come to be a core infrastructural element of every trading room in the decades to come. In the following section, I will show how the message brokers and queues developed for the financial industry (by TIBCO, IBM, and others) extended into a larger industry of “message-oriented middleware” throughout the late 1990s and early 2000s, as commercial organizations moved to access customers on the Internet.

5.3 The Middleware Concept

The term “middleware” was introduced by early UK software company founder Alex d’Agapeyeff in the landmark 1968 NATO Science Committee conference in Garmisch, Germany on “Software Engineering” (a then-provocative title, as the development of computer programs was at the time not considered to be a technical skill on par with the complexities of electrical and civil engineering). d’Agapeyeff provided a diagram called an “inverted pyramid” (see Fig. 2) in which so-called “middleware” is held to sit in between application programs and the system’s service routines, and which might in part protect those programs from their dependency on “lower levels” (Naur and Randell 1969).²⁶ However, the term did not catch on, and lay largely dormant for two decades, as software developers primarily continued to write programs directly “above” the operating system layer; exceptions included transaction monitors like IBM’s CICS (Yelavich 1985), which indeed provided a kind of “middle” ground which abstracted from some of the communicative complexity of online transaction processing.

Fig. 2

The “inverted pyramid” of d’Agapeyeff. Source: author, after Naur and Randell (1969)

In the early 1990s, the term middleware re-appeared, precisely in the same industry journals discussing “systems integrators” along the lines of Teknekron Software and the aforementioned Systems Strategies. The term’s revival was associated with the emergence of a so-called “open systems” approach to networking and software development, first positioned against the hegemony of proprietary IBM hardware and increasingly inspired by the proliferation of Unix operating systems, and later by the late-1980s formation of the Open Software Foundation (OSF) by Digital Equipment Corporation (DEC) and other manufacturers to organize against the potential dominance in the Unix space by AT&T and, at the time, Sun Microsystems.²⁷ Some commentators of the early 1990s saw middleware as a way to literally bridge the emerging world of “client-server” computing, characterized by personal computers (PCs) connected to Unix or VAX VMS minicomputers, and the still-present world of mainframe computing (Millikin 1992). The “hardware bus” metaphor (of a central place where data from heterogeneous peripherals pass) was thus transposed, not just to the level of streams of securities quotes, but to the motley and heterogeneous collections of systems which had come to characterize the data processing and IT departments of large firms (Linthicum 1999).

Initially, this revived term “middleware” conflated all types of communication in distributed computing systems; so file transfer protocols were considered to be middleware, but so was any kind of distributed database software; or an RPC library was considered middleware, but so were the more “message-oriented” middleware systems like Teknekron and its competitors. It became gradually apparent that message-oriented middleware (given the acronym “MOM”) was distinctive in its asynchronous and multicast qualities, and could be seen as complementary, but not identical, to RPC-style communication, which was largely synchronous and unicast (Dolgicer 1993)

IBM, in particular, realized in the late 1980s that it had no comparable product for what was needed in the financial industry—a way to communicate between existing (and popular) transaction monitors like CICS and an increasing variety of non-IBM systems. Two employees at IBM Hursley in the UK (the development site of CICS), Rob Drew and Dick Dievendorff, had previously worked on a prototype “queue manager” for the IBM operating system MVS; but in order to provide more thorough compatibility with Unix and other systems, they decided to partner with New York’s Systems Strategies—such an external partnership, for IBM, then being extremely uncommon—to provide the under-the-hood support for IBM’s newly proposed message queueing API (Flaherty 2011). The resultant product was called MQSeries—a message queuing system today known as IBM MQ—and in an early press release in March 1993, IBM was keen to emphasize the asynchronous aspect of MQSeries as “a concept a bit like programs leaving phone-mail messages for each other” (Newsbytes News Network 1993).

6 Conclusion: Financial Middleware at the End of the 1990s

Finally, we can return to the original site of the ticker, the New York Stock Exchange (NYSE). In 1998, Eliot Solomon, a Senior Technical Director at Securities Industry Automation Corporation (SIAC), the backend organization of the NYSE, gave an interview to UK-based industry journal Middleware Spectra (Solomon 1998). He detailed the growing importance of middleware at the exchange over his 12 years of tenure, involving the in-house development of middleware libraries (called “Common Software”) and describes the then-contemporary setup:

You will notice that we sandwich message switching systems between our functional systems (or our functional systems between message switching systems). We use extremely reliable asynchronous messaging middleware… which is a fundamental part of our basic approach to, and delivery of, ultra-reliable systems. Indeed, to achieve this, we had to build our own middleware.

Wherever you look into SIAC’s trading systems, you will find asynchronous messaging infrastructures. In some ways we regard ourselves as being primarily a message switching company. Sometimes, though, we look at it a little differently. From this angle we see ourselves as a factory automation or process control company.

Both are valid views. Both demand middleware to enable disparate systems to work together [emphasis added] (Solomon 1998).

SIAC had to develop its own middleware, linked to its core transaction processing and communications systems, because the scale and scope of multicast codata at a centralized stock exchange exceeded anything commercially available at the time. But the distinctive features introduced at the outset of this paper—the asynchronic, multicast, and potentially infinite qualities of the distributed message—first came to be crucial for the operation of a digitally enhanced stock exchange (and their traders). These techniques would soon be found essential for a new generation of web services (Emmerich et al. 2008), social communications platforms (Kreps et al. 2011), and environmental monitoring/Internet of Things (IoT) systems (Stanford-Clark and Wightwick 2010), making the “streaming” world of asynchronous, multicast codata flows a familiar characteristic of the digital twenty-first century to come (Berry 2011; Berry 2014; Amoore and Piotukh 2015).