Lev Manovich

The Language of New Media

Capter III / Menus, Filters, Plug-ins / p.120ff

The Logic of Selection

Viewpoint Datalabs International is selling thousands 3D geometric models widely used by computer animators and designers. Its catalog describes the models as follows: "VP4370: Man, Extra Low Resolution. VP4369: Man, Low Resolution. VP4752: Man, Muscular in Shorts and Tennis Shoe. VP5200. Man, w/Beard, Boxer Shorts..." Adobe Photoshop 5.0 comes with more than 100 filers which allow the user to modify an image in numerous ways; After Effects 4.0, the standard for compositing moving images, is shipped with 80 effects plugins; thousands more are available from third parties. Macromedia Director 7 comes with an extensive library of “behaviors” — ready-to-use pieces of computer code. Softimage|3D (v3.8), the leading 3D modeling and animation software, is shipped with over 400 textures which can be applied to 3D objects. QuickTime 4 from Apple, a format for digital video, comes with 15 built-in filters and 13 built-in video transitions. Geocities Web site, which pioneered the concept of hosting users’ Web sites for free in exchange for adding ad banners into users’ pages, gives users access to a collection of over 40,000 clip art images for customizing their sites. Index Stock Imagery offers 375,000 stock photos available for use in Web banner ads. Microsoft Word 97 Web Page Wizard lets the user to create a simple Web by selecting from eight predetermined styles described by such terms as “Elegant,” “Festive” and “Professional.” Microsoft Chat 2.1 asks the user to specify her avator by choosing among twelth built-in cartoon character. During the online session, the user can further customize the selected character by interpolating between eight values which represent eight fundamental emotions as defined by Microsoft programmers.

These examples illustrate a new logic of computer culture. New media objects are rarely created completely from scratch; usually they are assembled from ready-made parts. Put differently, in computer culture authentic creation has been replaced by selection from a menu. In the process of creating a new media object, the designer selects from libraries of 3D models and texture maps, sounds and behaviors, background images and buttons, filters and transitions. Every authoring and editing software comes with such libraries. In addition, both software manufacturers and third parties sell separate collections which work as “plug-ins,” i.e. they appear as additional commands and ready-to-use media elements under software’s menus. The Web provides a further source of plug-ins and media elements, with numerous collections available for free.

New media users are similarly asked to select from pre-defined menus of choices when using software to create documents or access various Internet services. Here are few examples: selecting one of pre-defined styles when creating a Web page in Microsoft Word or a similar program; selecting one of “AutoLayouts” when creating a slide in PowerPoint; selecting one of predetermined avatars on entering a multi-user virtual world such as Palace; selecting one of the pre-determined viewpoints when navigating a VRML world. (Avatar is a character or a graphic icon representing a user in a virtual world.)

All in all, selecting from a library or menu of pre-defined elements or choices is one of the key operations for both professional producers of new media and for the end users. This operation makes production process more efficient for the professionals; and it makes end users feel that they are not just consumers but “authors” creating a new media object or experience. What are the historical origins of this new cultural logic? How can we describe theoretically the particular dynamics of standardization and invention which comes with it? Is the model of authorship it puts forward specific to new media or can we already find it work in old media?

Art historian Ernst Gombrich and Roland Barthes, among others, critiqued the romantic ideal of the artist creating totally from scratch, pulling images directly from his imagination, or inventing new ways to see the world all alone. According to Gombrich, the realist artist can only represent nature by relaying on already established “representational schemes”; the history of illusion in art involves slow and subtle modifications of these schemes over many generations of artists. In his famous essay “The Death of the Author,” Barthes offered even more radical criticism of the idea an author as a solitary inventor alone responsible for work’s content. As Barthes puts it, "the Text is a tissue of quotations drawn from the innumerable centers of culture." Yet, even though a modern artist may be only reproducing, or, at best, combining in new ways preexisting texts, idioms and schemas, the actual material process of art making supports the romantic ideal. An artist operates like God creating the Universe — she starts with an empty canvas or a blank page. Gradually filling in the details, he brings a new world into existence.

In contrast, electronic art from its very beginning was based on a new principle: modification of an already existing signal. The first electronic instrument designed in 1920 by the Russian scientist and musician Leon Theremin contained a generator producing a sine wave; the performer simply modified its frequency and amplitude. In the 1960s video artists began to build video synthesizers based on the same principle. The artist was no longer a romantic genius generating a new world purely out of his imagination; he became a technician turning a knob here, pressing switch there — an accessory to the machine.

Substitute a simple sine wave by a more complex signal (sounds, rhythms, melodies); add a whole bank of signal generators and you have arrived at a modern music synthesizer, the first instrument which embodies the logic of all new media: selection from a menu of choices.

The first music synthesizers appeared in the 1950s, followed by video synthesizers in the 1960s, followed by DVE (Digital Video Effects) in the late 1970s — the banks of effects used by video editors; followed by computer software such as 1984 MacDraw that came with a repertoire of basic shapes. The process of art making has finally caught up with modern times. It has become synchronized with the rest of modern society where everything is assembled from ready-made parts; from objects to people's identities. The modern subject proceeds through life by selecting from numerous menus and catalogs of items — be it assembling an outfit, decorating the apartment, choosing dishes from a restaurant menu, or choosing which interest groups to join. With electronic and digital media, art making similarly entails choosing from ready-made elements: textures and icons supplied by a paint program; 3D models which come with a 3D modeling program; melodies and rhythms built into a music synthesis program.

While previously the great text of culture from which the artist created her or his own unique "tissue of quotations" was bubbling and shimmering somewhere below the consciousness, now it has become externalized (and greatly reduced in the process) — 2D objects, 3D models, textures, transitions, effects which are available as soon as the artist turns on the computer. The World Wide Web takes this process to the next level: it encourages the creation of texts that completely consist of pointers to other texts that are already on the Web. One does not have to add any original writing; it is enough to select from what already exists. Put differently, now anybody can become a creator by simply providing a new menu, i.e. by making a new selection from the total corpus available.

The same logic applies to branching-type interactive new media objects. In a branching-type interactive program, when the user reaching a particular object, she can select which branch to follow next by clicking a button or on the part of an image or by choosing from a menu. The visual result of making a choice is that is either a whole screen or its part(s) change. A typical interactive program of the 1980s and early 1990s was self-contained, i.e. it run on a computer which was not networked. In contrast to surfing the Web where it is very easy to move from one site to another, the designers of self-contained programs could expect undivided attention from a user. Therefore it was safe to change the whole screen after a user makes a selection. The effect was similar to turning pages in a book. This book metaphor was promoted by first popular hypermedia authoring software — Apple’s HyperCard (1987); a good example of its use can be found in the game Myst (Broderbund, 1993). Myst presents the player with still images which fill the screen. When the player clicking on the left or right parts of an image, it is replaced by another image. (For more on navigation in Myst, see “Digital Cinema” and “Navigable Space” sections below.) In the second part of the 1990s, as most interactive documents migrated to the Web and simultaneously became more complex, it became important to give all pages of the site a common identity and also visually display page’s position in relation to the site’s branching-tree structure. Consequently, with the help of such technologies such as HTML Frames, Dynamic HTML and Flash, interactive designers established a different convention. Now parts of the screen, which typically contain company logo, top-level menus, and page’s path, remain constant while other parts changed dynamically. (Microsoft and Macromedia sites provide good examples of this new convention. ) But regardless of whether making a selection leads the user to a whole new screen or only changes part(s)of it, the user still navigates through branching structure consisting from pre-defined objects. While more complex types of interactivity can be created by via a computer program which controls and modifies the media object at run time, the majority of interactive media uses fixed branching tree structures.

It is often claimed that a user of a branching interactive program becomes its co-author: by choosing a unique path through the elements of a work, she supposedly creates a new work. But it is also possible to see the same process in a different way. If a complete work is a sum of all possible paths through its elements, then the user following a particular path only accesses a part of this whole. In other words, the user is only activating a part of the total work that already exists. Just as with the example of Web pages which consist from nothing but the links to other pages, here the user does not add new objects to a corpus, but only selects its subset. This is a new type of authorship which corresponds neither to pre-modern (before Romanticism) idea of providing minor modification to the tradition nor to the modern idea (nineteenth and first part of the twentieth centuries) of a creator-genius revolting against it. It does, however, fit perfectly with the logic of advanced industrial and post-industrial societies, where almost every practical act involves choosing from some menu, catalog, or database. In fact, as I already noted when discussing interactivity in “Principles of New Media” section, new media is the best available expression of the logic of identity in these societies: choosing values from a number of pre-defined menus.

How can a modern subject escape from this logic? In a society saturated with brands and labels, people respond by adopting minimalist aesthetics and hard-to-identify clothing style. Writing about an empty loft as an expression of minimalist ideal, architecture critic Herbert Muschamp points out that people “reject exposing the subjectivity when one piece of stuff is prefered to another.” The opposition between an the indvidualised inner world and objective, shared, objective, neutral world outside becomes reversed:

The private living space has taken on the guise of objectivity: neutral, value-free, as if this were a found space, not an impeccably designed one. The world outside, meawhile, has become subjectified, rendered into a changing collage of pesonal whims and fancies. This is to be expected in a culure dominated by the distribution system. That system, exists, after all, not to make things but to sell them, to apeal to individual impulses, tastes, desires. As a result, the public realm has becime a collective repository of dreams and designs from which the self requires refuge.

How can one accomplishing the similar escape in new media? It can only be accomplished by refusing all options and customization, and ultimately refusing all forms of interactivity. Paradoxically, by followng an interactive path one does not construct a unique self but instead adopts already pre-established identitities. Similarly, chossing values from menu or customisng one’s desktop or an aplication automatally makes one participate in the “changing collage of personal whims and fancies” mapped out and coded into software by the companies. Thus, short of using command-line interface of UNIX which can be though of an equivalent of minimalist loft in the realm of computing, I would prefer using Microsoft Windows exactly the way it was installed at the factory.

“Postmodernism” and Photoshop

As I noted in this chapter’s introduction, computer operations encode existing cultural norms in their design. "The logic of selection" is a good example of this. But what was a set of social and economic practices and conventions now became encoded in the software itself. The result is a new form of control, soft but powerful. Although software does not directly prevent its users from creating from scratch, its design on every level makes it "natural" to follow a different logic: that of selection.

While computer software “naturalizes” the model of authorship as selection from libraries of pre-defined objects, we can already find this model at work with old media, such as magic lantern slides shows. As film historian Charles Musser points out, in contrast modern cinema where the authorship extends from pre-production to post-production but does not cover exhibition (i.e., the theatrical presentation of a film is completely standardized and does not involve making creative decisions), in magic lantern slide shows the exhibition was a highly creative act. Magic lantern exhibitioner was the in fact an artist who skillfully arranged a presentation of slides which he bought from the distributors. This is a perfect example of authorship as selection: an author puts together an object from the elements which she herself did not create. The creative energy of the author goes into selection and sequencing of elements, rather than into their original design.

Although not all modern media arts follow this authorship model, the technological logic of analog media strongly supports it. Stored using industrially manufactured materials such as film stock or magnetic tape, media elements can be more easily copied, isolated and assembled in new combinations. In addition, various media manipulation machines, such as a tape recorder and a film slicer, make the operations of selection and combination easier to perform,. In parallel, we witness the development of archives of various media which enable the authors to draw on already existing media elements rather than always having to record new elements themselves. For instance, in the 1930s German photojournalist Dr. Otto Bettmann started what latter became known as Bettmann Archive; at the time of its acquisition by Bill Gates’s Corbis Corporation in 1995 it contained 16 million photographs, including some of most frequently used images of this century. Similar archives were created for film and audio media. Using “stock” photographs, movie clips and audio recording become the standard practice of modern media production.

To summarize: the practice of putting together a media object from already existing and commercially distributed media elements already existed with old media, but new media technology further standardizes it and makes it much easier to perform. What before involved scissors and glue now involves simply clicking on "cut” and “paste. And, by encoding the operations of selection and combination into the very interfaces of authoring and editing software, new media “legitimizes” them. Pulling elements from databases and libraries becomes the default; creating them from scratch becomes an exception. The Web acts as a perfect materialization of this logic. It is one gigantic library of graphics, photographs, video, audio, design layouts, software code and texts; and each and every element is free since it can be saved to user’s computer with a single mouse click.

It is not accidental that the development of GUI which legitimized “cut and paste” logic as well as media manipulation software such as Photoshop, which popularized plug-in architecture, took place during the 1980s — the same decade when contemporary culture became “post-modern.” In evoking this term I follow Fredric Jameson usage of post-modernism as “a periodizing concept whose function is to correlate the emergence of new formal features in culture with the emergence of a new type of social life and a new economic order.” As it became apparent by the early 1980s for critics such as Jameson, culture no longer tried to “make it new.” Rather, endless recycling and quoting of the past media content, artistic styles and forms became the new “international style” and the new cultural logic of modern society. Rather than assembling more media recordings of reality, culture is now busy re-working, recombining and analyzing the already accumulated media material. Invoking the metaphor of Plato’s cave, Jameson writes that post-modern cultural production “can no longer look directly out of its eyes at the real word but must, as in Plato’s cave, trace its mental images of the world on its confining walls.” In my view, this new cultural condition found its perfect reflection in the emerging computer software of the 1980s which privileged the selection from already existing media elements over creating them from scratch. And at the same time, to large extent it is this software which made post-modernism possible. The shift of all cultural production to first electronic tools such as switchers and DVEs (1980s) and then to computer-based tools (1990s) greatly eased the practice of relying on old media content in creating new productions. It also made media universe much more self-referential, because when all media objects are designed, stored and distributed using a single machine — computer — it becomes much easier to borrow elements from already existing objects. Here again the Web became the perfect expression of this logic, since new Web pages are routinely created by copying and modifying already existing Web pages. This applies both for home users creating their home pages and for professional Web, hypermedia, and game development companies.

From Object to Signal

Selecting ready-made elements which will become part of the content of a new media object is only one aspect of “logic of selection.” While working on the object, the designer also typically selects and applies various filters and “effects.” All these filters, be it manipulating image appearance, creating a transition between moving images, or applying a filter to a piece of music, involve the same principle: algorithmically modifying the existing media object or its parts. Since computer media consist from samples which are represented in a computer as numbers, a computer program can access every sample in turn and modify its value according to some algorithm (see “Principles of New Media,” (2) and (3)). Most image filters work in this way. For instance, to add noise to an image, a program such as Photoshop reads in the image file pixel by pixel, adds a randomly generated number to the value of each pixel, and writes out a new image file. Programs can also work on more than one media object at once. For instance, to blend two images together, a program reads in values of corresponding pixels from the two images; it then calculates a new pixel value based on the percentages of existing pixel values; this process is repeated for all the pixels.

Although we can also find precursors to filter operations in old media (for instance, hand colorization of silent film), they really comes into their own with the electronic media technologies. All electronic media technologies of the nineteenth and twentieth century are based on modifying a signal by passing it through various filters. These include technologies for real-time communication such as telephone; broadcasting technologies used for mass distribution of media products such as radio and television; and technologies to synthesize media, such as video and audio synthesizers which originate with the instrument designed by Theremin in 1920.

In retrospect, the shift from a material object to a signal accomplished by electronic technologies represents a fundamental conceptual step towards computer media. In contrast to a permanent imprint in some material, a signal can be modified in real time by passing it through some filter(s). Moreover, in contrast to manual modifications of a material object, an electronic filter can modify the signal all at once. Finally, and most importantly, all machines for electronic media synthesis, recording, transmission and reception include controls for signal modification. As a result, an electronic signal does not have a singular identity — a particular state which is qualitatively different from all other possible states. Consider, for example, loudness control of the radio receiver or brightness control of an analog television set. They don’t have any privileged values. In contrast to a material object, electronic signal is essentially mutable.

This mutability of electronic media is just one step away from “variability” of new media (see “Principles of New Media” section.) As already discussed, a new media object can exist in numerous versions . For instance, in the case of a digital image, we can change its contrast and color, blur or sharpen it, turn it into a 3D shape, use its values to control sound, and so on. But, to a significant extent, an electronic signal is already characterized by similar variability, because it can exist in numerous states. For example, in the case of a sine wave, we can modify its amplitude or frequency; each modification produces a new version of the original signal without affecting its structure. Therefore, in essence, a television or radio signal are already new media. Put differently, in the progression from a material object to an electronic signal to computer media the first shift is more radical than the second. All that happens when we move from analog electronics to digital computers is that the range of variations is greatly expanded. This happens because, firstly, modern digital computers separate hardware and software, and, secondly, because an object is now represented as numbers, i.e. it become computer data which can be modified by software. In short, a media object becomes “soft” — with all the implications contained in this metaphor.

The experimental filmmaker Hollis Frampton whose reputation rests on his remarkable structural films and who, towards the end of his life, came to be interested in computer media, seemed to already understood this fundamental importance of the shift from a material object to an electronic signal. He wrote in one of his essays:

Since the New Stone Age, all the arts have tended, through accident or design, toward a certain fixity in their object. If Romanticism deferred stabilizing the artifact, it nonetheless placed its trust, finally, in a specialized dream of statis: the 'assembly line' of the Industrial Revolution was at first understood as responsive to copious imagination. If the television assembly line has by now run riot (half a billion people can watch a wedding as consequential as mine or yours) it has also confuted itself in its own malleability. We're all familiar with the parameters of expression: Hue, Saturation, Brightness, Contrast. For the adventurous, there remain the twin deities Vertical Hold and Horizontal Hold…and, for those aspiring to the pinnacles, Fine Tuning.

What Frampton calls “malleability” of television signal becomes “variability” of new media. While the analog television set allowed the viewer to modify the signal on just a few dimensions such as brightness and hue, new media technologies give the user much more control. A new media object can be modified on numerous dimensions, and these modifications can be expressed numerically. For instance, the user of a Web browser software can instruct the browser to skip all multimedia elements; tell it to enlarge font size while displaying a page or to completely substitute the original font by a different one. The user can also re-shape the browser window to any size and proportions as well as change the spatial and color resolution of the display itself. Further, a designer can specify that different versions of the same Web site will be displayed depending upon the bandwidth of user’s connection and the resolution of her display. For instance, a user accessing the site via a high-speed connection and a high resolution screen will get a rich multimedia version while the user accessing the same site via a small LCD display of a hand-held electronic will receive just a few lines of text. More radically, a number of completely different interfaces can be constructed to the same data, from a database to a virtual environment. In short, the new media object is something which can exist in numerous versions and numerous incarnations.

To conclude this discussion of selection operation, I would like to invoke a particular cultural figure — a new kind of author for whom this operation is the keys. This author is a DJ who creates music in real-time by mixing already existing music tracks and who is dependent on various electronic hardware devices. In the 1990s DJ acquired a new cultural prestige, becoming a required presence at art openings and book release parties, in hip restaurants and hotels, in the pages of Art Forum and Wired. The rise of this figure can be directly correlated to the rise of computer culture. DJ best demonstrates its new logic: selection and combination of pre-existent elements. DJ also demonstrates the true potential of this logic to create new artistic forms. Finally, DJ example also makes it clear that selection by itself is not sufficient. The essence of DJ’s art is the ability to mix the selected elements together in rich and sophisticated ways. In contrast to “paste and cut” metaphor of modern GUI which suggests that selected elements can be simply, almost mechanically combined, the practice of live electronic music demonstrates that true art lies in the “mix.”