ChatGPT × Hosaka | Problems and Limitations of Screen Essentialism

1. What is Screen Essentialism (Definition of the Concept)

Screen essentialism refers to the idea in the analysis of digital media and visual culture that "what is displayed on the screen is the essence of the medium" [1]. It is a bias that focuses only on "visible information on the screen," such as images, videos, and UIs displayed on computers and smartphones, while treating other elements (data structure, hardware, production processes, etc.) as secondary. The concept was originally raised in the context of digital humanities and new media studies, and is said to have been proposed by Nick Montfort around 2003 [2] [3]. Montfort criticized the tendency of critics in new media studies to focus excessively on computer displays (screens), calling this bias "screen essentialism" [4].

This concept is mainly discussed in fields such as digital media studies, digital humanities, and software preservation. It emerged as a reflection on the approach of analyzing only the visual information appearing on the screen when studying new media works and digital art [5]. In addition to the aforementioned Montfort, major proponents and theorists include Matthew Kirschenbaum and Mark Sample, who are involved in digital archaeology and software studies. In fact, media researchers such as Montfort, Sample, and Kirschenbaum have critically referred to "screen essentialism" in their respective contexts [6], and the recognition that **"what is on the screen is not everything about digital media"** is becoming widespread.

2. Problems with Screen Essentialism (Critical Perspective)

Why is screen essentialism considered a problem? A major reason is that it ignores the materiality and technical processes of media [7]. Because digital content appears intangible on the screen, we tend to think that "the cloud" or "cyberspace" has no physical substance. However, in reality, data is supported by electromagnetic records (inscriptions) on servers or hard disks, and networks are established by physical infrastructure such as cables and radio waves [8]. Falling into screen essentialism causes one to overlook these dimensions of hardware and physical media, leading to a superficial and conceptual understanding of digital technology.

Montfort points out that such discussions biased toward screen display are also historically skewed [9]. Computers before the mid-1970s mainly used printing terminals (teletypes) or lamp displays, and not everyone was looking at a display as they do today [10]. Nevertheless, if modern researchers only discuss phenomena on the screen, they risk missing past media forms and misunderstanding media history.

Furthermore, screen essentialism is criticized as an attitude that prioritizes "the experience of the surface (screen) visible to the user" while neglecting technical and physical realities. Matthew Kirschenbaum, a researcher in digital literature and forensics, states that the modern trend of favoring graphical user interfaces (GUI) has created this tendency, leading to "a state of 'screen essentialism' where technical and physical realities are disregarded in favor of prioritizing user experience" [11]. For example, when we edit text on a PC, we can instantly "delete" text or "adjust the brightness" of an image on the screen. However, behind the scenes, complex processes are actually running, such as how data is being rewritten on a magnetic disk [12] [13]. A screen-essentialist view does not focus on these underlying processes, and as a result, it reduces the essence of digital media to "symbols on a flickering screen" [14].

What are the specific harms of disregarding materiality and processes in this way? First, it becomes difficult to accurately understand and preserve digital works and data. If one thinks that saving only what is visible on the screen is sufficient, there is a risk that when preserving software or games for future generations, only the appearance will be reproduced without recording the internal operating principles [15]. In that case, it will be impossible to fully interpret how the work originally operated or under what conditions the screen was generated. In fact, in the field of digital game preservation, it has been reported that the idea that "saving is fine as long as the screen is displayed correctly" is insufficient. In interactive games, superficial properties such as colors and sounds on the screen cannot convey the important characteristics of the work, and in many cases, it does not make sense unless it includes the interaction with the player and the mechanisms inside the program [16] [17].

Second, screen essentialism leads to overlooking important contexts in digital culture. Content appearing on the screen (e.g., text or video) lacks background information on the environment and circumstances in which it was generated [18]. To take an extreme example, suppose you record and save all the footage of an online game. Based on screen essentialism, you might think, "If I can see the footage, I can reproduce the culture of that time." However, in reality, without the context of the player's operations leading up to that point, the in-game community, and the creator's intentions, the understanding gained from that footage is limited. It could end up in a state of **"having content (the substance) but no context (the background)"** [19]. Therefore, for media researchers and preservationists, relying too much on superficial screen information is dangerous, and the necessity of examining supporting technical documents and usage records together is emphasized [20] [21].

Third, it can be pointed out that screen essentialism has the limitation of narrowing the possibilities of media expression. If one ends up with only semiotic interpretations on a visual screen, expressions involving other perceptions (sound, touch, spatial experience) and physical elements are neglected. As a result, when evaluating and analyzing media works, one only has visual-centric criteria, and may miss the value of works that are inherently multimodal (multi-sensory). By not looking at phenomena occurring outside the screen—such as the sounds and vibrations emitted by the device, or the optical and spatial characteristics of the usage environment—one fails to grasp the media experience as a whole.

In summary, the critical perspective on screen essentialism can be summarized as "it is a mistake to grasp digital and video media only by the visuals displayed on the screen" [22]. The essence of a digital object is not limited to screen display; behind it exist vast amounts of bit information, code, and mechanical processes [23]. If that is ignored, the understanding of the media will be incomplete, and it will also hinder preservation and utilization.

3. Specific Examples of Screen Essentialism and Attempts to Overcome It

The influence of screen essentialism can be seen in various media such as photography, film, and digital art. Below are examples of each, and we will look at their problems and attempts to overcome them.
• Photography: In the digital age, it has become standard for photos to be viewed and shared on PC or smartphone screens. As a result, a trend has emerged of viewing photos only as "image data on a screen." However, in the field of photography research, there has recently been a growing movement to focus on the materiality of photography. This is a perspective that views photography not as mere image information, but as an "object as a thing" [24]. For example, in the case of analog photography, it is emphasized that photos always have a physical carrier, such as negative film or photographic paper, and even for digital photos, there are prints and storage media (CDs or hard disks) [25]. Treating photos in a screen-essentialist way ignores these physical characteristics (paper texture, aging, development process, etc.) and ends up only with analysis of image composition or the meaning of the subject. To overcome this limitation, photographers and researchers are daringly reviving old wet plate photography techniques or re-evaluating the texture of vintage prints. Also, for digital photos, there is a movement to archive them including Exif metadata and equipment information at the time of shooting, and an approach is being taken to understand and preserve photos as image data + context.
• Film/Video: In film studies as well, the analysis of images projected on a screen (story and screen composition) was once mainstream. However, film is a medium made up of film (material) and projectors (equipment). From a screen-essentialist perspective, the differences in film material and the influence of the projection environment tend to be regarded as secondary, but in reality, they have a significant impact on the viewing experience. For example, footage shot and screened on 35mm film differs from digital screening in resolution, color tone, and texture, and the brightness of the projector and the graininess of the film also shape the atmosphere of the footage. Recent media-archaeological video research sheds light on these material aspects of film. Projects are also being conducted to analyze old projectors and shooting equipment themselves and reproduce the film experience of the time. For example, attempts to screen at the frame rate of the time using hand-cranked projectors from the silent film era, and archival methods that intentionally preserve the material traces of deteriorated film (scratches and fading) complement traditional analysis that focuses only on the story on the screen. Through such approaches, it becomes possible to understand film from both content and medium. By overcoming screen essentialism, comprehensive film research including the history of the film itself and the evolution of technology (e.g., the introduction of soundtracks, colorization, widescreen, etc.) is progressing.
• Digital Art/Net Art: The influence of screen essentialism is prominently seen in the evaluation and preservation of digital art and interactive art using the Internet. For example, net art works that operate on the web change their behavior moment by moment depending on user operations and network environments. However, a superficial approach captures the work only by the visuals displayed on the screen, and in the worst case, there is a risk of being satisfied with saving only "screenshots of the work" or "recorded video." This is archiving a living work by cutting it out only as a "result" of a still image or video, which impairs the essence of the work such as interaction and real-time nature.
In response to such problems, digital art preservation institutions and researchers are taking initiatives to preserve and reproduce works along with their operating environments. At the ArtBase operated by Rhizome in the United States, projects are underway to enable works to be re-experienced as they were at the time by emulating the browsers, plugins, and OS environments of the time, rather than simply recording the appearance of the works. Behind this is the recognition that "the characteristic of net art is not just the appearance on the screen (screen essentialism), but the vast amount of processes behind it" [26]. In fact, some net art works involve vast amounts of data processing, communication, and random generation in parts that do not appear on the screen, and it is necessary to consider those as part of the work. By standing on the perspective that "things other than what is visible on the screen are also part of the work," digital art preservation is undergoing a paradigm shift from mere recording to restoration by environment.
In the world of digital art, creative methods that criticize and overcome screen essentialism are also being born. A method called glitch art is a prime example. This is art that attempts to visualize the "materiality of data" that is not visible in normal viewing by intentionally corrupting the internal data of digital files or opening them in different formats. For example, if you open an image file in a text editor, you can see that metadata such as shooting date and camera information is hidden within the garbled data that is unreadable to humans. If you further modify it and display it again in the original viewer, noise or color misalignment (glitch) appears on the screen, and the data structure itself is exposed as art. Glitch art can be said to be an attempt to show that "the finished product that looks beautiful on the screen" is not the only essence, and that "the data, code, and errors behind it can also be part of the beauty." By actively incorporating the material aspects of digital media that were previously excluded, it throws a stone at the screen-essentialist aesthetic.

Figure: An example of opening an MP3 audio file in a text editor. When an MP3 file, which is normally played as audio, is read in text format, as shown in the figure above, it can be confirmed that fragments of metadata such as ID3 tags, song titles, and collection information are exposed among strings of characters that are meaningless to humans. By opening files in a way different from their original purpose, information that does not appear on the screen (data structure and supplementary information) becomes visible [27]. Glitch art is an expression method that uses these principles to visualize the materiality and processes of media by daringly bringing the layers of digital data that are originally hidden to the surface.
• Methodology for Overcoming Screen Essentialism: As is clear from the above examples, a perspective that views media multi-dimensionally is essential to overcome the limitations of screen essentialism. A representative approach is a research method called media archaeology. [Media Archaeology] looks not only at the latest modern media but also at historical past media technologies, and attempts to excavate and re-evaluate the genealogy of technology and forgotten devices and formats. The work of German media researchers Erkki Huhtamo, Wolfgang Ernst, or Jussi Parikka is known, and they are building theories that go beyond traditional "text-analysis-centered media studies" and focus on material media devices and infrastructure [28] [29]. Media archaeology also sheds light on non-visual elements in electronic media (e.g., layers of communication protocols, Wi-Fi or Bluetooth radio spectrum, internal structures of old software and hardware) [30]. For example, it disassembles and analyzes media devices used in a certain era of the past to investigate how their technical constraints influenced the content of expression at that time, or conversely, makes conscious the invisible foundations of modern digital culture (noise and heat from server rooms, network latency, etc.). Through such approaches, it becomes possible to stratigraphically excavate the "layers" behind the media that do not appear on the screen [31].

Another approach is a trend that can be called the material turn. Although it is included in media archaeology in a broad sense, in the field of art and preservation practice, it is shifting more concretely toward actively handling hardware and media materials. For example, in the preservation of media art, there are cases where the originality of a work is maintained by repairing and continuing to use deteriorated videotapes or old-fashioned CRT monitors [32] [33]. Alternatively, academic fields that emphasize the analysis of platforms such as game consoles, OS, and programming languages, which are the foundations on which works operate, like platform studies and software studies, have also emerged. Montfort & Ian Bogost's book "Racing the Beam" focuses on the hardware of the 1970s game console Atari VCS and discusses in detail the influence of its technical constraints (beam-scanning display and memory capacity) on the game content [34]. This is an example that opens up new horizons in game history research by capturing the interaction between hardware and software, which is a finding that cannot be seen only by "considering the game screen displayed on the screen."

In short, to overcome screen essentialism, the attitude of exploring **"what is behind what is visible" is important. The methods are interdisciplinary and diverse. Sometimes it involves collecting and experimenting with old media equipment in a museum-like manner, sometimes it involves deciphering the source code or data structure of a program, and it also includes contemporary artists creating new works using old technology as material. What they all have in common is the point of expanding interest not only to the surface on the screen but also to the materiality, structure, and environment of the medium.**

4. Significance of Overcoming Screen Essentialism (Impact on Contemporary Art and Media Studies)

Criticism of screen essentialism and attempts to overcome it are bringing new possibilities and directions to contemporary art expression and media studies.

First, in the field of art, expressions incorporating materiality and processes have increased even for works using digital technology. This also leads to a trend called "post-digital" or "post-internet." For example, some contemporary artists incorporate physical devices themselves, such as old computers or typewriters, into their works, or intentionally print and process digital images analogically. Such works provide a multi-layered experience that does not close on the screen while dealing with seemingly digital themes. The rise of glitch art is also part of this, and by incorporating digital errors and noise as aesthetic elements, it overturned the conventional image that "digital = clean and immaterial." Artistic expression that has overcome screen essentialism gives the viewer room to exercise their imagination on the back side of the media. As a result, art appreciation is expanded from mere visual reading to an experience that mobilizes critical awareness of technology and the five senses.

In media studies as well, its significance is great. By breaking away from screen essentialism, researchers can comprehensively re-grasp media as social and material entities. In recent research, discussions that include the environment surrounding media, such as Jussi Parikka's "Geology of Media" [35], which focuses on mineral resources and waste that support digital technology, and infrastructure studies and platform studies, are active. This is revealing that digital media is an existence connected not only to human culture and history but also to the global environment and industrial structure. Themes such as power structures, economy, and material circulation, which could not be seen only by symbol analysis on the screen, have been brought to light, and the range of media theory has expanded.

Also, the perspective that goes beyond screen essentialism has brought innovation to preservation and archive practices. In museums and archives, "media archaeological" preservation that collects digital works and video games that were previously considered difficult to preserve along with their original hardware and OS environments, and reproduction methods using emulation technology are being adopted. For example, by reproducing early websites and Flash works in the browser environment of the time, it is an attempt to preserve including the user experience that cannot be conveyed by mere screenshots. This allows future researchers to re-experience the work including the operation feeling, response speed, and constraints of the time, enabling deeper understanding. In other words, the method for "leaving media works as historical materials" has been updated. A new layer such as design documents, source code, and hardware equipment collection has been added to the field that was previously limited to output recording (e.g., dubbing preservation of video) [36] [37]. Such a comprehensive approach also enhances the reliability and evidentiary nature of the preservation of media cultural heritage [38].

Finally, overcoming screen essentialism provides an integrated perspective in terms of theory as well. The bridging of fields that were previously divided, such as semiotic analysis and technical analysis, aesthetics and engineering, and humanities and information science, is progressing. For example, in digital humanities, interdisciplinary research is spreading where cultural materials are analyzed using knowledge of programming and data analysis, while philosophical and historical contexts are emphasized when interpreting the results. These can be said to be comprehensive knowledge that goes back and forth between the screen (surface) and its back side (foundation), and it has become indispensable for understanding modern complex media phenomena.

As described above, criticism and overcoming of screen essentialism are not limited to mere academic discussions, but are affecting everything from creative practices in art to the inheritance of cultural assets and the development of media theory. The perspective of looking at this side and the other side of the screen in an integrated manner will give us the power to read and interpret media more deeply and richly. It is the driving force that opens up new possibilities for expression and pushes media studies to the next stage.

References/Sources:
• Presentation and criticism of the concept of screen essentialism by Nick Montfort et al. [39] [40] [41]
• Criticism of screen essentialism in digital humanities [42] [43]
• Problems seen from the field of digital preservation/archiving [44] [45]
• Discussion on the materiality of photography [46]
• Dialogue at Rhizome regarding net art preservation [47]
• Media archaeological approach (Parikka et al.) [48] [49]
• Other examples such as glitch art [50]