Designing a Public Sensory Platform on the Net
Shin-ichi TAKEMURA <firstname.lastname@example.org>
Yoshiaki NISHIMURA <email@example.com>
This paper describes the experimental World Wide Web project "Sensorium," which is an attempt to develop the potential of the Internet as new "doors of perception" and to focus on the concept of "Webness."
New media always imitates old (established) media in its infant stage. The fact that the Internet is merely a conveyer of "paperless mail" and "cyber advertisement" for the general user indicates its lack of identity. What is central to the Internet's uniqueness as a new sociocultural media? What sorts of new experiences are activated through the Net? What is viable only on the Web? These queries are the motive behind the "sensorium" project, originally conceived as the Japan Theme pavilion of IWE96 (Internet World Exposition), the first global expo on the Net, and recipient of the Golden Nica prize at the Ars Electronica festival in Linz, Austria in 1997.
Sensorium is an experimental alternative Digital Museum designed to create a "public sensory platform on the Net." It is a new "forum" for sharing live experiences, such as the visualization of global seismic events and the circulation of our body cells, rather than the mere digitalization of existing knowledge and data found in textbooks and museums.
The project focuses on sensitivity towards the Internet itself as a living system. The more familiar we become with the new technological environment and its convenient output, the less conscious we are of the process and its mechanism; the once tangible and transparent Net becomes more abstract and foreign, a sort of "blackbox." Thus, rather than merely using the Internet and adding new pages to the already enormous Web catalogue, we are trying to "sense" concrete Net activity and make the Internet itself the content (subject) by, for example, monitoring the communications process of users and making it visual/audible on a real-time basis.
The points of our discussion are as follows:
In a deep blue space spreading on a computer screen, a vibrating silver globe is drifting. On closer look, one sees that bubbles are generating incessantly at various places as if the entire globe were breathing in and out. Here, the recent images of the inner movements of the earth (i.e., movements during the two weeks preceding the moment of accessing this home page), which are obtained by monitoring seismic events occurring daily all over the world, are compressed into CG animation of less than 20 seconds.
These seismometers are none other than numerous sensors which sense what may be called the fetal movements of the earth at every moment. Data of the earth's crustal fluctuations at these various sites are successively collected through the information network based on the Internet and make up a realistic seismic database of a global size which is updated daily on the WWW. In this context, we human beings are now acquiring "a global-sized nervous system" which monitors total changes in the "physical" and "emotional" conditions of this living and breathing planet.
But, unfortunately, such information resources remain as mere lists of dull numerical data used only by a handful of specialists like seismologists, and most people do not know of the resources' existence. They neither feel the presence of this "global-sized nervous system" in their daily life nor sense the pulses of the living body called the earth. We were thus prompted to create a program based on this seismic database, which is public information, and to convert numerical data, which are meaningful only to specialists, into visible and dynamic expressions that can be intuitively understood by the general public, including children.
We thus understand that the earth is constantly breathing as "a whole," that earthquakes are by no means abnormal phenomena that occur only occasionally but are "normal" for a healthy earth as they occur in numerous numbers every day all over the world, and that the earthquakes we experience in Japan are not isolated or abnormal but are part of "earthwide incidents" and are linked to seismic movement in Indonesia or the Philippines.
This is an experiment to "set up a public platform for global sensibility," an attempt to open a window in each of the individual terminals (PCs) in order to actually feel that every one of us potentially has the "global-sized nervous system."
This brief description of our contents named Breathing Earth symbolically illustrates the aim of Sensorium: that is, above all, to realize essential aspects of world experience in the Internet age and to develop the potential of "Webness."
This objective might be paraphrased in two complementary ways: we seek to design vital experiences unique to the Internet age by taking advantage of the essential characteristics of the Web environment; and we also are trying to be more conscious of "Webness" and seeking a new "shared sense" on the Internet by applying its potential for creating various catalyzing experiences.
Only through the Net has it become possible to realize such a daring attempt to accumulate the data of seismometers from all over the world at every moment nearly in real time and to totally visualize them on a screen accessible by all. This is indeed astonishing, and we wanted to share with others our astonishment at and gratification for the possibilities of such an experience.
The site name "Sensorium" expresses our wish to share such a new experimental environment and also to share a sense for the time and situation in which we are placed.
The Breathing Earth  offers a visualization of compressed earthquake data to let us see vivid movements of the earth. We convert the simple numerical data into a sort of time-lapsed computer animation, then upload the animation to our WWW site. Currently, data on earthquakes are available from the International Database Center (IDC) , which we have been accessing since 1996.
This system has been implemented as follows: An HTTP client program (Lynx) automatically gets the Web page of the IDC (the recent events) every day and extracts the earthquake data we need into a file. Then we create two files in which a 3-D model of bubbling earth images is described (the morning and afternoon version) and run the Povray program (a ray-tracing renderer without GUI) to create dynamic images. These are successively integrated in the animated GIF file composed of images from the last two weeks.
Star Place  is a live system that develops our sensation of the hyper-physical speed (approx. 30km/sec.) at which the earth constantly moves within the solar system, by showing on the real-time counter the rapidly increasing distance we have moved since opening the page.
"You are not who you were"  graphically shows what percentage of the body cells of the respective persons accessing the page have been replaced since their last access. Our body cells are said to be replaced at a rate of several hundred billion a day; the access record of each user serves as the basic data for recognizing the rate of cellular replacement in the user's body.
Net Sound  system is like a "stethoscope" on the Internet, through which we listen to a pulse-like soundscape caused by the actual data traffic tapped from a local network. "Stethoscope" is not a mere metaphor, because the basic technology for this system was invented and utilized by "doctors" of the network system. Large and distributed network management specialists at Ohno laboratory at the Tokyo Institute of Technology  have been developing a tool for diagnosing network conditions which they called "Stetho." It uses the output of tcpdump command to obtain the data of their local network traffic, then the data are converted into sound sequences. Stetho recognizes various kinds of Internet protocols and each protocol is mapped to a different sound.
Stetho is implemented using two PCs. One is a BSD/OS-based Unix machine. It taps the local network to detect network traffic, and it converts the traffic data into sound using software MIDI module. The output of the PC is line level audio signal.
The other PC is FreeBSD- or Linux-based Unix machine. This PC is prepared to run RealAudio Encoder and Server. It accepts audio signal from the BSD/OS-based PC and propagates the sound to the Internet. Currently, stetho can translate nine different protocols into sound data. They are ICMP (echo request and echo reply), RIP, NTP, FTP, HTTP, NNTP, NFS, and X-window.
Web Hopper  and its installation version which was structured at the Electronic Arts Museum (Ars Electronika Zentrum) in Linz, Austria (where the majority of visitors are children) is a meta-Web contents project that shows in real time on a world atlas on a Web page (Java applet of Sensorium) the traces of Netsurfing throughout the world made by many people.
By collecting tcpdump data from a backbone network, we get IP addresses of a proxy starting point as well as destination servers which indicate the movements of Netsurfers. The traces of these Netsurfers are described according to the geographic location data translated from the IP address data obtained. Web Hopper refers to the ip2ll server  of University of Illinois for the translation of IP address into geographical data in the North American area. Here again, our program is based on collaboration with other servers.
Basing our work on this system, we installed plural Internet terminals with different color screens and a monitor showing a huge world atlas in a room in the Linz Museum. When children browse through Tokyo or London pages from a red terminal, their "red" trace is instantly drawn on the atlas (for instance, from Linz to Tokyo to London). Children can intuitively understand how they are traveling the world by using the network. As they travel across the world, the "green" and "blue" traces of their friends, who are also Netsurfing on the green or blue terminals next to them, are visualized on the same monitor.
Web Hopper has been implemented using several Perl scripts and Java applet.
As Sensorium is a collective body of numerous projects, it is impossible to outline the research behind each project individually. We will therefore touch upon the most significant influences relevant to the projects introduced in this paper.
The reference used for Breathing Earth was the "Recent Events List"  page of the project's data source, IDC, where the most up-to-date earthquake observation data are automatically plotted on a world map and presented on the Web page. The fact that they were presenting ever-changing data inspired our pursuit of unlocking the potential of kinetic and dynamic presentations. Although there are various Web sites using similar systems, we were highly impressed by the University of Edinburgh's "Global Earthquake Map"  using "Xerox PARC Map Viewer"  as its platform. This system creates a cooperative operational link between two independently existing Web servers, which showed us further possibilities for dynamic forms of expression.
We were disappointed by the fact that although the systems for these Web pages were kinetic, the data they show consist of static text. This led us to experiment with time-lapse photographic modes of expression based on these systems.
Sponsored by the company Living Earth, Earth View is a Web site by John Walker in which astronomical data are calculated the moment that a user accesses the site, and the sun's shadow, as it relates to that moment, is projected graphically upon a cloudless rendition of the earth's surface. Earth View  made us aware of two very critical points. One was about interactivity on Web pages, and the other was about integrating data from multiple servers.
Much of the content of material presented on the Web is grasped by the user through a paginated mentality, i.e., the user interacts with each page separately, on a page-by-page basis. In the case of Earth View, however, the user's interaction continues on through links from previous pages, and the initial action is initiated at the moment the page was accessed. This is also the thinking behind Sensorium's Star Place and You are not who you were.
In addition to a projection of the sun's shadow, Earth View's expressive functions provide the latest weather information, cloud imagery, and other data acquired from outside servers. This function, characteristic also of the aforementioned Recent Events, was a resource not only for Breathing Earth, but for many other Sensorium projects.
A variety of research has been conducted on the use of sound as a means of conveying information through the computer. The bulk of this research is divided between the use of sound to signal human-computer interaction and sound used in the analysis of specialized information.
The latter, known as data sonification, is the category within which Stetho , developed by the Ohno Laboratory of Tokyo Institute of Technology, falls. Stetho is characterized by its ability to recognize macro movements via sound as compared to micro details that are recognized visually. Stetho is the engine for Sensorium's Net Sound, giving the computer networking process, which is perceived by most people as being cold and inorganic, a more vivid form of expression.
Based on research  conducted by the developers of Mosaic at the University of Illinois, this system analyzes and expresses, in real time, the geographic location worldwide of accesses made to the Web server for NCSA (National Center for Supercomputing Applications). Visualization is limited, however, to the accesses made to a single server only, and does not go as far as Web Hopper in pursuing movement through links, which characterizes the dynamic of the World Wide Web.
Secondly, in order to meet the needs of specialized mechanisms such as CAVE, it lacks in common applicability. A multi-level structure real-time visualization Gopher document, "A Gopher in a Forest," for instance, is not a part of the Web contents. This research was done in 1992, in what was called the system's early stages, yet it is still a valued reference.
Based on the data of regional weather servers worldwide, Weather Tracker  is a Macintosh application that provides the most up-to-date weather information available (air temperature/wind direction, etc.) for major cities around the world, which are selected from a pull-down menu and visualized via a thermometer-like interface. A shareware application by Christopher Kidwell, version 2.3, is presently being supplied. This application reminded us that as post-WWW users of the Internet, expression over the Internet does not presuppose a Web browser. This rediscovery resulted in a project called BeWare01: Satellite, an installation connected to the Internet that expresses the earth's temperature through the sense of touch (the description of which is to follow).
Sensorium aims not at empirically proving hypotheses, but rather at creating profound impressions, of and through the Internet, that are a factor of surprise. As such, research evaluation standards do not really apply; however, to date Sensorium has received praise in two general areas: with respect to the concept and with respect to mode of expression.
The Internet does not supersede traditional forms of media; its expressive potentials are inextricably linked to its unique characteristics as a media form. Sympathy for this belief is an important element of the acclaim Sensorium has received to date. This belief is also reflected in the pursuits of members of the Sensorium staff, who have a firm grasp of the essence of the system on an engineering-related level -- Sensorium's graphic design director's university background is in engineering, while the artist responsible for sound design on Netsound and other Sensorium projects has a background as a studio sound engineer and in game software design. Sensorium's various design components transcend mere aesthetics and aim to achieve total design experiences.
Sensorium's acclaim may also stem from the fact that on the whole, material appearing on the Internet tends to belong to one of two extremes -- either very decorative and commercial or dry and academic HTML forms of expression. Projects achieving a balance of the two expressive aspects are few. The acclaim also indicates that people crave and appreciate projects that achieve this balance.
Concept-related praise focused on the concept being integrally linked to its expressive methods. A mere introduction of a concept is feasible in other forms of media. Take the premise that network data traffic maintains a form of life rhythm, for example; others also hypothesize about a 1/f degree of vibration. To our knowledge, however, there are no other real-time sound-based experiential expressions of this concept.
Praise of this inextricable link between concept and method of expression may also reflect a prevailing technical trend over the past few years in Web design. Java and Shockwave as expressive techniques are less important than what concept can be demonstrated in what form employing these means -- we are made aware of this challenge again and again.
The above evaluation summary may be somewhat weighted towards a visual design perspective, however; in the form of very simple reactions we have received feedback on our Web page, and other sources, with comments such as: "Experiencing the Network a living organism sent a chill down my spine," "I saw the Network as the image of a murmuring stream," "For the first time, I felt the earth beneath my feet is alive," and "I realized that the Internet as an information environment has a northern hemispherical bias."
How do we extend our holistic sensory experience through the Net? This is essentially an "ecological" issue, insofar as ecology is a matter not only of physical but also mental and informational relationships between human beings and nature, a matter of perception or imagination towards the bio-social environment. We never think of replacing the totality of our physical experience and the fertile process of a real journey with a virtual trip created by National Geographic.
What Sensorium seeks is to set up a "catalyst" for strong experiences on the Web, which might trigger our imagination towards the hidden dynamism of this living world (including Cyberspace and Net-sphere itself).
In terms of the design of ecological experiences, our work might have something to do with the cultural concept of traditional Japanese landscape design and art. As an example, let's take the Japanese wind-bell called "furin": a daily tool of soundscape design which is hung at the edge of an open corridor facing the outside garden. Although we enjoy its subtle sound caused by the wind, this bell is not so much a sound instrument to which we listen, but something that catalyzes our sensitivity towards the wind blowing. We notice various micro-climatic changes through the stimuli of sound; it opens our ears and the perceptive channels of our skin. What is important here is not the soundscape design itself, but the design of our senses, or of our informational relationship with the living environment.
In this sense, Breathing Earth (1), and two other projects, Star Place (2) and You are not who you were (3) (on the top page of the site), can be seen as good examples of the design of unique "catalyzing" experiences on the Web which might reorganize the structure of perception and deepen our relationship to the reality around (or inside) us.
Based on the "interactive/mass-customizing" media structure, the latter two systems offer experiences that are unique to the Web environment and that "liven" and "update" the recognition of our lives and the surrounding world that we often mistake as something stable and immobile, but that actually continues to dynamically move and change.
These are exemplifications of the basic concept of interactivity in our understanding; the typical overt reactive programs of today are not the only way to realize an interactive system on the Net-sphere. The Web experience is always based on and revitalized through the active process of individual users accessing the pages at different times.
Even Breathing Earth, aiming at visualizing earthquakes and the living earth, also aims to indirectly bring out the substance of the Internet and to focus on the holistic sensibility of living in the new information sphere.
The structure of this global experience may be captured by an image which is just the opposite of that described by Russell Schweicart, an astronaut. He says that his experience of seeing the earth "as a whole" from outside undoubtedly played a decisive role in creating a new relationship between human beings and the earth. This is symbolically expressed by the metaphor of "a jumping flea." A flea sitting on top of an elephant never thinks that the elephant is a living thing because it is too big and instead sees it as a part of an endless earth. However, once it has jumped up high in the air and looked down, the flea realizes for the first time that the elephant is not part of a boundless earth but a limited and fragile living creature like itself. This certainly expresses the current situation: a new phase that began for 20th-century human beings once they gained the privileged viewpoint of seeing the earth as a whole from outer space -- capturing the earth as Gaia and considering its limited resources and its preservation of the environment in earnest. Our attempt to monitor vital movements of the earth through the Internet, however, exemplifies the possibility of another approach to recognizing the elephant or the earth as a whole. If the view seen by the jumping flea was a whole image looked from top down, the global sensing of earthquakes may be described as looking from the bottom up achieved by the accumulation and editing of the perception of innumerable fleas feeling minute shakes of the earth as they cling to various parts of the giant's surface. Recognition of the earth by each clinging flea might be fragmentary, like each small piece of a jigsaw puzzle; the fleas are simply like the group of blind men in the Japanese proverb, touching an elephant and having different perceptions. But when they are connected by the network and begin to share their fragmented "feel" data, the whole elephant appears, as in a completed jigsaw puzzle. It is an image of the world realized in Breathing Earth, as if sensed by "a network of blind people."
(Our new project Night & Day is also designed in this way: the real-time whole image of the global terminator is to be composed of numbers of small peephole images at various sites on this planet.)
This is a clear expression of Webness: a form of recognition inherent to the Internet age and the exemplification of the birth of a new mode of experiencing the world.
It's neither a one-dimensional "bird's eye view" nor a discrete and localized "insect's eye view." It is not a "personal" structure like the telephone or conventional mail nor is it a "mass" structure like TV and publishing, even though it started from individuals' recognition (sensors). It enables the creation and sharing of the whole image, which is quite different from conventional images in its nature, and rearranges the relation between the individual and the whole in an entirely new way.
Newcomers in the Net-sphere, however, might find it difficult to intuitively understand such a shared, bottom-up formation. When looking at home pages, for example, they might feel they were turning the pages of a book, hardly aware they were using one of the world's international servers or that the contents of the world are being supported and connected mutually as in a jigsaw puzzle. For them, it is not much different from looking at picture reference books or CD-ROMs.
In addition, because current media education tends to overevaluate its effectiveness in terms of the volume of knowledge acquired and to overemphasize the use and skills of computer as tools, it appears too preoccupied by the results of media technology, such as e-mail transmission and the use of home pages and databases, to notice the mechanism or process enabling such results (as in other media). The Internet tends to remain a "black box" for many.
We therefore designed in two ways an experiment to attract users' attention to the structural substance of the Internet and its actual process of data transmission: Net Sound (4) and Web Hopper (5).
The Sensorium team has collaborated with Ohno's lab in the application of the tool called "Stetho" in setting up a public sensory (audible) platform for general Net users. You can directly feel through this system that the Net is alive and beating.
You will also feel the actual presence and activity of unknown people through the Net, for the changing soundscape reflects the change of local Net activities as they are tapped by this "stethoscope" system. If somebody starts browsing Web pages or downloading file data at one of the local Net termini, sounds relating to "http" or "ftp" protocol will increase; if you feel such a "busy" atmosphere late at night, you will recognize the unexpected presence of working people indirectly "connected" to you through the Net. In the real world, we unconsciously perceive others' presence in a space; there are few active channels in cyberspace. Such a system therefore serves in various ways as cyberspace "sense-ware."
This "sense of connectivity" (sensing the "real" presence of people beyond cyberspace) and "sense of the Net" itself to the process and structure inside the "black box" are vital points also demonstrated in Web Hopper. As people share one cyberspace with others, they can appreciate in real time the sense that the accumulation of their respective information exchange is making up the substance of the Internet. Could this be another standard for thinking about the "transparency" of network media?
The jigsaw-puzzle structure is also reflected in the fact that most of the contents of Sensorium are essentially "inter-Web" projects: they are functioning through the continuous collaboration of several independent Web sites, as mentioned before on the relationship between Breathing Earth/Sensorium and IDC. (Needless to say, every site is basically an "inter-router" project.)
We are running the automatic rendering program for the CG animation of Breathing Earth based on 3-D software called "Povray," which we found on the network. Our project is composed of tools and data on the Net, functioning on the Net, and hopefully will contribute something important to the Net.
Who is the real creator of this program, then? Who owns this site? To whom do the contents belong? Just like many other Internet contents, Sensorium cannot be defined or described in the conventional language of modern art or of technological products, which are based on the concept (or illusion) of creative "individuals." Web sites are, in their nature, a common public asset, alive and supported by the shared Net-sphere.
In this sense, we would like to express our respect to the earliest creators of the Internet as a global tool, to whom Sensorium is dedicated. Like Mother Nature, the system of the Internet works as if the sheer contribution of each participant was accumulated then harmonized. We can't go beyond this fundamental reality, even if we were able to create extremely interesting contents. Therefore, as a participant of this system, we thought that our role and the best way to participate would be to actualize the available possibilities. The possibilities of the Internet world, the new world, are our own possibilities.
BeWare 01: Satellite (the prototype model has been on exhibit at Ars Electronica Center/Linz since September 1997)  is an installation which is connected to the Internet by IP. The data transmitted from the polar orbit satellite of the National Oceanic & Atmospheric Administration (NOAA)  are collected during its orbit of the earth. The image gained is projected onto a 9 cm by 160 cm plate, and the images change according to the speed of the satellite. The infrared images are analyzed as temperature data which are used to control the Peltier device attached under the plate. By touching the plate, one can feel the actual temperature of each part of the globe. Therefore, BeWare can be called a living object. BeWare is trying to expand means of expression using the Internet as well as to stimulate our imagination toward the living Earth.
A display monitor, a mouse, and a keyboard -- those three peripheral items of computer equipment won't remain as features of the computer interface. They are just temporary features on the path of development. Regarding the WWW interface, there is no reason why it should be used merely as a browser. In addition, current PC interfaces are biased towards visualized information terminals. NetSound was the antithesis of this in terms of using auditory senses. BeWare is an experiment created to pursue the possibilities of Web contents that are based on our empirical knowledge, more physical and more intuitive.
Via the computer network and systems connected with devices that enable us to "feel" the actual world, we can deepen experiences unique to the Internet world as well as make possible devices and the interface become real. These are the themes of Sensorium, which we have discussed in this presentation of our project.
How do we categorize our Sensorium projects? Are they edutainment or scientific teaching materials? Or entertainment? Art? We think the projects encompass all of these. We merely enjoy causing the "fermentation" of the Internet itself by creating concrete contents.
However, the Sensorium team strongly feels that their experiments could be used as teaching materials, entertainment, and art. These possibilities will be realized when we meet future prominent partners.
We originally started this "Sensorium" project as the Japan Theme pavilion of IWE96 (Internet World Expo 96). Here we must express our gratitude to Prof. Jun Murai and WIDE project of Japan and also to the Japan Committee of IWE96 for giving us this opportunity. Currently we are running this project in collaboration with WIDE project  and IMRF (International Media Research Foundation) .
 Internet 1996 World Exposition: the first global Expo on the Net (IWE96), http://park.org/
 Ars Electronica Center, Ars Electronica 97, http://www.aec.at/fleshfactor/
 Sensorium, Breathing Earth, http://www.sensorium.org/breathingearth/
 Prototype International Data Centre (IDC), http://www.cdidc.org/
 Sensorium, Star Place, http://www.sensorium.org/starplace/
 Sensorium, You are not who you were, http://www.sensorium.org/
 Sensorium, Net Sound, http://www.sensorium.org/netsound/
 Tokyo Institute of Technology, Ohno laboratory, http://www.ohnolab.org/en/
 Sensorium, Web Hopper, http://www.sensorium.org/webhopper/
 University of Illinois Pablo Research Group, Host name to Latitude/Longitude, http://cello.cs.uiuc.edu/cgi-bin/slamm/ip2ll/
 Prototype International Data Centre (IDC), Recent Events List, http://www.cdidc.org:65120/web-bin/recentevents/
 The Department of Geography at the University of Edinburgh, Global Earthquake Map, http://www.geo.ed.ac.uk/scratch/quake_all.html
 Xerox Palo Alto Research Center (PARC), Xerox PARC Map Viewer, http://mapweb.parc.xerox.com/
 John Walker, Earth View, http://www.fourmilab.ch/cgi-bin/uncgi/Earth/action?opt=-m
 Tokyo Institute of Technology, Ohno laboratory, http://www.ohnolab.org/researches/stetho/index.html
 Lamm S.E., Reed, D.A., and Scullin, W.H. "Real-Time Geographic Visualization of World Wide Traffic," http://www-pablo.cs.uiuc.edu/Projects/Mosaic/WWW3/
 Christopher Kidwell, WeatherTracker, http://www.weathertracker.com/
 Sensorium, BeWare01: Satellite, http://www.sensorium.org/beware01/
 National Oceanic and Atmospheric Administration (NOAA), http://www.noaa.gov/
 WIDE project, http://www.wide.ad.jp/
 International Media Research Foundation (IMRF), http://www.imrf.or.jp/