Figure 1: The audio agent
Peter Parnes <Peter.Parnes@cdt.luth.se>
Mattias Mattsson <Mattias.Mattsson@cdt.luth.se>
Kåre Synnes <Kare.Synnes@cdt.luth.se>
Dick Schefström <Dick.Schefstrom@cdt.luth.se>
Luleå University of Technology
Sweden
This paper presents an easy-to-use service creation platform for realization of applications handling real-time streams and data. This platform, called WebDesk, is implemented using the platform independent Java language. WebDesk includes an application-level network layer, called the Tunable Multicast Platform, or /TMP which is presented, as well as a couple of applications exemplifying the usage of /TMP; the audio and whiteboard agents, and mWeb, an application for scalable distribution of HTML files and synchronization of Web browsers.
Keywords: MBone, WWW, distributed applications, distance education, Java, reliable multicast, whiteboard, audio.
This paper presents WebDesk, a framework directly targeting this goal.
WebDesk is a framework for scalable collaborative applications on the Internet using the Multicast Backbone (MBone) [IP-multicast, MBone]. The framework is divided into an application-level network layer, called the Tunable Multicast Platform (/TMP), and a number of application components called agents.
Building a distributed collaboration environment is complicated, and it is therefore necessary to be able to integrate various existing tools and components, making them useful for distributed work. The purpose of such integration is:
Those goals are often in conflict, and a balanced compromise is a central design problem.
An agent is a program module responsible for a single and independent task. Such a task can either be self-contained within the agent or be implemented by integrating an existing tool. In the latter case, the agent acts as a mediator between:
The internal tool language is usually the Control Bus protocol (see section 2), and an example of an external tool language is the remote-control-api used to control the Netscape Web browser.
Examples of self-contained agents are the WebDesk audio, membership, want2talk, status, talk, and whiteboard agents, and an example of an integration agent is the WWW agent that controls external Web browsers.
WebDesk is also an application combining a number of agents into a useful conferencing tool. Another example of how agents can be combined into applications is mWeb, presented in section 5.
Previous results on WebDesk can be found in [WebDesk].
The rest of this paper is divided into the following sections: section 2 presents the /TMP, section 3 presents the audio agent, section 4 presents the whiteboard agent, section 5 presents the mWeb application, section 6 presents the current implementation and status, and finally section 7 presents a summary and conclusion about the WebDesk framework.
The library also contains implementations of two new protocols developed at the CDT: Scalable Reliable Real-Time Transport Protocol (SRRTP) [RTP/SRM] and Scalable Reliable File Distribution Protocol (SRFDP). These protocols add support for reliable multicast and file distribution. SRRTP is based on the ideas from the SRM framework [SRM].
The /TMP also includes support for communication between agents using the WebDesk Control Bus (CB) [CB]. The CB is a simple, but still powerful, mechanism for messaging between agents. The message format allows for addressing of a single agent or a selected group of agents. It is also used for interagent communication within the WebDesk application.
The audio agent is connected to other instances of audio agents through the CB. It is thereby possible to control an audio agent remotely over the network, and audio agents may also perform quality-of-service negotiations between each other (e.g., agree on a common format based on transmission quality feedback).
The graphical user interface (see figure 1) is implemented in Java. The Java package can be used outside the WebDesk to implement other audio applications (e.g., Internet telephony through usage of the Session Initiation Protocol [SIP] and the Session Description Protocol [SDP].
Figure 1: The audio agent
Delay equalization through buffering is a known remedy for audio transmissions over the nonisochronous Internet. When packets are received out of order or delayed, the buffer smooths out the transmission problems and instead introduces an increased latency. The International Telecommunication Union (ITU-TS) requires a very low end-to-end delay for normal conversations, while a longer delay can be accepted for lecture-style conferencing. To keep latencies to a minimum while achieving increased quality, we introduce semireliable audio. By using SRRTP, we add a new set of characteristics to audio transmissions:
Traditionally, when packets are lost, the audio stream is repaired with silence. This is not an optimal solution, since even noise is a better substitution for lost data. Retransmissions, as in the usage of SRRTP, decrease the number of lost packets, but a method to repair losses when this also fails is needed. Perkins et al. [RTP-RAD] have devised an RTP Payload for Redundant Audio Data, in which lower-quality audio data are piggybacked on the main higher-quality data. By using this RTP format, a good balance between redundancy and retransmissions is achieved (in which retransmissions of higher-quality data are not necessary when lower-quality data have already been received). If this also fails, losses are simply repaired with old data with lower volume (as in some GSM implementations).
The degree of redundancy and retransmissions is determined by the network load. Congested networks do not react well to either retransmissions or larger packets with redundancy.
Shared and distributed whiteboard functionality is central to any collaborative environment. This functionality is added to the WebDesk by the generic whiteboard agent. The functionality of this agent includes simple drawing, miniature views, telepointers, and user-programmable and dynamic drawing objects, as well as support for importing images and text. This agent can be combined with other Java programs, such as document viewers and Web browsers, to achieve annotation and conferencing support for those programs.
All drawing primitives in the whiteboard, such as lines, rectangles, or freehand drawing, are Java objects. They are responsible for saving and loading their state to and from byte streams, drawing themselves on graphic contexts, and constructing themselves from user input. This minimizes the connection between the whiteboard and the drawing objects. It is even possible to add new drawing primitives at run time.
A whiteboard session shows one document. This document is divided into several pages which, in turn, are divided into several layers. The pages are mapped to the data being annotated (see section 4.2). For example, if a text document is being annotated, it is natural to have each whiteboard page correspond to a text page. If other types of data are being annotated, other mappings may be more appropriate. The layers of a page can be thought of as transparent slides on which the users draw. All layers are combined to form the page, and layers can be turned on and off separately. It is also possible to mute certain members. All drawings made by these members will then disappear until the member is unmuted.
The whiteboard can be used as an annotation tool with which the user annotates a document. These annotations can be distributed as ordinary files for others to see or can be loaded in conferencing mode, which will cause the annotations to be multicast to all participants.
By utilizing the multicast support in WebDesk, an arbitrary number of participants can conference around a chosen document. Since multicasting of the data is chosen, there is no master copy of the session; each participant has a copy of the data.
The annotation and conferencing modes are of course not distinctly different. It is possible to conference around an annotation and to save a conference around a document as an annotation.
We have made the whiteboard as general as possible, so it can be used together with arbitrary Java programs with a minimum of modification. When the whiteboard is integrated with a Java program, it is possible to use the whiteboard's features (e.g., conferencing or annotation) on top of that program's display. The programs that can be integrated with the whiteboard are typically viewers of different types of documents (e.g., text, PostScript, and image viewers) but may be any type of program. Since text and most type of images are common and easily handled, support for these is built into the whiteboard.
When the program that the whiteboard has been integrated with does not cooperate with the whiteboard, only ordinary annotation of the documents may be done. That is, the whiteboard annotations have to be drawn at fixed coordinates with no semantic ties to the underlying document. This is often sufficient for static documents such as text or PostScript, whose layout never changes.
A more interesting approach is to have some cooperation from the program, which allows for a better integration and more functionality. For example, if the user of a Web browser whiteboard resizes the Web browser or changes the font size, the Web page's layout changes. This means that annotations at fixed coordinates becomes unintelligible, since they no longer are connected to the paragraph, image, or object they were intended to annotate. Therefore, a semantic tie between the annotation and the object that is annotated is needed. This semantic tie would allow the annotation to follow the object it annotates even if the layout of the page changes.
To achieve this semantic tie, cooperation from the Web browser is needed. When the user makes an annotation, the whiteboard asks for the object at certain coordinates (i.e., where the user annotates). The Web browser then makes a guess of which object the user wants to annotate and returns this object along with its coordinates. These coordinates will then be the origin of the coordinate space for this annotation. When the user causes a change in layout, the annotation will be redrawn at the same coordinates relative to the Web object it has been attached to and will thus retain the semantic intentions the user intended.
The major problem lies in guessing which object the user intended to annotate. The object closest to the point where the user draws is not always the object that the user intends to annotate. For example, when circling objects in freehand drawing, users often start the drawing at some distance from the object that is to be circled. This means that the tie between the drawing and the Web object has to be dynamic and that the user must ultimately be able to override the program's decision in an easy way.
The user can see which object the annotation has been tied to for the moment, since the currently selected Web object becomes highlighted. By constantly querying the Web browser as new drawing data comes in, it is possible to alter a first guess into a more reasonable one. For example, in circling a passage of text, the user might start near the image when intending to circle the image text. This would cause the program to guess that the intention was to annotate the image. As the user finishes the circling, it becomes clear that the bounding box of the annotation is intersecting the text's bounding box more than the image's. This would cause a change of semantic tie from the image to the text.
Miniature views are small windows without drawing capabilities that show a certain page of the document being annotated. These windows give the user an overview of what is drawn on a specific page rather than detailed information. Since the human eye is sensitive to movements, it is easy to see on which miniature view an annotation is being made even if the view is so small that the user cannot see exactly what is being done. This allows a user to see the whole document, or interesting parts of it, without occupying as much of the precious screen space as a full-blown view would. Work on miniature views has been done in [Gutwin96].
The user can resize the miniature views so that interesting views show more detail than other, less interesting ones. It is also possible to jump to a certain page by clicking on a miniature view. This allows for faster change of page than if the user first must determine what page the miniature view shows and then go to the page selection controls to change to that page.
The telepointer is a remote mouse cursor that allows a user to point and gesture at a certain part of the whiteboard. This is important because the users of the whiteboard are normally physically distant and it is very hard to direct someone's attention to a certain part of the screen by just describing it.
The mWeb application is a tool for real-time distributed presentations with HTML as its presentation medium. The application includes functionality for distribution of HTML pages, including in-line data and embedded objects, precaching of files to be used within a session, on-demand fetching of files, synchronization between browsers, and interfacing different Web browsers. It uses the WebDesk framework for distribution and control.
The problem of adding real-time distribution of HTML to the World Wide Web (WWW) can be divided into two parts: synchronization and distribution. This section discusses the architecture of the mWeb application and how these problems have been solved in mWeb.
The mWeb application acts as a gateway between a Web browser and the MBone (see figure 2), mediating distribution of HTML pages and display messages (see section 5.2). The application can also run in a lightweight mode, in which only the URLs to be displayed are multicasted. This is useful in smaller groups, as the delay becomes shorter and the network usage does not change significantly.
Figure 2: The mWeb architecture
HTML pages to be displayed during a session can be collected in three ways:
The last two methods can also run in so-called cache mode, meaning that the presenter stores the collection of URLs in a file before the actual presentation is held (this is, of course, a special case of method 1). This file can then be edited by the presenter to add or remove URLs. This is useful if a page that is to be displayed contains an embedded object, such as an Java applet (see section 5.3).
During the presentation a window containing a list of displayed pages is shown (see figure 3). On the presenter's side this list will contain all pages, but on the audience's side only pages that have already been displayed are listed.
Figure 3: The mWeb application
When a presentation is distributed over the MBone and a Web browser is used for presenting the slides, there is a need for synchronization between the involved browsers (so that all of them display the same page). This is solved by sending a display message to all members of the group by means of the CB (see section 2).
During the session, the listener can select what will happen when a new display message is received: display the new page directly, display a dialog box asking the user what to do, or just display in the page list that a new display message has been received.
A listener who wants to go back and view an already-displayed page can select the page of interest in the list over displayed pages, and that page will be displayed locally. A listener can also instruct the local mWeb client to send a display message to all other listeners, including the presenter. This is useful if the listener wants to comment or ask a question related to a page that is not currently displayed.
It is easy for a program to scan an HTML page for references to in-line images, as these are stated clearly in the file, but it becomes more complicated if the page contains embedded objects, such as Java applets. A reference to an embedded object only points out the first element of what could be a complex program.
The mWeb application also allows for remote pointers by using so-called bullet applets in the slides. These applets connect back to mWeb when displayed in an HTML page. The presenter can then send control messages using mWeb to indicate which bullet is currently active. The active bullet is then presented to the other users.
The mWeb application can also be used in so-called programmed presentation mode, in which the presenter instructs mWeb to send display messages for a set of URLs at certain times. This can be used to create a televisionlike system in which the receiving browsers display a group of pages. We call this functionality Web-TV. This functionality could be used for dissemination of information within one group of listeners (e.g., to spread important information within a company or organization).
Further information about mWeb can be found in [mWeb, mWeb-RTMW96].
Several different applications and agents have been built to date on the WebDesk framework. Some prototype applications are in daily use for project work and during electronic meetings.
The mWeb application is currently being used, with very positive experience, in a major effort called Utbildning Direkt (Education Direct). Among other things, the system is used within the project for distribution of an undergraduate course in distributed multimedia to high schools and other interested parties in the local county.
This paper presents a scalable framework for collaborative work over the Internet, called WebDesk. This framework allows for simpler creation of new scalable collaborative tools but also makes the task of integrating and extending existing tools much easier.
The WebDesk application-level network layer, the Tunable Multicast Platform (/TMP), was also presented. The /TMP includes functionality for distribution of data in a reliable and scalable way using multicasting and new protocol for transport reliability.
Three tools implemented using the framework were also presented: the audio agent, the generic whiteboard agent, and the mWeb application.
WebDesk has proved to be an efficient and easy-to-use framework for creation of Java-based distributed applications and tools.
Thanks should go to Ulrika Wiss, CDT, for interesting comments and encouragement.
This work was done within Esprit project 20598 MATES, which is supported by the information technology part of the 4th Framework Program of the European Union. Support was also provided by the Centre for Distance-Spanning Technology (CDT).
<URL:http://www.cdt.luth.se/~peppar/docs/>
<URL:
http://www.ncsa.uiuc.edu/SDG/Software/XMosaic/CCI/cci-spec.html>
<URL:http://www.cpsc.ucalgary.ca/projects/grouplab/papers/
Usability/cscw-final.html>
<URL:http://www.mbone.com/>
<URL:http://www.cdt.luth.se/~peppar/progs/mWeb/>
<URL:http://www.cdt.luth.se/~peppar/progs/mWeb/>
<URL:ftp://ds.internic.net/internet-drafts/draft-perkins-rtp-
redundancy-02.txt>
<URL:http://www.cdt.luth.se/~peppar/docs/rtp_srm/>
<URL:ftp://ds.internic.net/internet-drafts/draft-ietf-mmusic-sip-01.txt>
<URL:ftp://ds.internic.net/internet-drafts/draft-ietf-mmusic-sdp-02.txt>
<URL:http://www.cdt.luth.se/~peppar/docs/>