Manfred Bogen (Manfred.Bogen@gmd.de)
Arnold Krechel (Arnold.Krechel@gmd.de)
Up to now, messages and e-mail services are mostly text- and ASCII-based. With the availability and high user acceptance of multimedia-based services, the need for the transport of multimedia objects arises even in message-based applications.
In the BERKOM environment, a PTT-driven project to realise the communication infrastructure of Berlin, a superset of the X.400 protocol has been implemented to overcome deficiencies of present X.400 and the handling of large multimedia messages has been defined. GMD has been awarded by the DFN association being responsible for major mail components within the BERKOM environment to implement a multimedia mail gateway in this context.
In this paper the development of this gateway (MMMGate), is described. Major attention is paid to the handling of large multimedia messages, the management of partial messages and the co-operation with a global store. The experiences made and the planned embedding of the gateway into the national multimedia reference centre and into user-driven projects is presented finally.
2 The MMMGate Development
3 Results and Experiences
People get 80% of information by sight, 11% by hearing and 3.5% by smell (which is not supported by any multimedia format today). From this information they retain 20% of the information they see and 30% of the information they hear . This means that people heavily rely on images in their information acquisition and that exchanging multimedia information is efficient. The price to pay are increased storage and bandwidth requirements (table 1).
Table 1: Document Lengths
text and graphics 200
stored video clips > 1.000
interactive video clips > 1.000/min
Multimedia can be classified as interactive and real-time on one side and message-oriented and asynchronous on the other side. Speaking about e-mail in the following we restrict ourselves to the latter category as message-oriented multimedia obviously is based on message transfers. These messages are called compound, hyper or multimedia documents as they not only contain the information itself but also structural information about the document. Multimedia documents can contain
Figure 1: Overall Messaging Model
We can classify e-mail systems into three different categories depending on their operation (figure 1):
In the local area networks people use desktop and enterprise e-mail systems, in the wide area networks enterprise and backbone e-mail systems.
Most of the desktop e-mail systems like QuickMail, Microsoft Mail, Lotus cc:Mail, or NOVELL Global MHS use proprietary, non-standard e-mail protocols and formats internally. Thereby it is possible to offer a functionality and complexity for end users which is perfectly adopted to the proprietary architecture and system.
For the path through wide area networks they use directly, through gateways or through their enterprise e-mail systems (figure 1) SMTP and X.400 for the message transport. The same applies to CSCW software like Lotus Notes or Microsoft Exchange when they use embedded e-mail capabilities. For enterprise and backbone systems SMTP and X.400 is a must. By using SMTP and X.400 as open standards in the enterprise and backbone systems the e-mail backbone is kept simple.
In spite of the simplicity of the backbone, these systems must support multimedia body parts because otherwise the enhanced functionality of the desktop systems and information as such would be lost by passing the e-mail backbone.
The effective handling of large messages (see table 1) may not be important within an organisation in a local area network and individual solutions are acceptable here. But it becomes very important for world-wide enterprises in wide area network and for people using the e-mail backbone. Due to the simplicity principle open solutions have to be implemented here too.
[RFC1521] enhances the old format of ARPA Internet text messages [RFC822] without changing the related SMTP [RFC821], i.e. without implications on the message transfer
These capabilities to handle large messages with multimedia body parts effectively are important for the quality of e-mail systems . The question which arises immediately: does X.400 as second open standard in a simple e-mail backbone support similar capabilities? The answer is: [X.400-84,X.400-88] does not. The multimedia body types are supported in principle, however the precise definitions are still missing ("for further study").
Figure 2: The BERKOM Multimedia-Mail Teleservice
The BERKOM Interpersonal Messaging Service (IPMS) is part of the BERKOM Multimedia Mail Teleservices (figure 2) and it supports (externally-defined) body parts for
Other Basic Body Part Types used in BERKOM are ia5text and message.
A mechanism (reference manager) was implemented to handle external references in message bodies via distinguished object references (DORs) [DOR] and to access a globally available store service, the global store server.(1) A special ISO file transfer protocol, Referenced Data Transfer [RDT], is used for all components of the external reference mechanism.
As long as the BERKOM people only communicated with partners in the BERKOM and native OSI world, everything worked fine. Finally it was decided to enable interoperability with the large Internet community.
The DFN association started a call for tender in December 1993. A GMD consortium, consisting of two GMD groups (the FOKUS institute in Berlin and the Research Department for Network Engineering (NW) in Sankt Augustin(2)) and the Fraunhofer Institute for Computer Graphics in Darmstadt, was awarded in March 1994. The project started in July 1994 and was finally completed in May 1995 after the approval of the results by the DFN association and the DFN multimedia teleservices reference centre at the Technical University of Dresden. The Fraunhofer Institute was responsible for the development of graphic converters, GMD's FOKUS institute in Berlin for the file transfer component and the NW group for the mail component and project management.
Following the specification MMMGate should logically be located in between X.400 and X.400(BERKOM). We decided to respect this in our design strictly. The gateway from X.400 to X.400(BERKOM) should be called if and only if a message has to be sent to a BERKOM MTA (via X.400) and has arrived at the current MTA from a non-BERKOM SMTP or X.400 MTA. The other way round an X.400 message from BERKOM should pass the X.400(BERKOM) to X.400 gateway if and only if it has been accepted from a BERKOM MTA and has to be sent to a non-BERKOM MTA via SMTP or X.400. Both parts of the gateway should operate on the complete P22 content of the messages.
Figure 3: MMMGate Architecture and Embedding
We finally wanted the gateway to operate primarily, i.e. by default on body part level eventually thus following more the philosophy of PP.
Though we saw their points as well we had to deal with the problem of partial messages at our gateway as the BERKOM community is purely X.400 located and will never have any knowledge on MIME partial messages.
The overall design of the partial message assembler is a client/server architecture (figure 4).
A MIME partial message content part will be mapped to a Generic MIME defined Extended Body Part (GMEBP) in X.400 by PP's MIME gateway according to [RFC1494,RFC1495]. If the BERKOM gateway detects such a body part in the P22 content of the message (using the content-type and content-parameters of the parameter part of the GMEBP (see paragraph 5.2 [RFC1494])) several steps will be performed.
Figure 4: Handling of Partial Messages
pmaserv: partial message assembler server
pmacl: partial message assembler client
cmfserv: composed message forwarding server
cmsubmit: composed message submitter
The result of this assembly is an X.400 message with the following drawbacks:
The information on these P1 recipients is passed to the steps 4 and 5 which have to follow now.
Figure 5: Modification of PP for MMMGate
For an incoming X.400 message PP computes it's internal routing on a per recipient basis according to the encoded information types contained in the P1 envelope of the message and it's configuration. For each recipient the internal routing consists of a list of PP channels. These channels process the message in turn under control of PP's queue manager QMGR. To the list of channels computed by the SUBMIT process we had to add our gateway channel (from X.400 to X.400(BERKOM) or vice versa depending on the concrete situation).
This could be achieved by a single subroutine call which had to be inserted in the PP sources. Along with this we assigned the queue id of the message to an additional global variable which is used by the FTP/RDT part of the gateway for the co-operation and communication of non-correlated processes operating on the same message.
Our modifications of PP could remain very small and should not lead to real problems when adapting the gateway to subsequent releases of PP.
Table 2: MIME to X.400(BERKOM)
MIME Body Parts X.400(88) Body Parts X.400(BERKOM) [RFC1521] [RFC1494/1495] Body Parts  text/plain; EBP-General Text message with two body parts: charset=iso-8859-x IA5Text and EBP-generaltext (in link bp description alternative) multipart/mixed message; Heading Extension mixed summary/description in link bp multipart/digest message; Heading Extension digest summary/description in link bp multipart/alternative message; Heading Extension alternative summary/description in link bp multipart/parallel message; Heading Extension parallel summary/description in link bp message/partial GMEBP; Heading Extension partial-message the message will be composed and mapped.
Table 3: X.400(BERKOM) to MIME
X.400(BERKOM) X.400(88-MIME) MIME Body Parts Body Parts  Body Parts  [RFC1494/1495] BEBP-link ia5Text text/plain
Table 4: Reversible Mappings
MIME Body Parts X.400(88-MIME) Body X.400(BERKOM) [RFC1521] Parts [RFC1494/1495] Body Parts  text/plain; ia5Text ia5Text charset=us-ascii message/rfc822 message BP message BP message/external-body GMEBP BEBP-External Reference(DOR) application/ bilaterally-defined BEBP-Binary octet-stream application/postscript MEBP-postscript BEBP-document (postscript) image/gif MEBP-gif BEBP-Image (IIF - Colour look up table, uncompressed) image/jpeg MEBP-jpeg BEBP-Image (IIF-greyscale or JPEG-coloured) audio/basic GMEBP BEBP-audio (G.711) video/mpeg GMEBP BEBP-audio-video (mpeg) image/g3fax G3-facsimile BEBP-Image application/ g4-class1 BEBP-Image x400-bp(g4-class1)
Figure 6: External References at the Mail-GW (MIME2BERKOM)
Figure 7: External References at the Mail-GW (BERKOM2MIME)
Figure 8 shows the operation of the FT-GW after a MIME external reference (MEX) was translated to a DOR: the recipient of the message in the BERKOM environment decided to fetch the referenced data. The external reference manager (XRM) opens as RDT-client a connection to the FT-GW. The XRM at the gateway takes all the data needed in the DOR and opens an FTP connection as FTP client to the target FTP server. From there the data are sent back to the gateway. They have to pass the gateway's mail component (Mail-GW) in order to assure the same mapping functionality as for messages. Then they are sent as BERKOM data to the BERKOM RDT-client having started the whole process.
Figure 8: External References at the FT-GW (X.400)
Figure 9 shows the operation of the FT-GW after a BERKOM DOR was translated to a MEX: the procedures are analogous.
Figure 9: External References at the FT-GW (MIME)
The gateway component responsible for the conversion of the message structure uses this Heading Extension to check whether it is a body part of type message/rfc822 or multipart. In the latter case or if a message contains a multipart message, a MIME Heading Extension, a BERKOM Link Part and a message summary (figure 10) are produced.
Figure 10: BERKOM Message Summary
THIS IS A BODY PART GENERATED BY THE BERKOM MIME GATEWAY
Type now MIME Type Level BP No - multipart/mixed 1 ia5text text/plain 2 3 audio_g711_data audio/basic 2 4 modified by GW image_iif_data image/gif 2 5 modified by GW
Figure 11: MMMGate Test Configuration
After the gateway development itself we additionally performed some negative and stress testing with multimedia messages containing multipart/mixed with external references, GIF, JPEG, and PostScript body parts with the following results:
Apart from that we are planning an article in one of the next editions of the DFN Mitteilungen.
It's difficult to get all the examples needed for the tests as multimedia messages are hard to compose and as there are more standards and RFCs involved than expected.
We found several bugs i.e. deviations from the respective RFCs in the underlying software and reported them so that we expect improvement in the next official version (IC Release 2.1). The public-domain ISODE/PP software used had to be modified (RFC 1495). It is not really suited for a production environment. Configuration of PP is a pain which is confirmed by its supplier.
The better (and more expensive) way seems to be to have stand-alone work stations for each software developer and to have a machine for the system integration and the overall tests maintained by the system administrators.
It is very difficult to base a software development on prototype components still under development. Whenever errors or undefined conditions occur it is not possible to locate the reason on easy terms. To have the sources for each software component involved is a must.
The installation of the test configuration needed more time than expected. Our approach to install and configure all components by ourselves independent whether they belonged to the gateway development or not was wrong. It is a better way to let the people responsible for the respective components install them so that they can detect the problems directly.
As we participated in the work of the national group writing the first gateway specification  we restricted the number of meetings among the consortium members and with our contractor, the DFN association, during the project lifetime to
The rest of the co-operation and communication among the consortium partners was done through the network and via telephone. In Sankt Augustin, we had an informal, internal project meeting once a week.
In spite of the higher travel costs, it is better to have monthly consortium meetings so that problems among the people in the consortium can be solved directly and face-to-face. This is not achievable through the network or with telephone only.
We finally found out that we needed four kinds of distribution lists:
The MMMGate was developed on top of IC Release 2.0. We found problems in the internal PP routing of MIME messages to several recipients hindering a use of today's MMMGate for production purposes. We recommend to adopt it to IC Release 2.1.
From the experience with the public domain version of PP made in the establishment and operation of message-based value-added services and made in the development of MMMGate we strongly recommend to use the product version of PP as basis for the future work.
The BERKOM specification has been changed in the meantime, version 3.2 is available now. The implications on MMMGate have to be identified and necessary changes have to be implemented.
The handling of large messages can be improved in a way that MMMGate itself decides to put large messages into the global store and to send only an external reference to the recipient. This could be especially useful for composed messages being built out of partial messages. At present, this is only done by the originator of a message.
At last, to ease future developments, installation and maintenance we recommend to embed the MMMGate functionality into IC Release 2.1.
At present, MMMGate is not in a state to be embedded into a multiprotocol e-mail backbone. To overcome this we have identified a list of issues to be worked on next. However, through the MMMGate development the possibility to interconnect SMTP/MIME, X.400 and the BERKOM environment via multimedia e-mail and to guarantee the interoperability of the e-mail systems involved with minimal loss of functionality was shown. The biggest part of the work has already been done.
Arnold Krechel received his master's degree in mathematics from the University of Bonn, Germany in 1985. He worked in the area of parallelization of numerical algorithms till 1992. Since then he is engaged in the networking area, mainly concerning X.400 and its flavours. He is responsible for the operation, customisation and integration of the X.400 software at GMD, he performed the successful transition from EAN to PP in 1992 and he is an active contributor to the development of the ISODE and PP packages.
The authors can be reached at GMD - German National Research Center for Information Technology, D-53754 Sankt Augustin.
(2) We had been and still are responsible for the establishment and operation of DFN multiprotocol value-added services.
(3) We doubted this in the beginning.