An Approach to IP Telephony Performance Measurement and Modeling in Government Environments

Hsiaosu HSIUNG <hhsiung@mitretek.org>
Martin J. FISCHER <mfischer@mitretek.org>
Denise M. MASI <dmasi@mitretek.org>
Darren CUFFIE <dcuffie@mitretek.org>
Scott SCHEURICH <sscheuri@mitretek.org>
Mitretek Systems, Inc.
USA

Abstract

The General Services Administration Federal Technology Service (FTS) has recently been awarded the FTS2001 contracts. One of the services being offered under FTS2001 is Internet protocol (IP). To ensure that the federal government will continue to save on telecommunications costs in the future, the IP telephony is one area currently under investigation, especially international telecommunications services. Many voice over Internet protocol (VoIP) system issues, which include scalability, performance, integration, and cost, are considered. This paper focuses on the performance issues; in particular, the network access performance. We discuss the test bed and the computer models that have been developed to study the performance of VoIP in a government environment. Typical examples are presented as well as a discussion of our future plans.

Contents

• 1.Introduction
• 2. IP telephony testbed
• 3. Modeling capabilities: simulation and analytic approximations
• 4. Future plans
• References

1.Introduction

The Internet is fast becoming the preferred method of communication. This communication can be in the form of secured virtual private networks or communication over the public Internet. To date, the forces driving the need for Internet technology enhancements have been coming mainly from end users. However, business-to-business transactions will control most of the electronic commerce (e-commerce) market in the near future. The growth of the Internet is expected to rise as the number of U.S. and worldwide Internet users continues to increase; growth is currently at 100% per year for user counts, and the equipment capacity growth rate is doubling every 12 months. However, the Internet core traffic is growing at a rate of 1,000% per year, which is equivalent to a rate of 100% per 100 days. Results show that the available bandwidth is not keeping up with the demand. More users are staying on the Internet longer and the same users are using higher-access bandwidths as well.

Internet applications can be grouped into two categories: real-time and non-real-time. Real-time applications typically demand priority bandwidth within the network over a sustained period of time. Real-time applications are predominantly interactive/collaborative tools and "live" video/audio streams coming across the Internet. There are also real-time data-only applications, such as financial transactions and electronic stock trading that are increasing at a phenomenal rate. Non-real-time applications include such favorites as e-mail, compressed/digitized pictures, HyperText Markup Language files, and any documents in a textual format. The non-real-time documents require only bursts of network bandwidth over a short period of time.

In earlier days, the Internet was mainly a collection of servers and personal computers (PCs) hosting text-based files. As the proliferation of Internet technology increased, the ability to add graphics and animation soon followed. Web browsers have increased in flexibility and scalability to incorporate many functions previously limited to local area network (LAN)-based communications only, thereby producing a new form of collaborative efforts over the Internet. In addition, the ability to communicate has incorporated voice traffic over the Internet as well. The revolution has slowly started to shift toward convergence of data and voice networks sharing common transport facilities. The evolution of Video Over Internet Protocol and Voice Over Internet Protocol (VoIP) products has drastically increased in scale and popularity to the point where business users can communicate effectively, using these highly advanced collaborative tools. The increase in audio and video streaming over the Internet has led to such applications as radio and TV simulcasts over the Internet. Internet Protocol (IP) technology has also created Web-based call centers, where consumers can interactively use voice and video sessions on a Web page and have a call center agent engaged in business transactions looking at the same data as the consumer.

The growth of the Internet has presented many opportunities for the federal government. At the same time, it has brought many challenges for us to consider. The General Services Administration (GSA) Federal Technology Service (FTS) has just awarded the FTS2001 multibillion-dollar contract to Sprint and MCI WorldCom. In the later years of this 8-year contract, the cost per minute of voice service is under $0.02. Given this price, one could question the economic justification of introducing Voice Over Internet Protocol (VoIP) into FTS2001. However, IP is one of the services being offered under FTS2001, and the feasibility of agencies developing their own virtual VoIP networks under the FTS2001 umbrella is of interest to GSA. To get a better understanding of the technical and systems-level issues associated with the development of virtual VoIP within FTS2001, GSA and Mitretek Systems have become partners in a research program aimed at studying VoIP. This paper documents the program and the progress that has been made to date; the paper also discusses future plans.

Section 2 discussed the VoIP testbed that has been set up and some of the tests that have been conducted to date. Section 3 presents some of the computer models that have been developed and outlines future analyses that are being undertaken. The paper closes in Section 4 with a discussion of our future plans. GSA and Mitretek Systems are interested in exploring joint testing arrangements with other government agencies at the federal, state, or local levels that are interested in developing a better working knowledge of the potential applicability of VoIP in their environment.

2. IP telephony testbed

Within Mitretek Systems' Advanced Telecommunications Laboratory (ATL), a VoIP testbed has been set up. The testbed consists of two PC clients, two IP gateways, four analog phones, and a PC-based private branch exchange (PBX) (see Figure 1). The purpose of the testbed is to measure and calibrate traffic generated from the voice conversations on the PCs and the analog phones. To collect this information, a packet sniffer, Observer from Network Instruments, has also been installed in the VoIP testbed.

Figure 1: IP telephony testbed connectivity

The IP gateways use the standard H.323 call control protocol and several standard codecs: GSM120, G.711, and G.723.1. The PC-based PBX served as a voice switch in the testbed. The PC clients were connected to a hub with private IP address space. Different codecs were used in measuring the packet delay, jitter, packet loss, and bandwidth utilization.

Both PC-to-PC and phone-to-phone testing was conducted in an intranet environment. In the phone-to-phone experiment, the end-to-end delay was measured as 274 ms. A large portion of the delay is due to the analog/digital conversion, compression, packetization, and OS overhead in the IP gateways. When faster digital signal processors (DSPs) become available, the delay introduced by the equipment should be significantly reduced. The purpose of conducting intranet testing is to provide a performance baseline. When this experiment is expanded to the Internet, the additional delay elevated by Internet can be better understood.

Table 1 represents the results from a 30-second half-duplex voice session. The columns are packet sizes. With the G.723.1 coder, 81.2% of the packets from the Listener were 64 to 84 bytes in size. When using G.711, the talker sends data out in 310 frames. The remaining packets from the talker to listener are called control packets from the application ranging in size from 64 to 128 bytes. When using the G.723 codecs, the data are in 78-byte frames. When the listener sends data to the talker, the packet size is between 84 and 100 bytes, regardless of the codec. Other packets are from the Observer software or are call control packets from NetMeeting.

Additional tests will be taken in a mixed voice and data environment. The data behavior is highly application dependent. Instead of using a generic data generator to produce the necessary data load, a script language, SILK by Sagueway, which emulates the application transactions, will be programmed to produce Microsoft Word, Microsoft Excel, Microsoft PowerPoint, and Web traffic. It is felt that combinations of these application transactions represent the actual network data behavior better than the traffic generated by the network management device. However, the emulation approach is appropriate only for a reasonable system size. For a large-network system, simulation and analytic models are required to facilitate the analysis.

3. Modeling capabilities: simulation and analytic approximations

To better understand Quality of Service (QoS) in a VoIP access arrangement, we felt it was very important to build some computer models that would allow us to analyze, study, and size these access arrangements. We built two computer models: one a simulation model using Extend software and the other an analytic approximation using results from queueing theory. In this section, we discuss the simulation model that we developed and present some representative analyses using that model. Next, we present the analytic approximation and compare it with the results obtained via the simulation. The comparisons were very good.

The VoIP access system of interest is shown in Figure 2. It consists of a LAN and a line providing access to an external network such as the Internet or FTS2001. We will denote the external network as the "cloud." The LAN is assumed to be an Ethernet (e.g., 10BaseT or 100BaseT). The LAN connects m PCs that can generate voice or data traffic. The line providing access to the external network contains a gateway to which n phones are attached. This gateway handles voice compression and decompression as well as other functions. The line to the external network could be a T1, fractional T1, or T3, for instance. The parameters m and n, Ethernet speed, line speed to the external network, and voice call compression/decompression scheme used are inputs in the simulation. If a PC generates a voice conversation, it is assumed to have the appropriate card to handle voice compression/decompression. Our modeling focus is on transmission and switching rather than the impact of different compression/decompression schemes.

The simulation model allows voice calls to be generated at the phones and the PCs. These calls can be routed to PCs, phones, or the external network. The calls are broken into "talkspurts" and silence periods, as described in Sriram and Whitt (1986). The number of packets in a talkspurt is geometrically distributed, where the mean number of packets in a talkspurt depends on the codec used. A talkspurt length of 352 ms is assumed (Sriram and Whitt, 1986). For example, a G.729 codec with a frame size of 10 ms gives a mean of 35.2 packets per talkspurt. The silence period length is distributed exponentially with a mean of 650 ms (Sriram and Whitt, 1986).

Figure 2: Modeled LAN to external network system

Typical analyses using the simulation model are given in Figures 3 and 4. For these figures and the comparison with the analytic approximation, we used a G.729 codec. It has a bit rate of 8 kbps, a frame size of 10 ms, and a frame length of 10 bytes. With 20 PCs and 20 phones generating voice calls only, at least 10 DS0s are needed to give a mean packet delay of less than 30 ms (see Figure 3). Twelve DS0s are needed, however, to yield packet jitter less than 10 ms with 20 PCs and phones (see Figure 4). With 15 PCs and 15 phones generating voice calls only, the number of DS0s can be reduced to 8 with comparable QoS to the 20 PCs/20 phones case. Eight DS0s also resulted in a packet jitter of less than 10 ms for this case. The packet delay includes the compression time, queueing time, transmission time, and the decompression time. It is assumed to be half of the compression time. On the Ethernet, the sensing time of 0.0015 ms is included.

Figure 3: Mean packet delay in access network

Figure 4: Packet delay jitter in access network

Extend simulation software was used for the simulation model. Extend is a general-purpose, graphically oriented simulation software package. The limitations in the size of Extend models are not problematic for this VoIP access system. For example, Extend allows two billion blocks per simulation model, and our model with 20 phones and 20 PCs has about 1,100 blocks. However, the size of the simulation models and the experimental design of such models are limited by the software and computer speed. For example, on a 133-MHz PC with 16 megabytes of random-access memory with voice calls generated at 40 phones and 40 PCs, plus data packets generated at the PC, it takes approximately 10 hours to simulate one minute of operation of the VoIP access system. Thus, we are limited in the number of phones and PCs that we can simulate and in the run length and number of replications we can use. In fact, plots of the mean voice packet delay versus actual time for the 40 phone/40 PC example show that the simulation does not begin to reach the steady state until the simulation has been running for about 8 hours (see Figure 5). Thus, an analytic model of the system that closely approximates the simulated results could save a substantial amount of time and be invaluable in the analysis of such systems.

Figure 5: Initial transient behavior of mean voice packet delay

Obviously, housing the simulation model on a more powerful platform could reduce these runtimes; however, it does beg the question of whether a simple analytic congestion model for such systems can be developed. Analyses of integrated voice and data queues involve the use of in-depth mathematical queueing theoretical methods. It is not uncommon to use generating function methods (Gross and Harris, 1998) that also involve the solution of roots of equations and then only obtaining expected value results. Fischer (1980) is one such example, and there are many more (see our list of references). The restrictive nature of these approaches is further complicated by the fact that they require more restrictive assumptions regarding the voice and data packet arrival processes and their respective packet lengths.

Thus, one is forced to develop mathematical approximations if simplicity is desired. This area has been an active research area within queueing theory (see Fischer, 1998) and the references contained therein. In fact, Sriram and Whitt (1985) present a simple mathematical approximation model for studying the congestion voice and data packets experience in an integrated queueing environment. They use a two-moment characterization of the voice and data packet arrival processes and their lengths to more accurately model the desired congestion. Once the mean and variance of the arrival process has been determined, the Queueing Network Analyzer (see Whitt, 1983) can be used to develop simple congestion formulas for the mean packet delay as well as other measures of performance, such as jitter and the complete probability distribution of packet delay.

The VoIP access network is composed of two queueing systems: one on the LAN and the other on the line connecting the LAN to the "cloud." We use Sriram and Whitt to study the packet congestion on each of these queueing systems. To understand Sriram and Whitt, let us consider an arbitrary queueing system while presenting their results, and then discuss how to adapt their results to these two queueing systems.

For ease of reference, we use the same notation as in Sriram and Whitt. Let n₁ (n₂) be the number of independent and identically distributed voice (data) lines using the queueing system. The packet arrival rate for each voice (data) line is l₁ (l₂). The coefficient of variation (variance divided by the mean squared) of the voice (data) packet arrival is c₁² (c₂²). The mean and coefficient of variation of voice (data) packet service times are t₁ (t₂) and c_s1² (c_s2²). If t and c_s² are the mean and coefficient of variation of packet service times, then

t = (n₁ l₁ t₁ + n₂ l₂ t₂)/(n₁ l₁ + n₂ l₂)

and

(c_s² + 1) t² = (n₁ l₁ t₁² (c_s1² + 1) + n₂ l₂ t₂² (c_s2² + 1))/(n₁ l₁ + n₂ l₂).

The utilization or load of the system, r, is

r = (n₁ l₁ t₁ + n₂ l₂ t₂) = r₁ + r₂

where r₁ (r₂) is the voice (data) load.

Sriram and Whitt concentrate on characterizing the voice packet arrival process. First, they study the packet arrival process on a single voice line and then extend their results to the combined voice and data packet arrival process. Talkspurts and silent periods characterize a voice conversation. Let 1/a be the expected length of a talkspurt and 1/b be the expected length of a silent period. As mentioned above, Sriram and Whitt point out that tests have shown that 1/a = 352 ms and 1/b = 650 ms. Let T be the fixed interval of time in ms that packets are generated during a talkspurt. This time depends on the codec that is used. If the number of voice packets generated during a talkspurt is geometrically distributed, and the length of the silent period and talkspurts are exponentially distributed, the voice packet arrival stream for a single voice conversation is a renewal process, and

c₁² = (1 - p²)/((T b + 1 - p)²)

where p = (E(N) - 1)/E(N). The quantity E(N) is the expected number of packets generated in a voice talkspurt and is given by

E(N) = 1/(Ta).

Using Whitt (1983), the expected time a packet waits in the queue, E(Wq), is

E(Wq) = t r (c_a² + c_s²) g / (2 (1- r )),

where c_a² is the coefficient of variation of the combined voice and data arrival process and c_s² is given above. The quantity g in E(Wq) equals 1 if c_a² <= 1 and

g = exp{-(2(1 - r )(1- c_a²)²)/(3 r (c_a² + c_s²))}

when c_a² < 1.

After characterizing the voice packet arrival process for a single line, Sriram and Whitt concentrate on determining the overall (voice and data) packet arrival process and c_a² in the formula for E(Wq). Let

h = (1 - r )² (u - 1)

with

u = (n₁ l₁ + n₂ l₂)² / (n₁ l₁² + n₂ l₂²),

then

c_a² = (n₁ + n₂ - 1)/(n₁ + n₂)

if h >= 5 and n₁ + n₂ >= 10; and

c_a² = 1 + w (CAM² - 1)

if h <= 5. In this final expression we have w = 0 if h >= 5 and

w = (5 - h )/(5 + 10h)

when h < 5; and

CAM² = (n₁ l₁ c₁² + n₂ l₂ c₂² )/(n₁ l₁ + n₂ l₂).

At first glance there would appear to be a significant amount of calculations that have to be made in determining the packet performance; however, all calculations are straightforward and can be performed with a hand calculator. Thus, the methods presented in Sriram and Whitt are very simple to use. The interested reader is invited to read their paper for a more detailed discussion of their results and a comparison of their results with simulation results.

Sriram and Whitt give us a simple model that can be used to predict the packet performance on the connection from the LAN to the "cloud," because that queueing system is the same one that Sriram and Whitt studied. However, the packet performance on the LAN is a different story. (See Schwartz [1988] for an in-depth discussion of the modeling aspects of LANs.) We propose to use a simple modification to the method by Sriram and Whitt that is based on an observation presented in Schwartz (1988, p. 460). Complications of the queueing analysis of packet congestion on the LAN are the potential for packet collision and the requirement of transmitting the packet again.

Let a be the normalized average end-to-end propagation delay along the LAN; then following Schwartz, the maximum throughput of the LAN under CSMA/CD is

r _max = 1/(1 + 6.44 a).

Schwartz points out that as a gets smaller, the collision possibility gets smaller and the congestion on the LAN can be modeled as a G/G/1 queue. Sriram and Whitt's approximation deals directly with the packet congestion in a G/G/1 queue, but to use it on the LAN, we modify the LAN load, r _LAN, by r _max; that is,

r _LAN = r / r _max

where r is the load as calculated above using the LAN voice and data statistics and speed. In the tables below (Tables 2 and 3), we compare this modification with the results of our simulation.

We have conducted a comparison with our use of Sriram and Whitt's results and our simulator. The LAN is a 10BaseT CSMA/CD and the line from the LAN to the "cloud" is a T1. We used a G.729 codec, so that T = 10 ms, and we assumed that the LAN was 0.3 km in length. This resulted in a = 0.0015 with a propagation speed on the LAN of 5 ms. The packet size for both voice and data packets were 64 bytes on the LAN and 44 bytes on the T1, and we assumed that they were fixed-length packets. The G.729 code delay was 15 ms and was added to each of the packet delays as they accessed the "cloud." The PCs and the phones that are located off the IP gateway generate calls that must access the "cloud."

Table 2 compares the results of the analytic approximation and the simulation when there is only voice traffic on our VoIP access system. We are analyzing the access network when all the voice conversations are active, and the presented expected delay includes the transmission delay. As one can see, the analytics and the simulation are in very close agreement with respect to the expected packet delay on the LAN and on the T1.

In Table 3, we look at the case where there are both voice and data packets on the VoIP access network. The first comparison is for the case of 20 PCs and 20 phones with no data load; again, we see the same sort of agreement as in Figure 1. The second-column data traffic is added on the LAN and is also wishing to use the T1 line. As pointed out in Sriram and Whitt, their approximation seems to work best when the arrival rates of the voice and data packets are approximately equal. In Table 3, we assume these rates are equal and determine the number of data lines to match the data load. The arrival process for data was assumed to be Poisson. For this example, the approximation again does very well. The data load of 0.15 on the T1 equals the total packet arrival rate times 44 bytes times 8 divided by 1,544,000. The resulting packet arrival rate is then used as the data load on the Ethernet, but the resulting load is less because of the speed of the LAN.

We have developed an analytic approximation that can be used to study the congestion of a VoIP access arrangement. Some initial comparisons with the simulation for the expected delay have been conducted. Further comparisons are required for mean value results as well for other measures of performance, such as jitter and the probability of having to wait for service. Whitt (1983) gives some simple expressions for these measures of performance as well as the complete probability distribution of packet delay. What is not available at this time is the packet loss probability, which will be considered in our future work. Initial analyses using both models have shown that there is sensitivity to QoS when VoIP access systems are sized. Our initial comparison demonstrated that the analytic approximations are quite good, but we have not considered a wider range of potential scenarios. These comparisons need to be explored.

4. Future plans

A VoIP testbed and a performance modeling capability for a representative VoIP access system have been developed. The testbed initially focuses on the intranet experiment. After the intranet performance characteristics are understood, the Internet testing will follow. One of our first project objectives is to calibrate the voice packet delay as predicted via the modeling capability and as seen in the testbed. Various statistical assumptions were required for the modeling capability, like the length of talkspurts and silent periods. Based on these assumptions, modeling capability presents the packet delay, mean, and jitter. If the technology refreshment affects those assumptions, then our modeling capability will be modified. Our initial observation indicates that the VoIP performance is governed by the network switching equipment, gateway, and user terminal. The performance impact due to the link resource sharing is not as significant as expected.

The FTS2001 network is the target environment for both performance and cost analysis. The network system consists of various local-access (e.g., Metropolitan Area Acquisition, Washington Interagency Telecommunications System, and wireless) and long-distance arrangements (e.g., International Direct Distance Dialing, MCI WorldCom, and Sprint). The feasibility of using VoIP as an access alternative is one of our interests in this project. Besides the regulatory concerns, the determining technical factors of a viable option are performance and cost. The predicted cost per minute for domestic voice services under FTS2001 is $0.02 in the later years of the contract. However, the cost for international calls is higher. To examine the possibility of cost savings, our future studies will investigate the cost-effectiveness of VoIP under FTS2001 for the overseas environment.

We also will be establishing joint testing arrangements with other interested government agencies. Some initial contacts have been made and are being explored. The purpose of these joint experiments will be to better understand VoIP performance in a government environment.

References

1. El-Sherbini, A., M.S. El-Sherif, T.M. Kamel, and A.A.W. Fayez, "A Performance Evaluation of the Integration of Voice and Data in a TCP/IP Local Environment," 36th Midwest Symposium -- Circuits and Systems (IEEE), Detroit, 1993, pp. 1332-1335.
2. Fendick, K.W., V.R. Saksena, and W. Whitt, "Dependence in Packet Queues," IEEE Trans. Comm., 37 (11), 1989, pp. 1173-1183.
3. Fischer, M.J., "Delay Analysis of TASI with Random Fluctuations in the Number of Voice Calls," IEEE Trans. Comm., 28 (11), 1980, pp. 1883-1888.
4. Fischer, M.J. and T.C. Harris, "A Model for Evaluating the Performance of an Integrated Circuit and Packet-Switched Multiplex Structure," IEEE Trans. Comm., 24 (2), 1976, pp. 195-202.
5. Fischer, M.J. and C.M. Harris, "Approximating by Enhanced Interpolation in Queueing Analyses," Computers Ops Res., Vol 25, No. 9, 1998, pp. 707-717.
6. Gross, D. and C.M. Harris, Fundamentals of Queueing Theory, 1998, New York: Wiley Publishers.
7. Minoli, D. and E. Minoli, Delivering Voice over IP Networks, 1998, New York: Wiley Publishers.
8. Ott, T.J. and J.G. Shanthikumar, "On a Buffer Problem for Packetized Voice with an N-periodic Strongly Interchangeable Input Process," J. Applied Probability, 28 (3), 1991, pp. 630-646.
9. Ott, T.J. and J.G. Shanthikumar, "Structural Properties and Stochastic Bounds for a Buffer Problem in Packetized Voice Transmission," Queueing Systems Theory and Applications, 8 (3), 1991, pp. 225-236.
10. Rai, S. and Y.C. Oh, "Analyzing Packetized Voice and Video Traffic in an ATM Multiplexer," IEEE International Performance, Computing, and Communications Conference, February 1998, pp. 367-372.
11. Schwartz, M., Telecommunication Networks, 1998, Addison-Wesley Publishing Company.
12. Sriram, K. and W. Whitt, "Characterizing Superposition Arrival Processes and the Performance of Multiplexers for Voice and Data," Globecom '85: IEEE Global Telecommunications Conference, New Orleans, 1985, pp. 778-784.
13. Sriram, K. and W. Whitt, "Characterizing Superposition Arrival Processes in Packet Multiplexers for Voice and Data," IEEE Journal on Selected Areas in Communications, SAC-4 (6), 1986, pp. 833-846.
14. Suda, T. and T.T. Bradley, "Packetized Voice/Data Integrated Transmission on a Token Passing Ring Local Area Network," IEEE Trans. Comm., 37, March 1989, pp. 238-244.
15. Suda, T., H. Miyahara, and T. Hasegawa, "Performance Evaluation of a Packetized Voice System -- Simulation Study," IEEE Trans. Comm., 32, January 1984, pp. 97-102.
16. Weinstein, C.J., M.L. Malpass, and M.J. Fischer, "Data Traffic Performance of an Integrated Circuit- and Packet-Switched Multiplex Structure," IEEE Trans. Comm., 28 (6), 1980, pp. 873-877.
17. Whitt, W., The Queueing Network Analyzer, The Bell System Technical Journal, Vol. 62, No. 9, 1983, pp. 2770-2815.