Plugging Singapore to Internet-2 Grid: the SingAREN GigaPOP

Lek H. Ngoh

The Singapore Advanced Research and Education Network (SingAREN)

C/o Kent Ridge Digital Labs

21, Heng Mui Keng Terrace,

Singapore 111396

Fax: (+65) 774-4990

Tel: (+65) 874-6597

Contact e-mail: lhn@singaren.net.sg

Abstract

This paper describes the setting up of an Internet-2 Point-of-Presence (GigaPOP) for the Research and Education (R&E) community in Singapore, and to link Singapore to the worldwide Internet-2 network grid. The proposed GigaPOP is to be operated as part of the Singapore Advanced Research and Education Network (SingAREN) network infrastructure. The paper first outlines the current inefficiencies in providing Internet-2 services to the R&E community in Singapore, and proposes the GigaPOP as a model and possible solution. The GigaPOP is also expected to result in cost savings on many fronts and to all parties involved. These savings are derived from integrating access and maintenance of broadband services, as economies-of-scale on access to commodity Internet. In this paper, the roles of the various network and service providers within the GigaPOP are clearly spelled out. Some of the key implementation details of the SingAREN GigaPOP are also described.

Introduction

Singapore has some of the most advanced computer data network infrastructures in the world. It has five large-scale commercial Internet Service Providers (ISPs), and more to come with the recent liberalisation of the ISP market. A nation-wide broadband infrastructure called Singapore ONE became available commercially in 1998. With it companies, institutions and households can now be connected readily to these information highways. To look beyond the current state-of-the art broadband technologies, the Singapore Advanced Research and Educational Network (SingAREN) was set up in 1997 to promote R&D in these areas. Currently SingAREN operates dedicated high-speed links to Japan, Korea and USA. Using the services provide by Science, Technology and Research Transit Access Point (STAR TAP), the USA link is also interconnecting networks from Canada, Europe, Middle East and Russia. With these networks, Singapore has indeed plugged firmly onto the information highways both locally and internationally, and stand to reap the many benefits and spin-offs these advanced networks can offer. Figure 1 illustrates the state of Singapore broadband infrastructures.

 

Figure 1. Singapore Broadband Network Infrastructure Showing Singapore ONE and SingAREN International Links

Multiple Local Networks

The existence of multiple broadband networks within Singapore, however, has somewhat complicated the roles each of them play, and inefficient network configuration. The end-result can be confusing to potential users and subscribers of these networks. Currently the situation is as follows:

  1. Singapore ONE provides nation-wide commercial broadband access. A company called 1-NET is responsible for the core backbone network while Singapore Telecom (SingTel) and Singapore Cable Vision (SCV) provide the access networks using ADSL and cable modems, respectively.
  2. The ISPs provide commercial Internet services in Singapore. They do so through dedicated leased lines or other available access technologies, but also tunnel their services via Singapore ONE.
  3. SingAREN is a government-funded project, with its own broadband network links. It provides next generation Internet access exclusively to the R&D community in Singapore and is linked with counterpart research networks internationally.

 

A Sub-Optimal Situation

From the network operation standpoint however, these roles are not mutually exclusive; resulting in inefficiencies in the use of network resources. The situation is particularly evident in the research and education establishments, where all three networks are present. Figure 1 illustrates the current network situation, which gives rise to the following:

  1. Too many physical links (i.e. one per provider) to one organization and available raw bandwidth is not fully used leading to waste.
  2. Operationally, all these network services are not integrated within an organization. Instead each carries a specific "class" of traffic. This has resulted in low-usage on some physical links and network congestion on others.
  3. Users have difficulty deciding which link to use to carry out certain tasks such as a videoconferencing trial.
  4. Fine-tuning on the required bandwidth for each of these services is not possible. Each service is purchased and used with its own terms and conditions.

Unfortunately, an immediate negative impact as a result of the above limitations is that potential R&D users are discouraged from using one or more of these networks. This is because despite its high bandwidth and advanced services it is perceived to be restrictive (ie. only for R&D), high-cost, difficult to configure, and has limited coverage when compared to the commodity Internet. The real incentives of promoting the exploitation of next-generation Internet (or Internet-2) research in Singapore are therefore over-shadowed by these perceptions.

 

 

Figure 2. Current R&E Network Connectivity

Proposed Solution: a GigaPOP Model

To overcome the problems highlighted above, we propose that a GigaPOP (gigabits Point-of-Presence) be set up for the R&E community in Singapore. The basic concept involves the centralization of vital network services such as Internet access, Singapore ONE, and next generation Internet with intelligent policy-based routing, as well as bandwidth partitioning functions in one place. The most effective means are then provided for these services to be accessed by the respective connecting sites. A recent study carried out on research networks in the US, Canada, Asia and Europe has provided strong support for setting up such a GigaPOP operation. Many research networks are operated by a small group of technically competent engineers serving the networking needs of their R&E community. A number of these GigaPOP operations are not-for-profit i.e. they are either funded by government agencies, or self-supported through fees collected from membership subscription after a number of years of operation. Examples of these include the NC-REN in North Carolina, USA[3]; OCRInet and RISQ in Canada[5]; and DFN network in Germany[4]. Some of these GigaPOP operators also function as ISPs, and even collaborate with other infrastructure providers (e.g. utility gas company) to gain the "right-of-way" of providing high-speed links. In short, the various broadband infrastructure operators in Singapore need to examine the GigaPOP model closely and adapt it for local use.

Under the GigaPOP arrangement, instead each network provider deals directly with its users (e.g the R&E community), they are to deal through the GigaPOP operator. Given that one of SingAREN’s major goals is to experiment and lead Singapore’s transition into the next generation Internet. Together with its objective of serving the R&E community, SingAREN is therefore the best candidate to set up and operate the GigaPOP. In the rest of this paper, the various aspects of the proposal GigaPoP are addressed.

Roles of ISPs and Network Operators

Figure 3 illustrates the proposed SingAREN GigaPOP network. With this set up, the relationships between the various broadband network infrastructures will be much cleaner. More importantly, the conflict of roles and functions described earlier can be avoided completely. This will lead to cost savings and better use of available resources. The new arrangement is now as follows:

  1. 1-NET, SingTel and SCV will provide the commercial broadband "fat pipes" in Singapore, now used primarily to access Singapore ONE services. The same physical links can be used to provide access to SingAREN services, as well as commercial Internet traffic provided by the ISPs.
  2. SingAREN will provide GigaPOP services to the R&E community on a not-for-profit basis, and support network and application R&D locally and internationally. These services are offered via either the S-ONE links or SingAREN’s own dark fibre links. The S-ONE link is a pre-configured, stable broadband fat pipe suitable for application-level research and sustained testing. The SingAREN dark fibre link, on the other hand, allows for a wide range of flexible configurations and protocols to be tried and tested. For this reason it does not offer the same level of reliability as the S-ONE link, but its experimental nature makes it more suited for core network research.
  3. Currently users obtain commercial Internet services directly from the ISPs. Under this proposal, they can obtain the service via the GigaPOP. This means that the ISP service will first come to the GigaPOP before going to the connecting sites. The GigaPOP acts as a service broker for R&E sites to obtain their commercial Internet services from the ISPs. With economies of scale, as well as the effect of traffic aggregation and statistical multiplexing, the GigaPOP can expect to offer a more cost-effective commercial Internet service.
  4. The proposed GigaPOP will therefore be the aggregation point for all the three network services; namely the commercial Internet, Singapore ONE and SingAREN.

Implementation Issues

The above model, however, still leaves a number of implementation issues outstanding. This is because some of these issues are related to government regulation, as well as commercial consideration. Nevertheless, following issues have to be resolved in implementing the GigaPOP.

  1. Provider of the GigaPOP international links: There are a number of possible ways. The first is for an ISP to do so. However, an ISP may not be familiar with Internet-2 community. The second approach is for SingAREN to lease international lines from SingTel and operate like an ISP, handling both the Internet-2 and regular Internet traffic, but only for a restricted group of R&E organisations. The third approach is for SingAREN to carry the Internet-2 traffic and one or more ISPs to carry the regular Internet traffic. In terms of flexibility in operations, the third approach is preferred, however it may not most cost effective.
  2. Partition of bandwidth between R&D and commercial traffic: Given that the Internet-2 traffic tends to be "bursty" compared with the regular Internet traffic, dynamic adjustment of the bandwidth between Internet-2 and regular Internet use will enable more optimum allocation according to needs. Technically, this is not difficult but operationally the adjustment of bandwidth involves a number of parties – the ISP, the network provider SingTel and its partner in the foreign country and the ISP there.
  3. Effect on Singapore ONE in the proposed GigaPOP: Operators of Singapore ONE would be partners in the GigaPOP setup. This is because most SingAREN connections use Singapore ONE links. In fact Singapore ONE would enjoy greater demand for its broadband links, and the co-location of the GigaPOP with the Singapore ONE services will allow easy inter-working.

Last but not least, the proposed GigaPOP will operate its network in both the "production" and "R&D" modes (aka the multi-modal network[11]). This means a suitable network configuration should be determined so that both production and R&D related activities can be supported effectively. Figure 3 shows the proposed SingAREN setup with two separate nodes, namely the "R&D POP" and "production POP". As the names imply, the research node (to be at SingAREN’s KRDL site) is where the R&D links terminate. This supports broadband R&D activities including demonstrating, testing and developing of new protocols by the connecting sites. Research prototypes, access techniques (e.g. ADSL, SVC services) to be tried and tested before they are applied to the production POP will also be conducted here. Given its R&D role, this node will be able to access the same set of broadband services being provided by the production POP. However this service provision should only be for R&D purposes and are non-permanent. This part of the SingAREN network will also not provide the same level of reliability as the production POP. The production POP (to be at another SingAREN building site) on the other hand, will provide the aggregated GigaPOP functions as described in this paper. When necessary, the production GigaPOP will also be used to support application level research and project demonstrations and vice-verse. However, this should be treated with care to ensure that the reliability of the production POP is not compromised. All links terminating at this production GigaPOP are served through the Singapore ONE infrastructure.

 

 

 

Figure 3. Proposed Implementation of SingAREN GigaPOP

The production POP on the other hand will be the entry point into the various SingAREN & ISP services. Other international traffic services such as web caching, MBone , 6Bone and QBone will also make their entry and exit via this node. This is also where traffic arbitration and route server functions are provided. In order to maximize flexibility, the R&D and production POPs are linked physically via a R&D dark fiber link as shown.

 

Implementation Details

Figure 4 shows the physical set up of the proposed SingAREN GigaPOP. Two physically separated sites have been configured to be the research POP and production POP respectively. The diagram also shows the way the production node is linked with the S-ONE infrastructure, and how the various R&D sites are connected to the research POP.

 

 

Figure 4. SingAREN GigaPOP Setup – Physical Connections

Advanced Services

Once the basic infrastructure is in placed, various advanced services can be introduced to the GigaPOP. Some major service components have been identified for both the short and long-term needs of the proposed GigaPOP. Examples are the route arbiter architecture and virtual private network with premium (i.e. QoS guaranteed) connection service that works with for both wired and wireless hosts. In the rest of this paper, initial experience in providing these services are described. The actual details of these projects are, however, beyond the scope of this paper.

 

IP Service Aggregation Model

One important network service to be provided to the connecting sites is IP services. The IP service here should include both the current IPv4 and IPv6, and they should be provided based on best-effort and guaranteed Quality of Service (QoS) models. Some of these can be implemented quickly with a combination of IP and ATM technologies whilst others require further experiment and research. All these issues are expected to be tackled at the production and research POPs.

A suitable model of aggregation for all the services is to be provided. However, the exact mechanism will involve details on how the network (e.g. how many ATM VCs?) should be set-up and require collaboration with service providers such as 1-NET. It is also important to be able to control individual bandwidth on each "class" of traffic (e.g. ISP, SingAREN & S-ONE) to each organization and adjust these allocations flexibly. Needless to say, the final package offered must have a cost model that is economically attractive to the users. Figure 5 shows one of the possible and easiest implementations that involves the setting up multiple ATM VCs to each subscribing site. The purpose of having multiple VCs is to rate limit each class of traffic to and from each connecting site. This approach is simple to achieve, however, it has limitations in terms of the number of VCs that need to be managed and inflexibility in bandwidth subscription.

Figure 5. SingAREN GigaPOP Setup – IP Route Peering

 

 

Figure 6. Proposed SingAREN VPN Service with QoS Guarantees

Virtual Private Networks (VPNs) with QoS Guaranteed

Looking ahead, one of the major functions of the GigaPOP is the provision of virtual private networks (VPNs) to connecting sites. These VPNs are required on a temporary (e.g. demos) or permanent basis (e.g. virtual classroom between the two universities, or a "virtual" hospital network). Whilst the required VPNs can be set up relatively easily (though not necessary quickly) using the ATM virtual channels (VCs) and virtual path (VPs), scalability is a major issue. This is especially so when dedicated PVCs are used to reserve bandwidth on the international links for single experiment or session. A more tenable approach is to aggregate individual traffic flows using differentiated service (diffserv) model[8] at the router-level, and carry the aggregated traffic over large PVCs created for different "classes" of traffic. This will result traffic being further aggregated at the ATM level. Furthermore, multi-protocol label switching (MPLS[9]) techniques will also allow classification of flows into ATM VC pipes to enhance traffic engineering. Figure 6 illustrates the proposed scheme. The various approaches are currently being experimented. Further details of this work is elaborated elsewhere[10].

Looking ahead, these VPNs will also require the support for multicast and IPv6, as well as inter-working with wireless terminals and even satellite links. Currently a number of suitable solution(s) are being investigated within SingAREN to enable these next generation broadband services be made available via the GigaPOP to the R&E community in Singapore.

 

Session-level support for Audio, Video Applications

With its QoS guaranteed transport service described above, the SingAREN GigaPOP will be used increasingly to support time sensitive audio and video applications, examples of these applications are voice-over-IP and real-time video-based traffic information server. In order to support one or more of these applications effectively, session-level protocols in the form of H.323[12] and SIP[13] are to be provided to realize component functions such as media gateway, session negotiation and service registration. This service is particularly important given the close relationship between session level negotiation and network transport level requirements for these applications.

Route Arbiter Architecture

Another key components of the GigaPOP is the route arbiter (RA) architecture. This is mainly because the proposed GigaPOP will carry traffic which are destined for different links with different commercial considerations (e.g. the Internet-2 link is being sponsored by the government agencies). The concept of RA comes from the need to simplify the exchange of routing information at Internet exchanges such as a network attachment point (NAP). A RA will help to process the topology, connectivity, and routing policy (e.g. who can talk to whom) information to create and distribute a stable routing table. Another important reason for having a RA at the GigaPOP is to ensure that all routing decisions can be accessed and viewed by the respective network providers centrally. This is so that the GigaPOP routing policy can be transparent to all authorized parties who wish to examine it.

Similar to the reasons given in a related NSF-funded project carried out at the Information Sciences Institute, University of Southern California, USA and IBM TJ Watson Research Centre, the RA will be seen as a distinct logical, institutional, as well as operational component to ensure that:

  1. System routing will be analyzable with respect to consistency, connectivity, and configured route characteristics, in the presence of autonomously determined connectivity, configuration, and policy.
  2. Policies of individual networks will not inadvertently or arbitrarily impact connectivity.
  3. Routing will not be used as a means of discriminating or giving preferential treatment to particular NSP(s) – different NSPs serving the GigaPOP will be given fair and equitable treatment.

The RA architecture consists of the following components:

  1. Routing Registry Data Base (RRDB): a logical centralized database of routing information from SingAREN.
  2. Router Server (RS): to control the exchange of routing information among attached networks.

 

 

Conclusions

The introduction of broadband networks to Singapore has happened at a very rapid pace in the last few years. With the availability of Singapore ONE and SingAREN and the accelerated pace of internet penetration, Singapore is now one of the most wired up nations in the world, well positioned for the coming knowledge based economy.

As these broadband networks are deployed, it is inevitable some confusion and optimisation of resources would arise. This paper has identified such areas of confusion and sub-optimisation and proposed how the SingAREN network can be re-positioned based on a GigaPOP model. The overall objective is to maximise the potential benefits of broadband networking to Singapore, specifically to the research and education community which has the greatest need to access and exploit the various advanced Internet-2 services.

 

Acknowledgements

The author would like to acknowledge the generous funding support for SingAREN from the National Science and Technology Board (NSTB) and the Infocomm Development Authority (IDA) of Singapore.

References

  1. L.H. Ngoh et. al, "SingAREN: The Singapore Advanced Research and Education Network", IEEE Communications Magazine, Nov. 1998, Vol. 36 No.11, pp. 74-83. See also: http://www.singaren.net.sg.
  2. D. Estrin, J. Postel, Y. Rekhter, Routing Arbiter Archecture, ConneXions (8), August 1994, pp. 2-7.
  3. MC-REN, MCNC, 3021 Cronwallis Road, Research Triangle Park, NC 27709, USA. Http://wwwncren.net/Internet/sites.html.
  4. Http:///www.dfn.de/.
  5. RISQ RAN, http://www.canarie.ca/eng/networks/Canetii/gigapops/RISQ/.
  6. CANARIE CA*Net 3, http://www.canarie.ca/eng/networks/Canetii/backbone/map.html
  7. STAR TAP, http://www.startap.net.
  8. RFCs 2472 and 2575, IETF Documents on Differentiated Services
  9. IETF Draft Document "A Framework for Multiprotocol Label Switching (MPLS)", draft-ietf-mpls-framework-05.txt, Sept. 1999
  10. L.H. Ngoh, "QoS Provision and Traffic Engineering the SingAREN Network", working paper available from the author.
  11. R. J. Aiken et. al., "Architecture of the Multi-model Organisation Research and Production Heterogeneous Network (MORPHnet)", Argonne National Laboratory, Technical Report ANL-97/1, Jan. 1997. Available at: http://www.anl.gov/ETC/Public/research/morphnet.html.
  12. International Telecommunication Union, "Packet based multimedia communication systems" Recommendation H.323, Telecommunication Standardization Sector of ITU, Geneva, Switzerland, Feb. 1998.
  13. M. Handley et. al., "SIP: session initiation protocol", IETF RFC (proposed standard) 2327. Apr. 1998.