Wireless Networks: A Cost-Effective Way to the Internet

Danton NUNES <danton@inexo.com.br>
Ethy Henriques BRITO <ethy@inexo.com.br>
InterNexo
Brazil

Abstract

Access to the Internet in many places is either difficult or expensive or a perverse blend of both, due to poor public communication infrastructure and/or the high cost of existing services. This paper discusses an alternative to leased or switched lines based on wireless technology.

Two case studies are discussed: a college campus network and a small wireless backbone that brings Internet access to business users in five neighboring towns on the Paraiba's Valley, in the state of São Paulo, Brazil.

The University of Taubaté (UNITAU) is not concentrated on a delimited campus; rather, its buildings are spread over the town. Connecting all buildings with leased lines exceeded the acceptable budget and yielded unacceptable performance, so the school decided to go wireless. UNITAU's wireless network operates at 2.4 GHz direct sequence spread spectrum. The network topology is a star in a point-to-multipoint configuration. The central station is a router equipped with an omnidirection active antenna, and the satellite stations are equipped with high-gain directional antennas. The central station was installed in a clock tower that is visible from almost anywhere in the town, so straight-line visibility is not a hurdle. The routers on all stations are old 386 PCs rescued from the cemetery of obsolete machines. They are useless as desktop personal computers because they can't efficiently run the new graphic user interfaces with plenty of whistles and bells, but they are still useful as routers. UNITAU's wireless network is extensible in the sense that new satellite stations can be added easily. If the need arises, the topology can evolve to a tree, adding a new central station.

UNITAU's network design was based on InterNexo's own wireless network. InterNexo is an Internet service provider, and the wireless network is its main medium to offer Internet access to business users. At the time of this writing its configuration comprises three nodes (central points with omnidirectional antennas) plus eight satellite client stations. Two nodes are located in São José dos Campos, and the third one is in Taubaté, 39 km away. Communication between São José and Taubaté is carried through a dedicated 2.4 GHz link with active high-gain directional antennas. The system operates at 915 MHz in São José and 2.4 GHz in Taubaté. The original plan was to operate at 915 MHz there too, but strong interference caused by a radiocall (pager) operator made the 915 MHz ISM band unusable. Radiocall operates at 200 W or more, while ISM services are limited by Brazilian regulations to 1 W. The routers are PCs running Linux. They accumulate a number of functions besides routing, such as serving Web pages, time reference, and diagnostic tools. This system began operating in November 1996. The only outages were caused by the already-mentioned interference and a malfunctioning amplifier. The uptime is comparable to the much more expensive leased lines.

One year of experience running wireless networks at ISM bands shows that this technology is a good choice wherever cost, availability, or reliability of ground-based alternatives is problematic. Interference between different spread-spectrum systems is far from being critical; however, interference caused by other sources is, especially in the 915 MHz band, where powerful neighbors dump their spectral garbage.

Contents

• Wireless Internet access
• UNITAU's wireless network
• Notes
  • Routers versus bridges: a brief discussion
  • Interference: cope with it or die!
  • Amplifiers and cables as trouble sources
• Conclusions
• References

Wireless Internet access

Our company is a small Internet service provider established in São José dos Campos, in the valley of the river Paraíba, one of the most industrialized areas of the state of São Paulo, Brazil. Our goal is to provide high-quality Internet access services to business users in the region. We started with leased lines but at that time they were very expensive and performed poorly. They are a little less expensive and work a little better nowadays but price and performance are still burdens. In the beginning of 1996 we were looking for alternatives to the leased lines. Packet switching networks based either on X.25 or frame relay were available but even more expensive than leased lines, considering the expected volume of traffic. ISDN was not offered by the local telephone company. The wireless option was promising so we decided to spend some time and money on it.

We were really looking for

• Low-cost, reliable, off-the-shelf equipment and software;
• Frequencies and power we could operate without license; and
• A range large enough to encompass most of the town from our premises.

The answer to the first wish was Lucent's (formerly NCR's) WaveLan family of products on the hardware side and Linux, a hi-tech Unix clone, distributed freely under GPL (GNU Public License).

The second wish was fulfilled by the Brazilian Ministry of Communications, which issued a regulation on the use of the former ISM (Industrial, Scientific, and Medical) bands, waiving the need of license provided that only approved equipment was used (WaveLan is approved) and strict operational limits on frequency, modulation, power, and field intensity were observed.

We eventually learned about Brazilian manufacturers of antennas and ancillary equipment that allowed us not only to cover most of the town but also to establish a 39Km link at very low power. That was the third wish.

After some five minutes of very deep analysis we realized that the best topology for the wireless access provider is a star with the customers at the tips and the gateway to the Internet in the center. In order to reach all actual and potential customers, the central station should be equipped with an omnidirectional antenna. An amplifier was added to compensate for its low gain. The standard customer equipment included a high gain Yagi antenna, with or without amplifier according to distance and link conditions.

We choose to operate in the low ISM band (915Mhz), for, at that time, UHF equipment was cheaper and maximum legal power was higher than for 2.4GHz. This decision proved later to be a mistake. The low band is very sensitive to interference from ill-behaved equipment operating in neighbor bands. On the other hand, the prices of SHF equipment fell down practically to the same level of UHF.

São José dos Campos viewed from Central Station One.

The next little problem to be solved was the physical installation of Central Station One. Our office is on the sixth floor and the antenna is on the roof, above the eighteenth floor. There was no way to install a low loss coaxial cable from our office to the roof due to its diameter and mechanical characteristics; the building was just not designed for this. We decided then to put the computer there on the roof and send it power and LAN access through smaller and much more flexible cables than the coax waveguide. A special box with passive thermal control was built to house the equipment. The "pigeon loft," as we call it, was attached to the wall that receives the least direct sunlight all year round. In November 1996 the whole system was ready for beta testing.

The pigeon loft.

Tests were conducted with a link to our most faraway customer, a BBS and dial-up access provider. The results were pretty good; we finally got LAN class access almost 6Km from the hub. The results were received enthusiastically by our users and in a few months many converted to the wireless system. We managed to keep the prices of the new service below the price of leased lines, so it was attractive from both price and performance viewpoints.

First São José, then, the world! Thanks to the success of Central Station One and under the pressure of some customers, we started plans to have another point of presence in Taubaté, an important economic and cultural center of the Valley, some 40 Km away. The idea was to use the same technology to establish a link between Central Station One and the yet-to-be Central Station Two. The 915MHz band showed not to be feasible, for the antennas would need huge (and expensive) reflectors, so we moved to 2.4GHz. After some back-of-the-envelope calculations, we found out that something better than 55dBi plus amplifiers at both ends would be enough. Station One was endowed with a new 2.4GHz WaveLan board, an amplifier, and an antenna of 24dBi. Station Two was installed on an FM radio tower with a big 34dBi drum antenna. After some trouble with the alignment of the drum antenna, we managed to put the link into operation. The distance between both ends as measured by a GPS receiver is 39Km. Central Station Two also had an omnidirectional 915MHz antenna to convey the signal to the customers in Taubaté. Soon we would have to change it to 2.4GHz due to interference from a pager repeater (read the interference drama below). The first ping to Taubaté was issued at 4:00 PM, 19 May 1997. A few weeks later, regular operation began.

24dBi antenna with amplifier

Another important town in the region is Jacareí, some 15Km from São José. We could not reach it because a building in São José downtown eclipses it completely. As artillery was out of the question, we looked for a place to install another station in São José with a view to Jacareí, our office building, and to the few areas in São José we could not cover from Station One. So Station Three was born. It was equipped with an omnidirectional antenna (as always), plus a Yagi to enhance the gain in the direction of Jacareí. It entered on duty in December 1997.

Our wireless backbone now extends from Jacareí to Taubaté with three points of presence. Within the range and line of sight of any of the central stations and with favorable conditions to install antennas, we are able to put any network on the Internet in a matter of hours.

UNITAU's wireless network

Taubaté is a midsized town in the heart of the Valley of Paraíba, about 40Km from our offices in São José dos Campos and some 140Km from the big city of São Paulo. It hosts a complex of high education, research, and community service facilities, the University of Taubaté, best known by the acronym UNITAU. UNITAU is not concentrated on a delimited campus; its buildings are spread over the town instead. However good this may be, for it allows best integration of the university and the community, communication between the many university buildings becomes a problem. Connecting all the buildings with leased lines was far beyond acceptable budget and below acceptable performance. Fiber optics was out of the question in spite of its relatively low cost and remarkable performance. The telephone company does not allow passage of foreign cables in its galleries and digging new galleries would be very expensive, require special permits from City administration, and cause a lot of trouble to the citizens.

The solution was to go wireless. We presented a project based on our own wireless network and we won the bid. The initial goal was to connect the local area networks at six buildings. Due to physical constraints there was no way to make an antenna at one of the buildings see the other five, so we decided in favor of a star configuration. A seventh point was added to work as a central relay station. This station was installed in a high clock tower that is visible from almost everywhere in the town, including, of course, our six buildings. The radios selected were Lucent's WaveLan in the 2.4GHz band. We decided against the 915MHz band due to a previous experience with interference caused by a pager repeater on our own system (read the story below in this paper).

Omnidirectional boosted antenna

The routers on all stations are old 386 PCs rescued from the cemetery of obsolete machines. They are useless as desktop personal computers because they can't efficiently run the new graphic user interfaces with plenty of whistles and bells, but they are still useful as routers. Linux was installed and configured on a master disk. The master disk was copied seven times so at the end we had a disk for each station plus a backup unit. All disks are identical except for one file containing just the system name. The routers were assembled and tested at the laboratory of UNITAU's Information Science Department.

The central station received an omnidirectional antenna boosted by a HyperLink amplifier. Each satellite station received a high gain (24dBi) parabolic antenna and no amplifier at all. This antenna could be a little oversized for some of the points but it was adopted in all points for the sake of uniformity. All stations were endowed with uninterruptible power supplies (UPS) for two reasons: the UPS works as a power stabilizer and makes the system still usable during power outages.

At the time of this writing, UNITAU's wireless network has been operating painlessly for six months. Its star topology may not be very efficient in the sense that each data package is transmitted twice, from a satellite station to central station and back to (some other) satellite station, but it is not strongly constrained by city landscape and it is extensible by adding new satellite stations. If need arises, a new central station in another point can be also added to the system. The use of old computers, off-the-shelf components, and free software (Linux) made the whole project fit the low budget of the university.

Notes

Routers versus bridges: a brief discussion

The key element of the wireless networks described here is a cluster of radio stations with one single central station endowed with an omnidirectional antenna and a number of satellite stations with high gain directional antennas oriented towards the central station. At each station there is a PC with a radio interface that behaves pretty much like a standard ethernet card. The cluster, however, does not work like a standard ethernet LAN because each satellite station can see only the central station unless, against all odds, two opposite satellites and the central stations lie on an approximately straight line. On a well-configured ethernet LAN, each station can see each other.

If a data package must go from a satellite station A to another satellite station B, it must be relayed through the central station. There are at least two ways to implement the relay mechanism: the stations may be configured to work like bridges or IP routers. Linux running on the PC supports both. Which is better depends mostly on the application of the wireless network. If it is to merge several LANs into a big virtual LAN covering different protocol families, bridging is certainly the right choice. On the other hand, if it is an IP-only network and the connecting LANs must keep a certain degree of independence, IP routing performs best, for the routers may support packet filtering, IP translation, and other firewall features. In both cases studied here, IP routing was the relay method of choice. The wireless network of the University of Taubaté is IP-only and connects a number of different departments, each with its own access and security policies. The wireless Internet access provider has different commercial enterprises at each satellite station, so the differences in access and security policies are even more dramatic.

When writing down the routing tables of the satellite stations, one must take into account that while the radio interface looks like multiple access it actually works as if it were point-to-point. This peculiar behavior also makes dynamic route updating (e.g., by RIP or OSPF) something of a nightmare. If the point-to-point character of the radio interface is not properly accounted for, the satellite station will not be able to talk to other satellite stations although other hosts on its LAN will.

Interference: cope with it or die!

When we decided to install a point of presence on our wireless Internet access service in Taubaté, we designed the central station with two radios: one at 915 MHz and an omnidirectional antenna to feed our customers and another at 2.4.Ghz and a very high gain dish antenna to link to our main office in São José dos Campos 39 km away. We choose 2.46hz for the long-distance link because the size and cost of the antennas for 915mhz were prohibitive. The link to the customers was chosen to be at 915 MHz due to our previous successful experience in São José.

Three months after the Taubaté station went on duty, the nightmare began. From time to time, randomly, the 915 MHz radio went deaf for 20 to 30 seconds. When it was back to sanity, it reported good link conditions to all customers, very low background noise, and an astoundingly high count of rejected packets due to unrecognized network IDs. The malfunction happened more often in daylight, especially during business hours, and almost vanished late in the evening.

Using a spectrum analyzer we found out that there was a strong signal at 921MHz in the air during the deafness crisis of our radio. Such signal was inside the spread spectrum band, from 902 to 928 MHz. This could explain the high count of rejected packets. Finally, after some time roaming the neighborhood carrying weird equipment, like Ghostbusters, we managed to single out the source of interference: a repeater of pager operator, licensed to work at 931MHz, 150 Watts, against our mere 1 Watt. The 921 MHz signal, was -- still is, quite likely -- due to a parasite 10 MHz amplitude modulation on their FM carrier. This kind of spectral contamination is prohibited by Brazilian telecommunication regulations. To make things worse, the repeater antenna was just some 30 meters from ours. (The place is indeed overcrowded with antennas due to its superb view of the whole town.)

After unsuccessful negotiations with the owner of the radio call repeater and a never-answered e-mail to the liaison office of the Ministry of Communications in São Paulo, we reckoned that legal action would cost us more and take longer than changing our whole system to 2.4GHz. So we did.

Quosque tandem interferentes abutere patientia nostra

Amplifiers and cables as trouble sources

Amplifiers for outdoor use are not as reliable as we would like; some flaws in the operation of our system were due to amplifier failures. Cabling and connectors are also a source of problems if they are not handled properly.

Installation of outdoor amplifiers requires some care with proper protection against water pouring into its case. The case cover is tightly closed with an o-ring, and all possible points where water could get through are sealed with silicon rubber. Nevertheless we had an amplifier that died after nine months on duty. There was more than 2ml of water inside the case and some contacts were deteriorated by oxidation. The remains of the device were sent back to the manufacturer for analysis.

Another amplifier failure that annoyed us a lot was a switching from transmit to receive state before the transmitted signal went off. This problem was very hard to detect because it looked like as if some weird source of interference were in the air. We were kind of paranoid about interference at that time and that was our first hypothesis, quickly dismissed after tests with a spectrum analyzer. The funny thing was that by reducing the maximum transmit unit of the interface from the usual 1500 octets to a very low number, say, 168, the problem was minimized. We could only find out that the troublemaker was the amplifier after swapping every component on that line, from the radio board to cables, DC injector, and amplifier for their backup units.

The frequency response of the amplifiers goes far beyond the spread spectrum range. This makes them sensible to signals in neighbor bands. The symptoms are the same as those of interference but nothing is observable with a spectrum analyzer. The good news is that the use of cavity bandpass filters between the antenna and the amplifier to make it more frequency selective. The bad news is that a cavity filter costs about the same as the amplifier itself and is a rather clumsy object to install outdoors. Notice that a cavity filter would not avoid the kind of interference described above.

We experienced some failures due to cables with improperly assembled connectors. At least in one case it was very hard to identify the fault. Nowadays we test every cable and connector before installation despite the fact that they feature beautiful logos and quality certificates of well-known, respected manufacturers.

Conclusions

We have been successfully operating a wireless network for Internet access since November 1996. It proved to be cost-effective: we offer the service profitably at a price lower than that for leased lines. The project for UNITAU showed that the same technology fits the need for connectivity of distributed organizations in urban areas. Remote sites can be reached also at relatively low cost. Another point to remark is that once an infrastructure of base stations is set up, installing satellite stations is quick and easy. A portable satellite station made up by a notebook PC with a PCMCIA radio adapter may be useful in emergency situations. In areas where the conventional communication infrastructure is precarious or not existing at all, systems like those described here may be the best -- if not the only -- solution to bring Internet access.

The good cost-performance of this technology depends strongly on the availability of low-cost off-the-shelf components, like the WaveLan boards, cables, amplifiers, and antennas. Such equipment is not expensive due to mass-scale manufacturing, which is only feasible because the ISM bands are free areas of the spectrum (although in a free area, some regulations must be obeyed so everybody can coexist there). Look for radio equipment that does exactly what ours does but in another frequency range. It will certainly cost one order of magnitude or even more than if it were designed for an ISM band.

Another key point for the success of our wireless networks is Linux operating system -- not just because it is free, but because it is distributed with the source code of every component. The availability of source code was instrumental to make it work with a previously unsupported new generation of radio boards. The talk to the developers of the code via e-mail was better than any commercial software support we are aware of. Initiatives to produce free software like Linux and the Free Software Foundation must be encouraged and supported.

Interference, especially in the low ISM band of 915MHz, is a real drawback. Unfortunately this band is between analogue cellular telephony and ground-based applications. Cellular telephony is not a source of interference because its operation is very professional and in accordance with regulations but we cannot say the same about the ground-based applications around 931MHz and above. ISM unlicensed operation is a secondary service so it is supposed to accept any interference, though the only legal argument against the operators of interference sources is to prove that they often violate the regulations on side lobes and modulation. We never observed denial of service by interference in the 2.4GHz band.

References

1. Geier, Jim, Wireless Networking Handbook, New Riders, 1996, 413p.
2. Kirch, Olaf, Linux Network Administrator's Guide, O'Reilly & Associates, Inc., 1995, 335p. See also http://sunsite.unc.edu/LDP, the Linux Documentation Project home page.
3. WaveLan home page http://wavelan.netland.nl/
4. Spread Spectrum Scene Online http://www.sss-mag.com/