INET Conferences

Conferences

INET


NDSS

Other Conferences

[INET'98] [ Up ][Prev][Next]

A Platform for the Development of Extensible Management Applications

Noemi RODRIGUEZ <noemi@inf.puc-rio.br>
Michele E. LIMA <michele@inf.puc-rio.br>
Ana L. MOURA <ana@inf.puc-rio.br>
Michael STANTON <michael@inf.puc-rio.br>
Catholic University of Rio
Brazil

Abstract

This paper presents a platform for the development of extensible management applications based on the interpreted language Lua and several associated libraries. Applications developed on this platform can be easily extended, with no need for recompilation. A specific library developed for TCP/IP network management is described in detail. The paper discusses the flexibility provided by the resulting platform and its relationship to other similar projects.

Contents

1. Introduction

Current growth in the size and complexity of networks and also in the use of network services and applications has created a wide range of requirements for network management. The traditional solution of individual management applications for specific devices and services is inadequate, and the need for an integrated management environment is obvious. Because it is not possible to predict all the different devices and services which will need to be managed in the future, this environment must also allow for extensions. Another factor which points to the need for extensibility is that the usefulness of "black-box" management applications diminishes with the development of different patterns of network use. While some sites may rate reliability as their major concern, others may aim for lower average response times or may be concerned about accounting. System administrators should thus be able to configure management applications in accordance with their needs.

Many applications offer customization facilities through the use of menus. Although this is sometimes an appropriate solution, requiring little technical sophistication on the part of the user, the extensibility provided is very limited. In order to obtain a higher degree of flexibility we can offer a programming language as the tool for customization. The use of scripting languages for configuring applications is now recognized as an important design technique, not only allowing options to be configured by the user at run-time but also providing a cleaner design for the application. With a convenient syntax, simple tasks, such as changing colors and fonts, may be performed through very simple commands, while at the same time full programming facilities are available for the more sophisticated user.

However, the same language can be used not only as a configuration tool but also as the basis for the development of the applications themselves, providing extensibility to all components, from the user interface to core functions.

For this to be possible, the extension language must offer an interface providing full access to all protocols and services which are relevant to network management. By offering access to these services, along with other useful facilities such as support for building graphical interfaces, management applications with differing levels of sophistication can be written. Once an application is developed, it can be used without modification in different hardware platforms or customized to a certain level by users with little programming experience.

This article describes the use of Lua, an extension language developed at PUC-Rio, as the basis for a platform for building of extensible management applications. Lua [1], [2] is an extension language with a simple syntax, which can easily be mastered by inexperienced users for simple configuration tasks, while at the same time it provides rich programming support for advanced users. Lua is freely available for downloading across the Internet [3]. Its interpreter is written in ANSI C and is currently being used on operating environments ranging from DOS to CRAY. A set of predefined functions is available for interfacing Lua and C code. Using these functions, a series of libraries has been developed for Lua.

In the remainder of this paper, we discuss the platform composed by the use of Lua and these libraries. Section 2 presents a quick overview of Lua. In section 3, we describe in detail a library called LuaMan, which was developed specifically for network management; other relevant libraries are briefly discussed. Section 4 discusses extensibility aspects. Finally, section 5 contains some concluding remarks.

2. The extension language Lua

Lua is a general purpose configuration language developed at PUC-Rio, which integrates in its design data-description facilities, reflexive facilities, and familiar imperative constructs. On the "traditional" side, Lua is a procedural language with usual control structures (while, if, etc.), function definitions with parameters and local variables, and the like. On the less traditional side, Lua treats functions as first class values and provides dynamically created associative arrays (called tables in Lua) as a single, unifying data-structuring mechanism.

Lua is provided as a library of C functions to be linked to host applications. The host can invoke functions in the library to execute a piece of code in Lua, write and read Lua variables, and register C functions to be called by Lua code. Moreover, fallbacks can be specified to be called whenever Lua does not know how to proceed. Lua can also be used as a standalone programming language: the interpreter can be invoked on any chunk of Lua code.

Functions in Lua are first class values. A function definition creates a value of type function and assigns this value to a global variable. Like any other value, function values can be stored in variables, passed as arguments to other functions, or returned as results. This feature greatly simplifies the implementation of object-oriented facilities.

Lua is dynamically typed. Variables can contain values of any type. Whenever an operation is performed, it checks the correctness of its argument types.

The type table implements associative arrays, that is, arrays that can be indexed not only by integers, but also by strings, reals, tables, and function values. Associative arrays are a powerful language construct; most typical data containers, like ordinary arrays, sets, bags, and symbol tables, can be directly implemented by tables. Tables can also implement records simply by using field names as indices. Lua supports this representation by providing a.name as syntactic sugar for a["name"].

Because functions are first class values, table fields can contain functions. This property allows the implementation of some interesting object-oriented facilities, which are made easier by some syntactic sugar. For instance, the expression 

      receiver:foo(params)

is syntactic sugar for 

      receiver.foo(receiver, params)

That is, the function stored in field foo from object receiver is called, with receiver as its first argument (which plays the role of self). A more complete treatment of Lua's object-oriented facilities may be found in [1].

Reflexive facilities in a language allow a program to manipulate its own execution environment. Part of the reflexive nature of Lua is provided by predefined functions such as nextvar which allow a program to iterate through its global variables (returning the name and value of each of them). An important reflexive facility is provided by tag methods. Tag methods allow programmers to change the usual behavior of Lua, especially in situations where the interpreter does not know how to proceed. For instance, consider again the statement 

      receiver:foo(params)

Usually, this expression signals an error whenever receiver is not a Lua object and/or foo is not a method of receiver. With tag methods, the program can catch this error and give this expression a meaningful semantics in this context. This kind of facility is used to implement many language extensions [4], [1].

3. The development platform

The network management platform we present in this paper is based on the programming language Lua, described in the previous section, and its libraries. This development environment allows network management application designers to construct fully extensible applications and provides them with access to several relevant services, including network protocols and services, support for building portable graphical interfaces, and dynamic Web page development.

Lua can be used either together with C, which acts as a host language, or as a stand-alone language. In the second case, the Lua interpreter can be used as a command console, where an operator can interactively enter a Lua command and examine its results. By incorporating extension libraries to the Lua interpreter, the console operator has immediate access to all the facilities provided by them. A command console is extremely useful for application prototyping and testing, allowing different application components to be executed and tested independently, with no need of developing additional tools for that.

Section 3.1 describes LuaMan, a Lua extension library which has been specifically developed to provide for the particular requirements of network management. In section 3.2, some of the readily available Lua libraries are presented.

3.1 LuaMan

The construction of network management applications requires access to several relevant network services. Access to SNMP (Simple Network Management Protocol) operations allows management applications to interact with SNMP agents and so manipulate the management information stored in their MIBs. Access to ICMP provides mechanisms which can be used for managing devices which do not support SNMP. ICMP messages are also used in the classical algorithms for tracing routes and IP network discovery [5]. Another facility which is also extremely useful in a network management environment is access to DNS (Domain Name System) services.

Access to SNMP and other relevant network services was achieved in Lua through the development of LuaMan, a library which provides Lua with a network management API. Such an extension of the language with a library written in C is readily supported in Lua. Several features of LuaMan were based on Tnm, the Tcl Extensions for Network Management implemented in Scotty [6]. Similarly to Tnm, one of our main goals was to provide a higher level of abstraction than is usually provided by existing management platforms. Whenever possible, we tried to hide the complexity and details of the protocols and services to be accessed. LuaMan also provides integrated access to the several protocols and services relevant to management applications, not only SNMP operations.

The current implementation of LuaMan is based on the CMU SNMP Library [7]. The LuaMan library can be ported to a wide range of platforms. It is currently running on Linux, Solaris, SunOS, IRIX, and AIX. Work is under way for porting LuaMan to Windows NT.

The following sections present a brief discussion of the network management API provided by LuaMan. The complete documentation of the LuaMan API can be found in [8]. For a detailed discussion on LuaMan see [9].

SNMP access

LuaMan provides access to the basic SNMP operations (get, get-next, and set). Both synchronous and asynchronous operations are supported. The current version of the LuaMan library supports SNMP version 1 (SNMPv1). The next version of LuaMan will support the community-based SNMP version 2 (SNMPv2C).

Similarly to other SNMP APIs, LuaMan implements the notion of SNMP sessions. SNMP sessions are used to configure the parameters that control the protocol interactions with the associated SNMP agents. These parameters include the agent's IP address, the protocol version to be used, and authentication information. When creating an SNMP session, the LuaMan user can also optionally define default callback routines for receiving responses to asynchronous SNMP operations and for handling notifications (trap and inform messages) sent by the associated agent. When invoking an asynchronous SNMP operation, the default callback routine configured for the session can be overridden by a specific routine.

When invoking an SNMP operation, the MIB variables to be accessed can be identified by their names (MIB labels) or object identifiers in dotted notation. The successful execution of an SNMP operation returns a variable list, where each variable is described by a varbind table: a Lua table which contains the variables oid, type, and value. SNMP data types are mapped to Lua basic types (string, number, table, and nil). Whenever possible, formatting rules are applied to the returned values.

SNMP MIB definitions

The MIB functions implemented by LuaMan provide access to SNMP MIB definitions. With the support provided by these functions, management applications can retrieve the syntax and semantics of the management information they obtain from SNMP agents and are thus able to present and manipulate this information more effectively. The MIB functions are also extremely useful for developing MIB Browser tools.

The mib_load function implemented by LuaMan enables management applications to dynamically load SNMP MIB definitions stored in MIB files and add them to the global MIB tree.

ICMP access

By providing access to the ICMP protocol, LuaMan enables management applications to implement simple monitoring procedures, like testing host reachability or obtaining their network masks. Another useful facility is provided by the icmp_trace function, which sends UDP packets to invalid ports and interprets the resulting ICMP error messages. This supports the development of IP networks routing traces similar to the traceroute utility.

In order to simplify IP network discovery procedures, LuaMan implements a icmp_netecho function. Given an IP network address and its mask, this function sends ICMP echo requests to all possible IP addresses on the network, returning a table which contains the discovered active IP devices.

Access to DNS services and local databases

The DNS functions provided by LuaMan were defined not only to provide access to the information maintained by DNS services but also to support experienced users in the development of DNS debugging tools. The DNS debugging support is provided through additional result values for DNS queries. Because Lua syntax allows unnecessary result values to be ignored when a function is called, this extended support does not affect adversely our goal of offering a simple syntax for basic DNS queries.

LuaMan supports all basic queries to DNS services, including translation from Internet names to addresses and vice versa, retrieval of host information and mail exchanger records, and the authoritative servers for a given domain.

Finally, the DB (database) functions provide access to the information stored locally in the Unix configuration files hosts, services, protocols, and networks.

Programming example

This section presents a programming example to illustrate the use of some of the functions provided by the LuaMan library. In this example, a Lua function -- walk -- implements the classical "MIB walk" tool in Lua. 

function walk(host,commStr,subtree)
  s=snmp_open{peer=host,community=commStr}
  root = mib_oid(subtree)
  if root = nil then
    print("snmpwalk: invalid subtree ("..subtree..")")
    return
  end
  vBind = {oid=root}
  repeat
    vBind = snmp_getnext(s,vBind)
    if vBind then
       if not strfind(vBind.oid,root) then 
          mibEnd = 1
       else
          print(sprint_var(vBind))
       end
    end
  until (not vBind) or mibEnd
  snmp_close(s)
end

The first LuaMan function called -- snmp_open -- creates an SNMP session to be used for invoking SNMP operations on the agent (host), identified by its host name or IP address. For simplicity, we only specify the community string to be used for authentication and use the default values for the other configuration parameters (protocol version, port number, etc...). Function snmp_open is essentially a Lua table constructor. Note the use of the syntactic sugar snmp_open{...}, instead of snmp_open({...}). This invokes the function with a new table as its parameter.

The next function called -- mib_oid -- translates the label that identifies the MIB subtree to be traversed to its corresponding object identifier.

The agent's MIB subtree is traversed with the use of snmp_getnext. Note the use of strfind which, along with several other useful functions, is provided by Lua string manipulation library. LuaMan also provides some utility functions to format the results returned by some of its basic functions. One of them is sprint_var, which receives a varbind table and returns a displaystring containing the variable's name (MIB label), its type, and its formatted value.

Finally, snmp_close terminates the SNMP session, releasing all associated resources.

Other programming examples can be found in [6].

3.2 Other Lua libraries

In this section we describe several libraries currently available for Lua which are important in the context of network management. We also mention some of the tools currently under development using these libraries.

Access to CORBA [10][11] agents is becoming an important feature in management applications, especially in the area of telecommunications. The LuaOrb library [4] provides a binding between Lua and CORBA. Unlike other bindings, however, this mapping is based on the dynamic nature of Lua and the Dynamic Invocation Interface of CORBA, without the use of stubs. This allows us to use Lua as a management console; using this console, the programmer may query the interface repository and access the newly discovered objects. By providing LuaMan and LuaOrb libraries, our network management platform enables application designers to develop management applications which integrate access to both SNMP agents and CORBA clients. This facility is explored in [12].

A management console is an extremely useful tool for tests, avoiding the need for developing a specific program for each required type of test. However, more permanent applications obviously need graphical interfaces. The IUP library [13], [14] is implemented for a number of different environments, such as X-Windows, OpenLook, Motif, Microsoft Windows, and Macintosh. IUP allows a Lua program to control graphical interface elements such as windows, menus, and so on.

Another important facility for the development of management applications is access to databases. The DBLua library [15] allows Lua programs to access any database with an ODBC interface.

A recent trend in management applications is the support for Web-based interfaces. In the Web-based management approach, a WWW browser is used as a management console, avoiding portability problems. However, an interactive Web-based management tool requires the construction of dynamic Web pages and some kind of gateway between the CGI programs and the conventional SNMP agents [16]. The CGILua library supports dynamic Web page development and manipulation of data typed in HTML forms [17] Using CGILua, pages may be defined either through pure Lua scripts, which must produce an HTML page as output, or through mixed HTML files, where Lua and CGILua instructions are interleaved in conventional HTML documents. The combination of mixed files and pure scripts is quite powerful, allowing specialized Lua scripts to be activated by mixed files, where the graphical layout is defined.

Because the Lua scripts can contain calls to LuaMan and to the other libraries mentioned above, a gateway between HTTP server and SNMP agents is directly available. Using CGILua and LuaMan, a Web-based management application called has recently been developed.

4. Extensibility

As stated in the introduction, a major goal of our effort is to provide a platform on which easily extensible applications can be built. This section evaluates our progress in this direction.

Presented in [9] are several examples of typical management tools developed using Lua. Two of these examples use LuaMan to implement the well-known Internet facilities ping and traceroute. Other examples use IUP and LuaMan to provide graphical query interfaces to DNS, enabling the user to retrieve information about mail exchangers, domain responsibility, name to address conversion, and so on. The time and level of complexity involved in the development of these example applications were considered quite satisfactory. The last of the applications described in [9], NetManager, is an extensible network discovery tool. This was also developed with LuaMan and IUP and allows the user to dynamically add new menu items and associate them with Lua scripts. NetManager provides a graphical window for the creation and modification of scripts, with basic editing facilities. This is interesting because the environment of the management tool itself is used for the creation of new functionality, in contrast to other tools, where scripts must be edited offline.

We consider the development of LuaWebMan [18] especially important as an evaluation of our management platform. One first positive point was the fact that LuaWebMan was the first management application in Lua which was not written by the authors of the LuaMan library. To evaluate the effectiveness of the use of LuaMan in conjunction with CGILua, typical management facilities, such as MIB browsing and network discovery, were again chosen to be implemented. The development of these facilities proved to be quite straightforward. Of particular interest is the use of a tool called ImageMaker [19], which dynamically creates sensitive maps. In LuaWebMan, the discovered network is shown as a set of clickable machine icons, each of them associated with a script which contacts the corresponding SNMP agent. This kind of feature shows that it is feasible to dynamically build interactive Web pages using only standard WWW resources. However, the most interesting point about LuaWebMan is that it is an extensible Web-based management application. LuaWebMan, analogously to the Net Manager tool described above, offers the programmer an environment for defining new CGILua scripts and associating these scripts with new menu elements. Moreover, the windows in which the results of these scripts are shown may also be configured by the user. LuaWebMan provides an explicit testing facility for new scripts. After editing a script, the user may select a "test" option. In consequence, a standard window is shown, containing either the result of the execution or the Lua error messages. The use of the same Web-based environment for managing the network and for developing and testing new management facilities seems to highlight very well the potential of the platform we are proposing.

5. Conclusions

The use of Lua with LuaMan and other available libraries is very similar to the platform described in [6]. In fact, we owe many of the concepts of the LuaMan library to Tnm, Scotty's Tcl extensions for network management.

Currently, Tnm offers a wider range of services than LuaMan. Some of the services offered by Tnm, such as HTTP and CMIP access, are out of our scope of interest. Other facilities will be added as they become necessary. Two different network administration groups at PUC-Rio are now starting to use LuaMan to develop applications for their everyday use. Their requirements will guide our plans for extensions. Although the idea of basing the development of management applications on an interpreted language is not new, most of the projects with this approach use Tcl [20]. We believe, however, that Lua is much simpler to learn and use than Tcl [21]. As a simple example, the following Tcl line of code, which updates a variable "s" with the value "s+I", 

  set s [expr $s+$i]

is written in Lua as 

  s = s+i

which seems quite a bit more intuitive. As to performance, preliminary tests have shown Lua code to execute more than one order of magnitude faster than the corresponding Tcl Code [1].

The number of "interesting'' libraries available for Lua is also an important factor in its use as a basis for developing network management applications. The philosophy of offering a simple syntax coupled with a powerful semantics has been extended to many of these libraries. As an example, consider the LuaOrb library, which allows a Lua program to transparently access Corba objects using the DII. Access to the DII is also offered for Tcl in [22]. However, in that case the programmer must explicitly command each step of the dynamic invocation. With LuaOrb, the programmer may directly access the methods of the remote object, and the steps of the dynamic invocation are carried out by the library. This kind of transparency makes it easier to blur the distinction between programmer and end-user, especially in the case of network management, where the end-user frequently has some programming background but is not necessarily an expert in all the related interfaces and services.

As discussed earlier, one of our main goals while developing LuaMan was to offer a higher level of abstraction than is currently provided by existing network management platforms. Many of the more complex details of the SNMP management protocol, such as data encoding, composition and interpretation of PDUs, and error handling, are dealt with internally by the library and hidden from the user interface. However, we believe that an even higher level of abstraction should be provided. We are designing a new programming interface -- based on the object-oriented features supported by Lua -- which tries to encapsulate not only the management protocol details but the management model itself.

While CORBA seems to be gaining increasing importance in the network and systems management context, the traditional SNMP and CMIP management models will certainly persist in the network management environment in the near future, r both technological and economic reasons. It is thus reasonable to assume that heterogeneous management models will be likely to coexist and that transparent access to the distinct manageable objects will be a major concern to network management application designers [23]. By providing the abstraction layer mentioned above, and with access to CORBA provided by LuaOrb, we believe our platform will be able to meet the requirements set by this new environment.

References

  1. R. Ierusalimschy, L. Figueiredo and W. Celes. Lua -- an Extensible Extension Language. Software: Practice and Experience. 26(6). 1996.
  2. R. Ierusalimschy, L. Figueiredo and W. Celes. The Programming Language Lua. http://www.tecgraf.puc-rio.br/lua/
  3. ftp://ftp.tecgraf.puc-rio.br//pub/lua/
  4. Roberto Ierusalimschy, Renato Cerqueira and Noemi Rodriguez. Using Reflexivity to Interface with CORBA. International Conference on Computer Languages. IEEE, Chicago, 1998 (to appear).
  5. J. Schönwälder and H. Langendörfer. How To Keep Track of Your Network Configuration. Proc. 7th Conference on Large Installation System Administration (LISA VII). Monterey, California, 1993. ftp://ftp.ibr.cs.tu-bs.de/pub/local/papers/discover.ps.gz
  6. Jürgen Schönwälder. Scotty -- Tcl Extensions for Network Management. http://wwwsnmp.cs.utwente.nl/~schoenw/scotty/
  7. Carnegie Mellon University. CMU SNMP Library. http://www.net.cmu.edu/projects/snmp/
  8. Michele E. Lima and Ana Lúcia de Moura. LuaMan. http://www.inf.puc-rio.br/~michele/luaman/
  9. Michele E. Lima, LuaMan: a platform for development of extensible management applications. Master's thesis (in Portuguese), DI, PUC-Rio. January, 1998.
  10. Object Management Group, Inc. The Common Object Request Broker Architecture and Specification; Revision 2.0. 1995.
  11. Jon Siegel. CORBA Fundamentals and Programming. John Wiley & Sons, 1996.
  12. N. Rodriguez, R. Cerqueira, R. Ierusalimschy and A. Moura. Aplicações de gerência com comportamento dinâmico. In Anais do SFBSID'97, pages 476-487, Fortaleza, Brasil, 1997.
  13. C. Levy, L. Figueiredo, M. Gattass, C. Lucena and D. Cowan. IUP/LED: a portable user interface development tool. Software: Practice and Experience, 26(7), 1996.
  14. TeCGraf (Group of Technology on Computer Graphics). Multi-Platform user Interface System - IUP/LED. http://www.tecgraf.puc-rio.br/publico/ingles/cenpes/produtos/iup.html.
  15. TeCGraf (Group of Technology on Computer Graphics). DBLua Library. http://www.tecgraf.puc-rio.br/manuais/dblua.
  16. Steve Steinke. Network Management meets the Web. Network, 12(12), 1997.
  17. A. Hester, R. Borges and R. Ierusalimschy. CGILua: A Multi-Paradigmatic Tool for Creating Dynamic WWW Pages. XI Simpósio Brasileiro de Engenharia de Software (SBES'97). Fortaleza, 1997.
  18. Edison Ishikawa. Web Based Management . Master's Thesis (in Portuguese). DI, PUC-Rio, 1998 (to be submitted 04/98).
  19. Flavio Spolidoro. ImageMaker for Network Management. Final Project. DI, PUC-Rio, Dec. 1997. http://www.tecgraf.puc-rio.br/~spol/imagemaker.
  20. M. Rose and K. McCloghrie. How to Manage your Network using SNMP. Prentice-Hall, 1995.
  21. J. Ousterhout. Tcl and the Tk Toolkit. Addison-Wesley, 1994.
  22. G. Almasi, A. Suvaiala, C. Goina, C. Cascaval and V. Jagannathan. TclDii: A TCL Interface to the Orbix Dynamic Invocation Interface. http://www.cerc.wvu.edu/dice/iss/TclDii/TclDii.html.
  23. Bela Ban. Generic Management Model for CORBA, CMIP and SNMP. PhD thesis, University of Zurich, 1997.

[INET'98] [ Up ][Prev][Next]