Evaluations and Recommendations Toward a Generic User Support Toolkit
Christine CAHOON <email@example.com>
Today user support and education are more critical than ever. Appropriate channels are required to advise users on the efficient and responsible use of the Internet, and such methods are suitable for providing advice and support on a wide range of other topics also. There has not yet been any highly visible "public" consolidation of how to leverage network services for the provision of this support. The work described in this paper will result in clear guidelines for building scalable, effective, cost efficient, "connectable," widely accessible, easily managed, online support systems. Final recommendations will be based on the results of an international survey; members of the ETINU task force mailing list (representing input from over 35 different countries); and an evaluation panel of experts.
User support is potentially much easier for an organization or national service today than ever before. However it is also much more likely to become a nightmare, draining resources beyond worst expectations, with little or no assurance of effectiveness.
This paper describes work to progress towards a vision of what we believe user support in a networked environment should and could be. From a consideration of the requirements of today's networked users and those "supporting" them, we are working towards a wide consensus for the desirable features of an online support "system" and recommendations as to how these features could be implemented. This is the work of the TERENA (Trans European Research and Education Network Association) ETINU (Environment To Inspire Network Users) "task force" . The end result of the work is a toolkit for online user support, which will be freely available and include comprehensive recommendations for user support implementation standards (due June 1998).
The ETINU task force is part of the TERENA technical program. The e-mail discussion list includes around 300 individuals. A panel of ten experts has been formed to evaluate the final reports of the task force before publication. TERENA project development officers have worked with the authors (the task force leaders) to build a foundation for the exchange of ideas on providing better user support. Earlier work has been reported at other meetings, including INET'97. In addition to establishing standard methods of providing support, the project is promoting wide cooperation and collaboration by encouraging the linking of user support systems which conform to agreed standards.
Our starting point has always been a belief in the potential of established network services to revolutionize how user support can be provided. With the wide deployment of, in particular, electronic mail and Web browser software, there is the possibility for all to use these means to both request and receive support. As techniques such as sending and receiving e-mail, checking for news on the local Web server, or downloading software upgrades increasingly become second nature to most users, then easy-to-use network resources can be built which provide a single familiar interface to many helpful functions. With former support services and many existing support services, these functions may only be available through a confusing array of different contact points and access methods. Our vision consists of a framework for a support interface which can accommodate all, or a selection, of a set of "standard" functions, satisfying varying support requirements, with recommendations for choice and implementation of software.
The choice of support functions is made to produce an environment where
We intend that the results of this work will be applicable to a single organization providing a support service for computer-related applications. However, it should not stop there. With the proliferation of the Internet and its pervasion into daily life in every country of the world, we aspire to provide guidelines which will also help Internet service providers to encourage subscribers to use the Internet responsibly and promote sharing of knowledge at all levels within an order which enhances value to all. Further explanation of these ideas is found in the section entitled "Proof of concept projects."
ETINU is an acronym for "Environment To Inspire Network Users." The actual words of the acronym are not as important as the concept: to provide local resources for network users which enable and empower them to take full advantage of the powerful interface available on a desktop computer.
We believe that introducing new working practices which help bring this to a fuller realization is the top-priority task for the whole area of user support today. Consequently the ETINU task force work involves the IETF (Internet Engineering Task Force) user services working group (USWG). Regular briefings are given at USWG meetings and USWG members participate in the ongoing discussions and deliberations of the task force.
At the first formal meeting of the ETINU task force in May 1997, it was decided that a survey should be conducted to gather evidence of what was currently happening with user support, rather than making assumptions on what could have been a small and unrepresentative sample of user support experiences.
Upon a successful application for project funding from TERENA and with the involvement of the IETF user services working group and TERENA's information services and user support working group, the authors organized a global survey on user support. The survey questionnaire is online .
The survey was conducted between November 1997 and February 1998. The results identify the main difficulties being experienced in the provision of support and what approaches are proving successful. These are now informing the compilation of a user support "toolkit" or software "building blocks" for effective online support.
Since the target audience was the online community, the questionnaire was designed as an electronic Web-based form. Figure 1 shows the banner of the online form, with simple instructions for respondents.
The objective of the form was to obtain information from a respondent about the user support service at his or her site in as simple a manner possible. Both user and staff perspectives were invited. There was opportunity both to provide set replies with dropdown menus, radio buttons, and selection boxes and to supply "free-form" comments.
The content and layout of the questions were proposed and modified in several cycles of refinement on the ETINU e-mail discussion list. The final form contained 11 sections on the following:
Figures 2 and 3 below are screen shots of different sections of the questionnaire. Point-and-click selection made completion of the form as enjoyable as possible. The division into simple sections avoided the impression of an onerous list of questions. Colored text highlighted the questions to be answered as compared to simple instructions about the form.
We felt strongly that the survey results should be summarized automatically as the survey proceeded and that respondents should receive immediate "reward" for their contribution in the form of a very useful report. The program which processes the form therefore also performs a first level of analysis by maintaining a simple frequency count of all set responses as forms are submitted. The total frequency counts after the addition of those on any form are presented to the respondent in a graphical format. Figure 4 shows a section of a summary such as a respondent would see following submission of the form.
It was also desirable to make available online all the comments which had been provided. The program which processed submitted forms also formatted the supplied comments and appended them to relevant comment files. The summary of results presented to a respondent then contained links to all of these files, as shown in Figure 5. In this way all the results from the whole survey were available as a network resource. Respondents were also given a URL to regenerate the results at any later date .
At the time of writing there have been over 360 questionnaires submitted. The following observations are based on the first 336 of these. A complete set of observations will be published during March 1998 and will be accessible from the ETINU task force Web pages . Figures 6.1-6.8 are presented here as samples of the many possible views of the data now available for full analysis.
Figure 6.1 shows the role breakdown of those who participated in the survey. One-third were managers, another third were local support staff, and another third consisted of system administrators, users, and national support staff.
The relative use of telephone, e-mail, in-person interview, and World Wide Web within an organization's support service was selectable under three categories: much use, some use, and no use. For "much use," 68 sites selected telephone and 53 sites e-mail. For "some use," e-mail, Web, and interview were selected by around 45 sites each, while 31 sites selected the Web under "no use."
Figure 6.2 shows the selections for main method of providing support.
44% indicated telephone, 24% e-mail, and 20% in-person interviews.
Only 3% stated the Web. Other methods used to provide support
included video conferencing; bulletin boards; fax; training sessions;
old-fashioned handwritten memos and being caught in the hallway!
Figure 6.3 shows the distribution of methods used to enable users to help themselves. Both paper and electronic documentation were popular, followed by network-based FAQs and self-training material. Relatively few sites provided a query database for users.
Other methods mentioned in the free responses to enabling users to help themselves included training courses (more than half of those that commented); e-mail discussion lists; newsgroups; special interest groups; help within applications; assistance from other users; and newsletters.
The questionnaire requested up to three names of software packages which were used to help with user support at the respondent's site. Of all those mentioned, Figure 6.4 shows that 53% had been purchased, 31% were public domain, and 11% had been developed in-house.
A section on features of the software used to help with support revealed that 31% allowed user submission of queries, 34% query progress reporting, and 30% access to a knowledge base, and only 6% had computer telephony integration. Other features mentioned included customer history and linking with experts.
Figure 6.5 shows the relative numbers of sites related to the extent of the support service offered. 25% of sites were specific about what they actually supported. The rest allowed for queries to be answered on a best-effort basis or with no restrictions.
Figure 6.6 is very informative, suggesting that most respondents (60%) felt that the answers to past queries could be reused in 20% to 60%+ of cases.
Figure 6.7 indicates the respondent's satisfaction with the local support service. In most cases the rating was fair to good (62%) but with a significant number (25%) registering uncertainty.
A more specific aspect of the support service was the provision of native language material. This was rated good to excellent for most cases.
Figure 6.8 illustrates one of the most significant results of the survey. The perceived importance of a standard approach to providing online support was very important for 62% of respondents. 22% felt that it was of limited importance and only 4% thought that it was not important. Furthermore, 75% of those who thought it was very important were managers.
Summaries of estimates of the numbers of supported users suggested that most sites supported 1001+ users. The most common quota of dedicated support staff was 4-10. The normal frequency of use of the support service by typical user was several times a month. With 30% of sites, 10-50 requests were received daily and 15% received 51-100.
The native languages of the respondents who provided a native language are shown below.
Language Frequency _____________________________ Albanian 1 Brazilian-Portuguese 1 English-French 1 Georgian 1 Italian 1 Korean 1 Luxembourgish 1 Norwegian 1 Polish 1 Portuguese 1 Ukrainian 1 Chinese 2 Croatian 2 German 2 Greek 2 Spanish 2 Swedish 2 Welsh 2 Arabic 3 French 4 Dutch 9 English 250 _____________________________ Total 291
Not all provided a comment to the following prompt (Q11 on the questionnaire):
Explain, if applicable, what elements of your user support service are not provided in your native language and how this affects the usability of your service.
All the meaningful comments that were provided are registered below. We have filtered out the "not applicable" and "no problem" comments which were usually from English speakers. (The full story is online .) We've divided the 42 meaningful comments that were left into two very crude classifications for both English and non-English native speakers, as follows:
English native language respondents 19 Describing some difficulties 8 Emphasizing no difficulties 11 Non-English native language respondents 23 Describing some difficulties 12 Emphasizing no difficulties 11
The result tentatively suggests that half of the respondents who provided optional comments on the language issue perceive that there are difficulties to be addressed.
Many desirable features of an online support system were identified. Ease of use was mentioned more than anything else, both from the user and staff viewpoints. The other highly desirable features mentioned in a free response section were
Other features mentioned included links to other resources; 100% availability; access to customer histories; flexibility; scalability; an announcement facility; integration with other aspects of the service.
A wide range of software was identified by respondents. Further investigation is ongoing to check the correspondence of different supplied names. The table below lists all packages which were mentioned by three or more different respondents along with the frequency of occurrence.
Software Frequency _____________________________ Remedy ARS 41 Microsoft Access 10 Support Magic 9 Web-based solutions 8 McAfee Help Desk 5 File Maker Pro 5 RMS Helpdesk 4 Lotus Notes 4 Help-Desk 4 DTS 4 Vantive 3 Req 3 Helpline 3 Clientele 3 _____________________________ Total 106
It was noted that although the most popular package is a commercial help-desk program, many of the replies indicated software tools which would cater to parts of a support service.
The telephone is used a lot. Yet there is relatively little computer telephony integration mentioned in survey responses. Telephone support is quite limiting in that only so many calls can be received at any one time. Scaling the service requires additional phones and corresponding people to answer them, unless a form of impersonal automation is introduced.
Electronic mail is the second most popular method of providing support. This is encouraging. This method allows requests to be made at any time without depending on immediate interaction with support staff and is therefore theoretically an optional component for a much more scalable service.
It would appear that the Web is not being used widely as a main support medium. The reasons are not clearly deducible from the survey results. In fact, many of the desirable features described by respondents would appear to be addressed by Web-based systems. Some of the software identified as being used to provide support related to the Web will be correlated with the satisfaction ratings at those sites in the ongoing analysis to determine whether or not use of the Web suggests more satisfaction.
A study of the numbers involved in the user support scene at different sites suggests that similar numbers of support staff are involved regardless of how large the number of users supported is. Further analysis will reveal whether or not there is direct correlation between satisfaction and the number of users per support person ratio.
As a very rough summary of the potential resource savings of an effective online support system, we could combine the most frequent responses to particular questionnaire sections to state that a typical support service with up to ten dedicated staff provides a service which processes approximately 9000 queries a month from 3000 different users. One member of the staff could receive 900 queries a month or 30 per day. Of these 30 daily queries, at least 6, and perhaps 20 or more, could be automatically handled from the answers to past queries.
The headings in this section are the proposed main components of a generic online user support system. Together they represent what we believe is the essential and desirable functionality of networked user support. This is derived from ETINU meetings and survey results. The indented text (C1-C8) at the end of each subsection below indicates the initial specification of software to be met. The authors do not regard this as a definitive schema by any means but do recognize the importance of "starting somewhere" and making that starting point an object of discussion.
Common practice involves query submission in different ways, mainly telephone; electronic mail; Web-based forms; and in person. It is desirable for all queries to be "framed" at submission so that subsequent processing can be common to all queries.
Ideally, of course, queries would be dealt with upon submission and a satisfactory reply given immediately. Any recommended system should not prevent this.
C1: Accept queries submitted from a variety of communication media and convert all to a common form for processing
In order that past queries can be learnt from and used for future reference, all or a selection of queries must be stored. Corresponding responses to queries must also be stored. For now the details of exactly how these "packets" of information are stored is not as important as the requirement that any stored query or response can be retrieved using specified criteria which satisfy a particular support service. Our own recommendations for these criteria are described below under "Indexing scheme."
Having stored a "query" in a format suitable for subsequent processing, it is desirable to identify different types of queries which need to be handled in different ways. A first classification of queries could be into two main categories, those requiring a response and those not, followed by a further breakdown as follows:
requiring a response fault reporting (hardware, software, network, person, mixture) request for assistance or advice not requiring a response useful information comment on another query
A "query" which does not require a response and provides useful information could be considered to be as much a response as a query. In fact, all queries and responses could be considered as "packets" of information with different properties.
C2: Store "queries" and "label" appropriately
Once a query has been "labeled" with a scheme such as in C2 above, different properties can be associated with it depending on the label. The most important properties for consideration are those of queries requiring a response, i.e., a status which at different stages of processing must be one of the following:
outstanding in progress cleared
Another property of a query which requires a response would be a pointer for responses to that query.
Responses could be handled in a similar way to queries. Properties of responses would include a pointer to one or more queries which that response relates to and, perhaps, whether or not the response was considered "definitive."
A facility for updating a query status is needed which ensures that only an appropriate authority can action such.
C3: Update query and response status and clear when appropriate.
A large store of queries and responses could be arranged to provide a very useful "knowledge base"; however, an effective user support service should provide a higher level of guidance for users, particularly the new and naive, in the form of frequently asked questions. These would serve as "first call" reference material for both users and support staff and provide suitable content for responses to recurring queries.
In order to be of maximum relevance, a frequently asked questions (FAQ) resource should be constructed from "real" recurring questions. A tool is required to extract these from the store of queries and responses and propose a suitable makeup for an addition to the FAQ.
If queries and responses have been assigned sufficient properties which are readable from the store, then it may be possible to automate the production of a new frequently asked question to a large extent. However, it would seem prudent to provide an optional level of approval, whereby an appropriate individual would be used to moderate additions to the FAQ resource, employing easy-to-use editing tools to modify the addition or reject it.
It is also conceivable that a number of FAQ resources could be generated for different topics which have been identified in a classification of support areas.
C4: Optional level automated FAQ compiler and associated editing tools
The store of queries and responses will become very useful whenever particular items can be located or linked easily. A site may already use a database system for this. However, in this section we are proposing that standard parameters could be chosen and applied in almost any system to provide similar functionality.
The key to effective searching and linking is quality data, i.e., queries and responses stored in a "structured" manner with adequate "tags" of supplementary information or "metainformation" or accompanying database fields defining the properties of a query or response.
The embedded properties define the criteria which can be used in retrieving any item, matching items to other items (e.g., previous responses to new queries, etc.), updating or modifying items, generating an addition to a FAQ resource, and so on.
However, the embedding or tagging process that defines the properties of an item should be as automatic as possible and not interfere significantly with query processing. If queries are submitted through an electronic form, then some properties can be provided with a few mandatory point-and-click selections before the form is submitted.
An indexer program would construct a searchable index from the store of queries and responses on a regular basis using a selection of the properties associated with each item. During this indexing, URL link checking and reporting could also be performed. A search facility using the index may also include a thesaurus lookup to enhance effectiveness. This thesaurus could be generated locally with simple tools to record similar words at a particular site.
It is desirable to conform to wider standards where possible and so the 15-element set of descriptors of the Dublin Core initiative  should be used as a reference base for the properties of a query or response. These include
Other "tags" which may be useful in a support system context are
status of a query or response outstanding, in progress, cleared -- for query definitive -- for response associated URLs local remote pointer to related responses -- for query
Some properties will be basic and always be present for a query or response; others may only be added at a particular stage of processing.
C5: Indexer and corresponding search facility with optional thesaurus lookup for keywords
A measure of the use of support resources (people, documentation, programs, etc.) is necessary in order to assess the effectiveness of approaches or improvements to a service. With an online system most of this can be automated to produce, for example, logs of query submission; logs of query clearing; or numbers of outstanding queries at regular intervals, all of these sorted by support topic, or person who resolved, and so on.
A process to check the occurrences of nontrivial words and phrases in the query store with the objective of identifying possible candidates for addition to the FAQ resource could be regarded as a subprocess of this component and precursor to C4 above.
C6: Automatic reporting configurable for variety of output options
With today's network services, a wealth of material is available online, or can be made available online, to enhance a support service and a user's learning activity. When pointers to this material can be juxtaposed with a current response to a query, then even more value is delivered to all concerned. The following resources should complement any query resolution service and pointers to relevant parts of them should be "insertable" in responses to queries and items of an FAQ resource:
help screens about using the support service training modules for user self-learning documentation on a range of services and facilities available software archives containing locally used programs and updates pointers to relevant Web sites search engines, subject-based information gateways, etc.
An up-to-date list of such resources should be available to users at a highly visible level of the support service, perhaps as a preamble to the FAQ resource.
C7: Annotation facility with easy resource pointer insertion
Although our proposals so far have suggested "batch processing" of queries, ideally the initial registration of queries would happen in a very short time and, in many cases, appropriate action could be initiated by support staff immediately. This may require interaction with a user at that user's desktop. This is possible using the network without the necessity of support staff physically visiting the user. To highlight the development of such remote control tools within widely deployed browser packages, we have added this final component:
C8: Set of tools for "synchronous" interaction with a user's desktop when desirable
One of the main challenges for ETINU is to address the above specifications using lightweight software tools which are easily accessible to everyone. Modular solutions are much more attractive since sites wherein a well-established support service is in use could apply "incremental" improvement.
The following provides a brief overview of some of the most frequently mentioned packages in the survey replies. At the time of writing, requests for further information about other packages have been sent to sites using those packages.
Remedy  was the most popular package reported. The Action Request System (ARS) integrates components to give a highly customized help-desk environment. Components include Remedy Help Desk, which automates problem management, problem resolution, and reporting; and ARWeb, which allows users to search for solutions themselves, submit queries, or update database information using the network.
Microsoft Access databases can be easily accessed via the Web if a Windows NT server is used, or loaded into a package such as MySQL  on a Unix Web server for wide access via standard browser programs. Furthermore the MySQL database can be kept up-to-date from the original Access database using ODBC (the open database connectivity standard) calls. FilemakerPro is a similar generic solution which was mentioned in the survey, providing "hooks" for seamless Web access to a repository of support material.
SupportMagic  is a package for "automating" a help desk. It contains a full-text search engine for fast resolution of queries. Modules included with the basic license: call tracking; customer accessibility; system monitoring; and reporting.
McAfee  is a solution designed to work with SQL database servers such as Oracle, Sybase, and MS SQL. It features call tracking; a search and retrieval engine; a knowledgebase authoring tool (fully customizable); reporting capabilities; and "orchestrator" technology which translates user e-mail messages into trouble tickets and forwards them to an appropriate address.
Lotus Notes  is a widely used tool for general exchange of information within an organization, providing a "distributed database" facility. An associated module on a Web server (Domino) enables controlled Web access to the Lotus Notes information by anyone with an ordinary browser.
DTS  SmartCall is a computer telephony integration package, designed to automate telephone-based calls to a help desk as much as possible, with options for machine-generated replies. It is a modular solution, requiring a back-end database to work with, in order to provide a full support service. It contains ODBC interfaces for Oracle or other SQL servers and support for Access, Paradox, FoxPro, dBase, and Btrieve.
Vantive  is a system in which users submit queries using the network. Queries are classified into "specialty areas." User self-help options are included with automated table-driven problem routing and escalations. Facilities also include asset management (hardware, software, and network), a cumulative knowledgebase which can integrate third-party "knowledge packs," and a variety of reporting tools.
Helpline  encourages the creation of a data warehouse of help-desk information. Facilities enable the data collected through a help desk to be used as feedback to improve the effectiveness of the support service. Specific features include keyword search; case-based reasoning; artificial intelligence; utilizing commercial knowledge bases; and asset management.
Clientele  is a system designed to manage all aspects of dealing with users. It provides for the organization of relevant information, tracking problems, making information available to users, and even ordering shipments for customers in a commercial setting. It includes an easy-to-use report generator, and the "AnswerBook" module provides a Web interface.
We note that two types of software solutions have been reported in the survey. The first is a typical commercial help-desk package which will attempt to manage all aspects of a support service from the help-desk administrator's point of view. The second is a generic tool which was not primarily designed for user support purposes but is being applied in different ways at different sites within the overall support process. We believe there is scope to extend turnkey solutions using modular extensions within the user realm (as opposed to tools for help-desk managers). There is also an opportunity to bring more consistency to the way in which generic database or other tools are being used in a support context.
Very little mention was made in the survey of browser modules which have become freely available during the past year. Packages such as Microsoft's Internet Explorer version 4 with NetMeeting  or Netscape's Communicator with Conference  present the possibility of sharing desktop environments (e.g., a user's with support staff) or of taking control of a browser on a remote machine and leading through a series of Web pages. Whiteboarding and chat facilities also allow display of screen shots or other material related to problems or their resolution. And supplementary packages such as Realplayer  enable reasonable quality video with audio to be used for training or education purposes. Furthermore, Push technology  could also be employed to disseminate useful support-related information on different support topic channels in an effort to proactively alert users of potential problems and avoid them before they occur.
Software identified in the survey or mentioned above represents only a fraction of that available commercially. Packages range in price and complexity and are often simply referred to as "support center" software. We highlight here three of the many useful resources on the Web which provide pointers to further information about such software.
Philip Verghis maintains an excellent source of help-desk-related software . His comprehensive "FAQ" on help desks includes information on software packages and related tools. Of interest also is a set of pointers to reviews of current help-desk software.
Iain Middleton's help-desk pages are also an excellent resource with pointers to relevant software  and reviews.
The "decision-maker" Web site contains a library of overview documents and a valuable list of reviews of current popular support packages .
Table 1 summarizes the matrix which we intend to refine with further analysis of the survey results and follow-up discussions and software reviews.
Codes of practice for help-desk organization have been published . These relate to promoting the provision of high-quality IT services and have been coordinated by the Central Computer and Telecommunications Agency. A help-desk section proposes standards for planning, implementing, and operating help desks.
Our recommendations (full publication due in June 1998) will focus on using the network for support and promote the adoption of components similar to those described above. They will also offer guidelines on their introduction, maintenance, and development. Reviews on specific software will replace any generic terms in the software column of Table 1. Where applicable, recognized "technical" standards, such as MIME compliance and the use of international character sets, will be emphasized.
It is also intended to provide suggested implementation details, such as a pump priming set of support topics into which support effort can be divided. A fuller treatment of metadata for queries and responses will also be included.
One of the authors is a trainer for the Internet Society's workshops for developing countries and will be using a demonstration support system to maintain contact and self-support between participants at the 1998 workshops. A full description will be available on the ETINU task force Web pages  and discussed at the INET'98 conference.
As suggested in the introduction above, the methods and ideas of ETINU are by no means restricted to technical support. A further demonstrator project has been proposed and is, at time of writing, in the preparatory phase of discussion between a number of participants. This involves Northern Ireland community advice groups working with the general populace through an "ETINU system." Also, the ETINU approach was presented to members of the Belfast City Council Information and Communications Technology (ICT) Forum in April and that Web site  will be transformed over the coming months to include a new Web-based communications system for project groups and task forces.
In May 1998, one of the authors will describe the ETINU approach to a conference of linguistic analysts (ICAME 98). Some interest has been expressed by the conference organizer in encouraging this group of experts to apply the ETINU recommendations in an online system to share expertise in this field. An update on any progress with this project will be available on the ETINU task force Web pages .
At the time of writing, the final analysis of the ETINU survey has not been completed (report due in April 1998). However it is already clear that the aims of the project have been endorsed emphatically by the group of 350 respondents who have completed the questionnaire. The full analysis report will be available on the TERENA Web server  shortly after the time of writing this.
We are not so arrogant to assume that this work is exhaustive. We trust that we are putting in place a firm foundation for the further development of useful meaningful services for sharing expertise which will continue to promote the Internet themes of cooperation, collaboration, and consolidation.