The tremendous progression of the Internet has set the ground for the rapid development of distance learning on the Web, so that many universities, colleges, and private companies throughout the world are developing courses in this new medium. The context has several characteristics that make it very favorable for the construction and fine-tuning of Web courses:
The consequence is that Web courses are getting more and more sophisticated, and course designers are geared towards integrating several technologies for presenting text, sound, images, animations, and video in sites for a rich multimodal interaction. That trend has a direct impact on the scope, complexity, and cost of the projects.
The construction of a Web course requires the resources of a multidisciplinary team that is expected to master different technologies and tools, work efficiently, and yield a good product while dealing with important time and budget constraints. To simplify the development of certain parts of the project and reduce the production time and costs, designers often have recourse to commercial solutions. Nevertheless, this approach is of limited help because of several other problems:
As a consequence, the institutions that are currently investing in this new domain lack basic information to make enlightened decisions. This study stems from these problems: it presents a critical review of several key issues about distance learning on the Web, and aims at helping course designers and managers to make better decisions in this new domain. In addition, it suggests several research directions for the future.
The paper is divided into three main sections. The first section presents the advantages and disadvantages of Web courses, the progression of the enrollment for courses, the services available in Web sites, and the underlying technologies. The second section is about the cost evaluation, and the commercial solutions available for building Web courses. The third section is about the performance of distance learning on the Web. It presents an evaluation of the impact of different technologies (e.g., hypertext, e-mail, newsgroups) on learning. In the conclusion we discuss several important questions and suggest promising research directions for the future of Web courses.
Table 1 presents a list of advantages and disadvantages of distance learning on the Web from the point of view of four different stakeholders, namely the student, the professor, the institution, and the society [11, 16, 19, 21, 22, 29, 30, 31]. Because they are self-explanatory, we will not comment on each of these advantages and disadvantages.
Let's notice that the main advantages for the students are the accessibility to the course for those living away from the training center, in the far regions, or in other countries, for those with restricted mobility (e.g., the handicapped, the injured, the elderly), for those with irregular work schedules, or for those with family duties; other important advantages are concerned with self-paced and just-in-time learning, and the absence of waste of time in transport.
The main disadvantages for the students are the loss of direct (i.e., non-mediated), visual interaction with the professor and the other students, the loss of immediate feedback, the loss of motivation and the high rate of failures and drop-outs, and the problem of isolation. The main disadvantages for the professor are the loss of the dynamics of the class, the loss of immediate feedback (e.g., the students' mimics, questions, and comments), the difficulty of evaluating the student's work, and an increased workload at the beginning since one must prepare new material and anticipate the students' reactions.
The number of courses available on the Internet is rapidly increasing. Most of the North American universities and several European universities are already present on the Internet, via information sites or courses. Representatives from the International Data Corporation (IDC) survey firm concluded in 1999 that the number of students registered to higher degree training program on the Web was expected to reach 2.2 million students by the year 2002 . Distance learning is expected to represent 15% of the total number of the students registered to a higher degree program. Moreover, this trend will continue during the next 5 years at an annual increase of 33%!
A senior analyst from IDC  asserted that
Advancements in technology are breaking down barriers and changing the way teachers can interact with students.... The Internet is the catalyst attracting more schools and students to distance learning than ever before.
Until recently, the courses available on the Internet were only made of hypertext documents. The ever-increasing capacity of bandwidth now gives access to more sophisticated technologies. Modern sites now include different types of information and more complex functions or tools such as:
The use and the implementation of those services on the Web vary from one site to another.
The majority of Web sites devoted to distance learning use asynchronous communication technologies (e.g., e-mail, discussion list, sites with sound and video). These are considered simple to develop and implement and not too expensive (as compared to the synchronous ones). Most sites use three basic technologies: the Web, Newsgroups, and e-mail; a few of them also use more demanding technologies that support audio and video materials.
A minority of sites, belonging to a small group of institutions, use synchronous communication technologies (e.g., chat, IP [Internet Protocol] telephony, videoconference). These require a more complex infrastructure and can cost up to twice as much as the asynchronous ones.
The choice of technologies, in each institution, depends on several factors: the availability of the supporting infrastructure, the scope of the project, the nature and criticality of the information to convey (e.g., for medical applications), the allotted development time, the budget capacity, and the will to surpass the competitors. Although it is a decision with big implications, the choice of technologies often seems to be made without basic information on the real performance of each technology, its effect on learning, and the implementation costs.
Table 2 shows a fixed costs evaluation made by Bartolic-Zlomislic & Bates  from the University of British Columbia in Canada for producing a university-level course on the Internet over a 4-year period (N.B.: years 2, 3, and 4 allow the teacher to upgrade and fine-tune the original course).
The amounts in Table 2 indicate that the development and the formatting of a HTML (hypertext markup language) document are relatively inexpensive, since they represent only 11% of the total development cost. With such amounts, though, only simple HTML pages are being developed, without any form of automation or calls to the databases.
In addition, these amounts do not take into account various expenses related to the modernization of auxiliary services in the institution, such as the virtual library, on-line registration, admission, payments, and assistance. On-line registration, admission, and payment are fairly complex and expensive operations, especially when they involve foreign students.
As compared to a traditional course, the development of a Web course generates extra costs pertaining to the production, the diffusion, and the updating of the product. However, the absence of physical infrastructure (i.e., building) allows the saving of up to 15% of the cost of traditional courses [23 p.18].
The total cost of a new Web course is difficult to evaluate precisely. According to Bartolic-Zlomislic & Bates [21 p.10], there are three categories of costs to consider:
When considering both the fixed costs and the variable costs, the authors estimate that a new Web course costs $98,000 (US) for a 4-year period. Fixed costs are higher ($54,000) than variable costs ($44,000). The cost of a course amortized over a 4-year period comes to $24,500 US/year. At the University of British Columbia, the threshold of rentability can be reached with an average enrollment of 44 students per course during a 4-year period [21 p. 36].
Equivalent data are provided by different studies:
These are the minimum costs to build a new Web course. The purchase of complementary pedagogical material, the use of video technology, or the additional copyright fees generated by the use of the Web will make them increase. For instance, a study from the State Board of Directors for Community Colleges of Arizona [23 p.14] shows that the addition of multimedia functions to a course can double the production cost (see Figure 1). With animations and interactive programs, the cost of a Web course may easily climb to $120,000 - 250,000 US.
Figure 1: Allocated cost per course enrollment based on the average course enrollment per year (5-year period) (from Farmer, 1998 [23 p. 25])
These numbers do not take into account the expenditures made by the institution to manage the academic programs, or the fees paid by the students to access the system. The purchase or location of a computer, the costs of software, and the telephone service fees are difficult to evaluate because they vary a lot with the origin of the students.
New commercial solutions aim at reducing the development cost of distance learning courses and improving their quality. The automatic management of the exchanges can be obtained without having recourse to complex programming tools. Table 3 presents several commercial products for developing courses on the Web.
|Student home page|
|Chat session log|
The principal advantage of these products lies in the possibility of integrating synchronous and asynchronous communication functions without augmenting the production costs of the project. Their main disadvantage is the rigid framework they impose on the designers.
At the present time, there are no solid results on the performance of distance learning on the Web. The majority of the authors in favor of this technology justify their point of view only on the basis of their personal experience.
A report of Phipps & Merisotis  from the Institute for Higher Education Policy concludes that the majority of the studies done so far on distance learning lack scientific rigor, so that the conclusions are untenable. There are multiple problems in these studies:
Moreover, Phipps & Merisotis assert that the comparisons of performance between distance learning on the Web and traditional learning are biased towards the latter because of important content differences between the two. One major cause of this bias is that the Internet services offered by the libraries of several institutions are often limited because of legal, financial, or equipment reasons. Such limitations in the availability of pedagogical materials could have a significant impact on the choice and content of a Web course. The professors have to consider these limitations when they create a Web course.
Phipps & Merisotis are not the only authors who contest the validity of the studies on distance learning. Rouet [8, 12], a CNRS (National Center for Scientific Research) researcher at the Université de Poitiers, in France, observes that the frequent use of hypermedia systems generally leads to a significant improvement of the user's performance because of a phenomenon called "progressive accommodation." He concludes that the performance of distance learning courses is often hypothetical, and that one should be cautious about their results.
The impact of each single technology (e.g., hypertext, e-mail, newsgroups) on distance learning is poorly known, as will be shown in the next paragraphs, and there is no theoretical framework for explaining the impact. A fortiori, the impact of a combination of technologies on distance learning is still less known.
Hypertext links do not generate, as such, a better understanding of the matter by the students. However, one can suppose that the structure of these links may have an important impact on the internal representation of knowledge . For instance, problem solving could be advantaged by a hierarchical presentation of the problem data .
Special attention should be devoted to the structure of the links. Navigation through an educational site should be easy to understand, and the internal structure of a document should be transparent for the students. The majority of sites use a spatial map to represent the structure of the hyperlinks. This type of aid is generally considered useful to help the students navigate through the document, but is it the most efficient?
There is a choice between a spatial map, a conceptual map, or the absence of any help. A study tends to demonstrate that conceptual maps are better suited to distance learning sites than the others . Spatial maps have a positive impact on navigation but a negative one on learning. So one should reconsider their use in Web sites in favor of conceptual maps.
The evaluation of students can easily be automated through the use of specialized CGI programs. Self-evaluation questionnaires and final exams can be used to show the progress of the students.
The evaluation systems used on the Internet are not reliable because it is difficult to clearly establish the identity of the students during the exam. Some types of exams, such as multiple choice, make fraud detection almost impossible because the responses are never personalized. Nevertheless, this type of exam is the most privileged in Web courses.
When used isolatedly, the evaluation systems do not add any plus value to Web courses. They should be associated with another system that checks the real participation of students in different learning activities.
This solution can take various forms: for instance, a HTML (hypertext markup language) calendar can show the participation of the students through the use of different exchange programs (e-mail, chat, newsgroups, etc.). This has the advantage of monitoring the real implication of students while it allows the identification of mute students who require special attention.
The daily supervision of operations, through a log function in the system, can also provide the professor with precious information on the spread, duration, and particularities of each pedagogical activity. This has multiple advantages; it permits the professor to:
Such information allows the professor to build a work model of the student, so he/she can precisely identify, for instance, which parts of the course are the most difficult or the most neglected by the students. In this context, the professor has relevant information to improve the course.
E-mail is certainly the service that is the most used in Web pages. In the context of distance learning, it allows the students to ask questions, demand help, and receive feedback, in addition to breaking their isolation for some of them. As such, it plays a crucial role in supporting the interaction between the professor and the student.
Despite these advantages, e-mail does not improve the performance of the students in a significant way. A recent study  reveals that e-mail discussions do not produce in-depth learning. Neither do they improve collaboration among students.
Generally one would expect that mature-age students, owing to broader experiences, would provide a more reflective approach to discussions. From the analysis there appeared to be little difference in the responses....There was also no indication of interaction among the students while they used the e-mail discussion list. The participants tended to write their response in isolation without considering the responses of others in their group. (Ruth Geer) 
Some authors think that the desire to collaborate is closely related to the social context of the students. Those who live far from the cities would be more prone than those living in the cities to ask for help from their fellow students because of the scarcity of local resources . The desire to collaborate is not a priority for adult students because they much more prefer to get in contact with the professor than with their fellow students [11, 21]. Consequently, the impact of e-mail on learning remains very limited.
Another inconvenience of e-mail in the context of distance learning is that it generates an increase of the workload of the professor who is expected to answer the messages at irregular hours, during evenings or week-ends.
However, e-mail has the big advantage of accelerating the requests for help of the students to solve technical problems, which are considered a major source of annoyance in distance learning [21, 30].
The dialogue between the students of a course can take place within newsgroups. The permanence of the messages can be considered as a major advantage for the students but the technology of the newsgroups is difficult to manage. The problems are numerous: necessity of a common schedule, fear of the students related to the permanence of the messages, poor group organization, divergent discourse, and necessity of a permanent regulator [21, 27, 28].
The sequential organization of the discussions has the advantage of keeping the discussion well focused and limiting the dispersion of the group. However, it imposes a rigid communication model that does not take into account the individual differences; moreover, the participants may come to forget the purpose of the discussion. It follows that this creates an effect of "tunnel vision" which often leads to breaking the discussions into distinct categories.
Newsgroups take much time to develop , and it may be necessary to segment the discussions into distinct groups to help launch them. Sometimes this practice creates an opposite effect, because it leads to a segmentation of the discussion into divergent groups .
The use of newsgroups represents a challenge to the professor. On one hand, segmentation and a large variety of topics are necessary but they do not guarantee the development of the group. On the other hand, too much homogeneity can yield the same result. As a consequence, there is no way to launch a successful newsgroup on the Internet.
The improvement of the Internet permits the use of synchronous communication technologies for distance learning. Some commercial products already propose off-the-shelf solutions to use such technologies.
For some professors, there is no doubt that chat, video, and collaboration space are essential tools for distance learning. They bring some realism to the exchanges; nevertheless their advantages remain to be proven.
Communications in text mode encompass several constraints: the student must be at ease with the keyboard, he/she has to master the operation of the system, and he/she must understand the different usage conventions.
Communications in video mode are less constraining for the students because they are simple to use; however, they require a larger bandwidth. The difficulty of indexing video documents limits their use as reference documents for distance learning. This lack could soon be eliminated by products such as the Microsoft Research Annotation System (MRAS), which allows one to directly index video documents [1, 20]. The MRAS is an ActiveX application that is inserted into a Web page to communicate with an ActiveX server via the HTTP protocol.
Another possible usage of video consists in linking the professor and the students in real time. This type of system can also be inserted into a collaborative space to improve the exchange modalities. For instance, the Flatland project from Microsoft allows the students to receive a video image of the professor in four different synchronous collaborative work zones [15, 32]:
At first sight, this type of tool seems ideal to re-create the environment of a class. However, the first tests on this system showed that the complexity of the interface reduces the quality of the interactions. The confusion between the different modes of communication generates disorientation among the participants and a lack of naturalness in the conversations. This type of solution has to be refined in order to eliminate the complexity of use and reduce the users' cognitive workload.
There is a race among training institutions for building distance learning courses on the Web, in light of the anticipated benefits for the students, the institutions, and the whole society. This is another clear demonstration of the rapid and ubiquitous progression of the digital world. Here, the educational world is strongly invested. Numerous training institutions are already very active on the ground because they want to acquire experience, master and exploit the technology, adapt to the needs of their future students, be more attractive so as to conserve or improve their share of the market, have more visibility throughout the world, and be among the leading users of the new technology in education. Nevertheless, one should remind us that we still know very little about this new medium, and that the decisions to develop Web courses are still made without solid and complete justifications.
In the next few paragraphs, we raise several basic questions that directly call for more research about distance learning on the Web.
First, and the most important (this was mentioned a few times in this paper), the effect of Web courses on learning is still unknown. What do the students really learn in a distance learning course taken on the Web, what proportion of students stay enrolled in the course, and how does it compare to traditional courses? We need a metrics of the performance of the course that will take into account all the relevant variables, including the abandonments or drop-outs by the students. We agree with a professor at the New York University's School of Continuing and Professional Studies , when he says that it would be wise for the moment to limit the number of distance learning courses until their real impact on learning is better known.
On-line courses can be developed very cheaply [in our opinion, this is quite arguable], but what people are discovering is that the success rate are not very high. The issue here is not to have thousands of courses, but to have highly focused courses that meet large needs. (Gerald A. Heeger) 
Second, and this flows from the previous point, it is necessary to develop a common metrics for measuring and comparing the performance of different Web courses. This will incite the professors and the institutions to strive for high-quality courses, it will encourage the research community to develop some benchmarks, and it will allow the students to choose the best Web course available in their domain on the market.
Third, a theoretical framework is needed to better understand the cognitive and pedagogical foundations of distance learning on the Web, with two priority topics to investigate: (1) the relation between Context x Content x Students x Level of competence to be reached, and (2) the impact of each technological solution (e.g., Web, e-mail, newsgroups, chat, bulletin board, video) on knowledge acquisition, representation, and use. Those priorities correspond to several questions: Is Web course suitable for any context? Any matter? Any type of students? Any level of competence to be reached? Does a hypertext structure change the way a student acquires and represents the knowledge? We believe that to raise the questions is to answer them. This will allow the trainers and managers to know what is best for the students.
Fourth, the community should shift from a technology-centered approach, as is currently the case, to a student- and content-centered approach wherein the decision to go for a Web course would not be driven by the technology and what it can do or not, but by the students' needs and profile (e.g., education level, cognitive style, motivation), and the nature of content to acquire (e.g., conceptual vs. applied, declarative vs. procedural). After all, the goal is to train people successfully, not to develop and use the technology for itself.
Fifth, and finally, in light of the disadvantages of Web courses for students (see Table 1), the Web may be valuable for a few courses only in a training program, not for the whole program of the student. And yet, some institutions are proud to announce that they offer a complete virtual program. In our opinion, they jumped the gun since they made a decision without even knowing the impact of a single Web course on learning. At least, we hope they will keep the community informed of their results and thus allow us to learn from their experience.