By Aryeh M. Friedman and John Sokol

Overview

Early models for the World Wide Web suggested that its primary value would be to serve as a global client - server network, or to provide a distributed database solution to users. Today, the best accepted model for the Web is that it serve as a total solution for information provision and content delivery. However, the technology has not come close to realizing this vision. This creates a huge problem for the staffs that administrate the corporate web servers. Chief among these problems is the simple fact that standard methods of socket programming lead to severe limitations both in response time and in the maximum number of connections.

The IDBS Afterburner Web Server was designed to overcome these limitations by using John L. Sokol's systems architecture. In arriving at the solution, Sokol tested 18 off-the-shelf solutions, and designed and tested 30 of his own solutions. The architectural design was implemented by Aryeh M. Friedman. This paper will detail our unique approach to solving these problems.

Traditional Web Servers

The best way to think about how a web server works is by examining either the workings of inetd or ftpd. For the sake of simplicity we will use the ftpd model in this paper. It is our belief that a web browser is nothing more then a very intelligent ftp client and display utility.

A model of ftp shown in the following chart (Figure 1). As you can see, this is a conceptually simple model; the reality is much more complex. One aspect that makes it more complex is the fact that older versions of ftpd (pre-inetd versions) spawned a child for each request. This severely limited the number of simultaneous users. For the most part this is not a problem for ftp, since there is not a high "hit" count on ftp servers, except for a few notable exceptions such as ftp.cdrom.com and ftp.uu.net.

Simplified Model of FTP ( Figure 1)
Picture pending

The vast majority of web servers use the same basic architecture as pre-inetd ftpd. That is, they spawn a child for each request, if no free children are available.

Please see Figure 2 below.
Simplified Model of NCSA and Apache

UNIX has two major methods of creating new processes. The first is to use fork(2). This creates a complete copy of the calling process; this method is normally referred to as heavyweight multitasking. The second method is to use vfork(2), which creates a "thread" of the calling process. Threads are basically programs that have some or all their memory copied to new address space and create a second program counter in the kernel. Threads are often called lightweight processes. Most web servers use fork(2) to create children. As can be seen from the ftp model, this creates major limitations. The Afterburner Web Server

The Afterburner does not spawn children. It simply scans all available connections for requests and/or readiness to receive the response.
Please see Figure 3 below.
Afterburner Model

Afterburner Advantages

A. Speed

Since the Afterburner does not require the huge overhead of heavy- or light- weight multitasking and it sends all responses from preloaded files, it is lightning fast. Tested performance exceeds 10M hits per day (this is extrapolated from a 30 minute test). We were not able to test the server on anything faster then a 10 Mbps connection. At that bandwidth we had saturated the LAN it resides on. We speculate that it can handle much higher loads on a faster LAN.

B. Response Time

Since the Afterburner uses preloaded files, a typical response is much faster than under NCSA or Apache.

C. Binary Log Files

With NCSA, Apache, and most other servers, logs are in ASCII and grow in size with great speed. The Afterburner’s logs are binary (5 bytes per hit). Thus, a typical log partition can hold more then 20 times more data than most other servers.

D. Anti-Hijacking

The original reason for designing the Afterburner was to prevent an external site from stealing our images and our bandwidth. This was accomplished basically as follows:

IMG SRC="http://our_site.com/img_name.jpg"

Images we had posted on the web were extremely popular. About 90 percent of the hits came through the (pirate) external web site, and we derived no benefit from this since they gave no credit. So, we decided to find a way to prevent this. The obvious solution was to put all the images behind CGI's and check the referrer, but even with the remaining 10 percent of the hits, the machine was dying under the load -- it ran out of process table slots. Our solution was to design the Afterburner to both increase speed and prevent such hijacking. To this end, the Afterburner checks the referrer. If it not the right one for that image, it sends back a default image. In our case the default image is an advertisement for our site. Afterburner Limitations

A. No CGI Support

In order to maximize the performance of the Afterburner, we decided not to include CGI support in version 1.0. In version 2.0 we plan to include either limited or full support for CGI’s. We have a possible method for solving the fork(2) method of creating new CGI processes.

B. No Virtual Hosting Support

In order to optimize the code, we have not included support for virtual hosting. Several solutions are under consideration for implementation in version 2.0.