Linux Means Business at Holt Public Schools
Holt Public Schools is located in the mid-Michigan area, and is comprised of approximately 5,400 students ranging from pre-school to 12th grade. We have a wide area network of 10 schools and an administrative office building. Within these sch ools, are a total of about 900 workstations, some 600 Macintosh machines (LC3's and Centris 610's) and around 300 Intel machines. The network, as installed, was comprised of a number of 10mb/s Ethernet LAN segments connected by 56k leased lines. The DSU/C SU units were controlled by Novell 4.x fileservers using Newport Systems LAN2LAN cards and routed only IPX traffic. The Novell fileservers provide NetWare services to the Intel machines and AppleTalk services to the Macintosh machines.
Since 1993, the computer networking world has been turned upside down, and we recognized the need to update our system. In addition to the need for greater bandwidth for applications, getting Internet access to the classrooms has become a must. Public schools have always been faced with tight financial budgets, and implementing a major change in a network is usually quite costly. Because of these factors, our district was forced to examine alternative ways to upgrade our system. Through the use of two key technologies: spread spectrum wireless WAN links and Linux proxy servers, we have been able to provide everything our district wants at a minimum of cost. Both of these technologies, when combined, give us a system that is highly functional and low in cost. In fact, based upon our financial calculations, the entire system (including the Linux machine hardware) should pay for itself within 5 years, based solely upon the recurring costs of 6 leased lines.
There were two major problems in upgrading our network. The first was that our WAN links were slow and very expensive. Using 56k DSU/CSUs, we were able to pass GroupWise e-mail, do some management, and replicate our Novell directory services. At this s peed, copying files was slow, but Internet access for the entire district was unthinkable. To provide for more bandwidth, we replaced our 56k lines with a wireless 4mb/s bridge unit from Pinnacle Communications of Dayton, Ohio. The wireless links can tran smit data at a rate of 2mb/s on each channel simultaneously and function as an Ethernet bridge.
An Ethernet bridge, in its simplest form, is a device that will selectively forward or drop a packet based upon its destination. The bridge learns which network devices (based upon its unique MAC address) are on which network interface and records the information into its internal tables. When a packet reaches the bridge, it looks at the destination of the packet. If the packet is destined for the opposite side of the bridge, it forwards it. If the packet is destined for a network device on the same si de of the bridge , it is ignored. In this way, the bridge only passes packets that need to be sent and eliminates unnecessary traffic between segments. The wireless bridge product that we use performs this exact function, between an Ethernet segment and a Wireless "virtual" network. This makes network configuration very simple and efficient. Consider the following examples:
Originally, we used a routed 56k solution such as seen in figure 1. This worked relatively well for our Novell network. However, it required that all of the traffic be routed. This is fine for NetWare, which uses protocols such as RIP to automatically configure routes. However, in order to pass TCP/IP data, it would be necessary to break our class C range of IP addresses into a number of smaller subnets. This would result in a net loss of available IP addresses, and also not solve the bandwidth problem .
With the wireless links, we were able to set up a much more flexible network. Because the links can be configured as either point-to-point or multi-point, it was possible to create a single virtual bridged radio network comprised of numerous locations. At the education center, we have OMNI-directional antennas which are configured as repeaters. Because the remote locations use directional links, and point directly at the education center, they cannot communicate directly with one another. To alleviate this problem, the education center bridge is configured to forward all traffic not intended for it's own Ethernet LAN segment back out to the wireless network. Thus, while one elementary school cannot communicate with another elementary school directly, t hey can communicate by making an extra hop through the OMNI. This happens transparently in the hardware, and is unseen by the network devices. With the exception of a single remote school (Dimondale), we were able to connect every location into a single w ireless network. We used a repeater at the west campus area to connect this auxiliary school to the larger network. At this location, there are essentially two different wireless links plugged into the same hub. Although the link to Dimondale is actually a totally different wireless network, it appears logically to be part of the larger wireless network. In all, the physical wireless network looks something like the picture in figure 2.
With the bridging topology, we were able to maintain our Novell communications, while expanding our TCP/IP functionality. For our TCP/IP services, we deployed a number of Linux proxy servers. These Linux servers are Pentium computers, ranging from P-90 s to P-150s. They have between 32 and 64 megabytes of parity ram, and IDE hard disks from 850mb to 2.1gigs. They each have two D-Link NE2000 compatible Ethernet cards. The machines have a minimal Slackware 3.2 distribution installed on them. The machines are configured as IP Masquerading firewalls, and act as Internet gateways for our remote LAN segments. Also running on the servers is the Squid Internet Object Cache software, which allows us to cache HTTP, FTP, GOPHER, and WAIS data on the server. Most o f the other Linux software, such as login shells, FTP services, etc. has been disabled or restricted to a single management machine.
Between IP Masquerading and the Squid Object Cache, we are able to provide the Internet services we need. With masquerading, we give access to our 900 or so clients with only 7 true IP addresses. In addition, we can configure the firewall to allo w different types of access to different workstations. For example, we might wish to configure the firewall in such a way as to restrict a computer lab while allowing a teacher station access. Also, because of the nature of the firewall, our clients are m ore or less unreachable by the outside world, thereby conferring a certain amount of security. Using the standard ipfwadm program, there are a number of possible configuration options.
Overall, the speed of the wireless links has been quite good. When the network first went up, we began gathering information about the speed and reliability of the links. To do this, a script was set up which runs a program called tcpspray to tr ansfer 100k bytes of data to each location and measure the amount of time it takes to get there. The script runs constantly on a management station and tests each of the links every 15 minutes. Below is the actual output of one of our wireless links - in this case, between the education center and the senior high school:
=================================================================== Tue May 27 18:19:35 EDT 1997 Sycamore/HS: Transmitted 102400 bytes in 0.551536 seconds (181.312 kbytes/s) ===================================================================
Running this same test to a machine connected via. PPP on an external USR 28.8k, we get the following results:
=================================================================== Tue May 27 18:25:43 EDT 1997 PPP: Transmitted 102400 bytes in 47.041576 seconds (2.126 kbytes/s) ===================================================================
Admittedly, that PPP link should be running a little bit faster. I had expected to see throughput more along the lines of 3k, or possibly 4k bytes/s. Lastly, to compare it to another popular networking option, take a look at the throug hput attained by a 10mb/s LanCity cable modem which I have connected to my personal Internet host:
=================================================================== Mon May 26 18:28:13 EDT 1997 Cable Modem: Transmitted 102400 bytes in 0.294357 seconds (339.724 kbytes/s) ===================================================================
Bear in mind that these figures don't give you the whole picture. For example, the speed of the throughput varies from moment to moment by as much as 50%. All it takes is a split-second delay in the network to give a very poor reading. The readings I have provided represent an average throughput. Also, the speed of the computers that are sending and receiving the information makes quite a big difference. All of the machines used for the above testing are running Linux, but if they wer e not, the speed of the TCP/IP stack would also be a factor. In addition, it's important to note that certain services have more data overhead than others. Thus, performance might vary depending upon the service you are using. FTP, for example, runs at roughly the same speed as a tcpspray test.
In addition to speed, controlling Internet access is important. As a public school district, we need to have a certain amount of accountability as to what our students do and see on the Internet. To this end, we have made the use of the Squid Cache man datory. This allows us to monitor what kinds of documents have been accessed, as well as to let us require a valid username and password to access the cache. By filtering all traffic for port 80 at the firewall, client workstations must use the cache to g et WWW documents. Squid allows for the use of htpasswd style authentication, exactly like web server packages such as Apache. Using this authentication method, we can manage user access to the cache, and consequently to the World Wide Web. In addition, Sq uid allows us to configure access control lists, which will disallow certain "known naughty" sites that might endanger the innocence of our students.
At each of the individual LAN segments, we have a configuration similar to the picture in figure 3. The wireless link is plugged into a small hub, which is also connected to the NetWare server and Linux box. The NetWare server is responsible for routin g the IPX/SPX traffic and the Linux box is responsible for the TCP/IP traffic. All of the client workstations are configured for the 10.0.0.0 network, and have their Linux box set as the default gateway. When a client sends TCP/IP traffic, it goes through the Linux box, out through the wireless link, to the education center, and eventually out to the Internet.
On the Novell side, configuration is quite simple. We simply left the original Ethernet card at it's original settings, and configured a second Ethernet card for the wireless LAN. Naturally, I had to configure this second card with the network number 3 141, so that I could call it the "Pi in the Sky." We must all seek humor where we canJ
The proxy servers have been working quite well. Our first Linux proxy server, which was based on kernel version 2.0.18 has run for months and never crashed. Even with the ever-demanding (and ever-popular) PointCast traffic, it has performed wonderfully . With Squid, all documents that pass through the proxy are cached, allowing subsequent requests to come from the proxy server at near-Ethernet speeds. This reduces traffic across our Internet connection, as well as across the Internet in general. In a s chool situation, this works very well. For example, when a teacher wants to visit a certain web site with the whole computer lab, they simply view the pages the night before. When the class comes in the next day, the documents are served very quickly, mak ing it possible for a whole class to browse the site at Ethernet speed. With the combination of helpful utilities such as wget, teachers can recursively cache a whole site with a simple shell command.
Overall, we have used Linux to put together a greatly improved network We have also provided controlled Internet access to all of our workstations with limited resources. We have increased our WAN bandwidth from a painful 56k to a respectable 4mb/s spe ed. With the help of the Squid Internet Object Cache, we have become responsible Internet citizens, and helped to reduce unnecessary net traffic. Now, we can even call all of our e-mail airmail. None of this could have been possible without the effort of the hundreds of people who contribute to Linux every day. In education, in particular, Linux fits perfectly - it's cheap, it' s flexible, and it's powerful.
Mark Lachniet is a Network Systems Specialist for Holt Public Schools. Mark's hobbies include disc golfing, "nerdin", and home brewing beer. Mark can be contacted at firstname.lastname@example.org. Mark maintains a small home page which includes a tutorial on IP Masquerading, a HOWTO-in-progress on creating a proxy server like the ones described above, and other miscellaneous stuff. His home page is accessible at http://scnc.holt.k12.mi.us/~lachniet.