Howto Transparent Caching Server or Appliance

From openSUSE

Introduction

What is this?

A transparent caching service is as implied, a service which is not visible to the logical network but supplies a caching or web traffic redirection service. But wait, there's more...

Description

The goal of this project or build is to have a server which cannot be seen as connected the way it is yet supplies several services, namely:

• Web based traffic caching
  • Keeps temporary copies of all Internet requests for later requests, speeding up the end user experience and reducing bandwidth utilisation.
  • Reports on utilisation and effectiveness.
• IP Firewall
  • Can filter unwanted traffic.
  • Can throttle lower priority traffic.
  • Reports on traffic throughput and utilisation.
• Application Firewall
  • Can inspect traffic and filter based on content as much as type.
  • Reports on utilisation and use.
• Web based management
  • Can be managed from any HTTPS browser.
• Appliance like
  • Is designed as a stand alone service and can be installed on just about anything from big server to small ITX embedded thingy (but not really discussed here, all you need is Suse to install and you are away).

For the above to materialise, the server will be connected in between an ADSL router (or any other entwork device) and the logical network in such a way as one would expect a router to be connected, yet the network address ranges on either side are identical making this a network bridge.

Target Audience

This project is suitable for anybody wishing to achieve the above objectives or install the above features, but the design consideration is for a network of <200 users who share a 3.5Mbps ADSL connection for email, web surfing, etc.

There will be a throttle facility available toward the end of the build if somebody else contributes, I am stuck on that part, please add that section if you know how.

Who can build this?

Anybody with little Linux experience. This project is intended to be the IT departments first Open source build and as such, your Windows colleagues need to be in a position to support this build in the event the metaphoric bus runs you down. This project is simple to build and commission as it is to decommission. In the event something fails, the system can be removed from the production environment with a simple cable replacement.

Skills Required

• Basic CLI experience.
• Some Linux knowledge, but not much.
• Intermediate TCPIP and Proxy knowledge.
• Some knowledge around DMZ and application firewalls.

Time Required

• About 3-5 hours for n00bs.
• Less than 2 hours for pros or somebody with the identical hardware and set-up as I have used.

What is Needed?

This describes the physical requirements for the project.

Hardware required

• 1GHz processor or above (can be one of many architectures).
• 512MB RAM or more (more is better as this is a caching application).
• 10GB of disk or more.
• CD or DVD drive.
• An Ethernet port.

Software required

• SuSE 10.1 media.
• Valid TCPIP information for a staic IP address.
• Internet connectivity.

Administrative Requirements

• Basic CLI experience.
• Some Linux knowledge, but not much.
• Intermediate TCPIP and Proxy knowledge.
• Some knowledge around DMZ and application firewalls.

This Build

For the purposes of clarification, I have used this setup for this build and documentation. I built the server three times using the instructions below, word for word.

• HP DL360G2 P3 1.4GHz, 2.5GB RAM, 2x18GB SCSI U320 Hot Swap with RAID1+0
• Novell SLES 10.1 SP1
• Active Directory network with lots of Windows drones around.
• New Zealand locality, (Quarter Finalists of the Rugby World Cup).

The Build

This is the build instructions to reach the end goal.

Installing SuSE

This is the base operating system of choice. Any Linux system would do, but SuSE is known to work well on HP and is supported by HP as such. Also, SuSE is supported as an enterprise system at a very moderate expense and is freely available directly from Novell in New Zealand, so it has the shortest distance to source. These instructions will likely work on any platform and may be interpreted for such purposes.

Hardware Prep

As the chosen platform is an HP DL360, there is some basic hardware prep-work. First off, the server needs to be updated using the Firmware CD from the HP website and the server needs to be prepared using SmartStart. The Firmware CD and SmartStart CD should be the same versions. Also, the server needs to be set-up such that it will easily accommodate Linux. The default set-up on these servers is for Windows based environments and this needs to be changed manually using RBSU link in SmartStart setup.

Media Acquisition

Media can be acquired from Novell, from their website or from a software distributor. There may be a charge related to distribution and packaging unless the media is acquired directly form the Novell website. When acquired from the website, there will be a registration requirement which will result in a activation key. Keep the key for the installation below. Failure to install the activation key will result in no software updates being available.

Check downloaded media against the MD5Sums from Novell. This will ensure no problems or hold ups are caused by corrupt or faulty disk media or dodgy ISO images. Instructions for this are on the Novell site from where the media was acquired.

A good application for this is a Linux desktop which will have md5sum on it or you can use MD5Summer from md5summer.org for Windows.

Installation

The best way to do this is to follow these instructions below and substitute as required for your particular build.

1. Boot to the CD and from the Installation Menu, test the memory, continue if there is a pass and no errors occur.
2. Select 'Installation' from the start-up menu, the installer will begin.
3. Select 'English UK' from the language selection and click Next.
4. Review and accept the license agreement and click Next.
5. Select 'New Installation' and click Next.
6. Select 'Pacific' and 'New Zealand' from the time zones available and check the hardware clock is set to 'Local Time' before clicking Next.
7. On the summary screen:
   • Change the keyboard setting to 'English US' and
   • Change the partitioning by:
     • Clicking the 'Partitioning' hyper link and,
     • Selecting 'Create Custom Partition Setup' and ,
     • Select 'Custom Partition' and,
     • Erase or remove all partitions and,
     • Create a new primary partition of 12GB at the beginning of the disk, using ReiserFS and,
     • Create a new swap partition using 1GB of disk or a size equal to your physical RAM up to 4GB and,
     • Create an EXT2 partition with the balance of the disk for caching - be sure to set the mount point to '/cache'.
   • All other defaults should be fine, the only other gotcha's here are the partitioning and the keyboard selection.
8. Click Accept to continue to the AGFA license agreement and confirmation message. The file copy procedure will ensue. The server will reboot automatically.
9. Once up, put in an administrator password (this is the local root password and should be hard to understand).
10. Check the host name and domain name are correct and uncheck the option to have the host name updated through DHCP.
11. On the Network Configuration screen,
    • Enable SSH by clicking on the 'blocked' hyper link,
    • Disable IPV6 by clicking on the 'Disable IPv6' hyper link,
    • Click on the Network Interfaces hyper link to configure the network cards and,
      • The card that is plugged in will be shown as 'DHCP', edit this card by highlighting it and clicking 'edit' and,
      • On the General tab, make sure the card is set to Internal Zone under Firewall and the Firewall is disabled or manual and,
      • In the Address tab, cycle though all the options to configure the device using the TCPIP settings you reserved for this server and click 'Finish' to return to the Network Configuration page once complete.
    • Click Next to move on.
12. On the Test Internet Connection screen, elect to test the connection. If this fails, step back and check all network settings. Click next when it works or skip if you are sure it will work later for whatever reason.
13. On the NCC Configuration screen, check 'Configure now', 'Hardware Profile', 'Optional Information', 'Registration Code', and 'Regularly Synchronize...'
    • The following screen will present you with a dialogue asking for your novell.com account information, complete and press Next.
    • Once complete, a message to this effect will appear and an option will appear to update the installation. This is a good idea, but not necessary, whichever way, click Next.
14. On the Installation Settings screen, accept the default settings for CA management and LDAP configuration and click Next.
15. On the User Authentication Method, select 'Local' (this server is an appliance, so no integration is required) and click Next.
16. For the New Local User, use anything but root and squid, then click Next,
17. The release notes will update you as to what has changed since the last release.
18. The hardware configuration screen will attempt to auto detect your graphics adapter and screen. Don't panic if it fails or is incorrect, we are going to use web based management. Click Next when you are satisfied.
19. On the Installation Completed screen, check the 'Clone this system for AutoYast' check box to have a script created of the build just completed. Make a note of the path of the file and ensure it forms part of the backup process to automate all the steps so far. Click finish for the path of the file.
20. The server will reload.

Testing

Test the installation by trying to SSH to the server using PuTTY or similar. Check the server can be pinged and check you can log into the server at the console and through PuTTY or any SSH client using the above supplied user credentials. By default, the root user cannot SSH (although I later found this not to be true...). Once on the server console, check the Internet is accessible. Resolve any problems till you are:

• Able to access the Internet.
• Able to remotely login using SSH.

Note: Although not required, in some cases the server needs to be rebooted. Reboot if there are any silly problems.

Setup NTP service

Linux services require the time to be correct, this step will get the server talking to your NTP service on site:

1. Open a PuTTY or any SSH session and login as root.
2. Open Yast and setup NTP services:
   • yast
   • Navigate to 'Network Services' on the left menu and,
   • Select 'NTP Configuration' on the right and,
   • Set NTP to start 'During Boot', select 'Run NTP Deamon in Chroot Jail' and click 'Add' to add server sources and,
   • Once in the Add dialogue, add 'server' type for each service available and use the IP Addresses.

Beware that Linux will wait for NTP availability for a while on boot, so use local service or create one, using one from the Internet may be a problem if the server needs to restart while the Internet is down.

Register and Update

Once the server comes up, if registration failed earlier, register now, through Yast and run the updates for all recommended updates, through Yast, as well..

Network Bridging

This is the step where the two Ethernet cards are set-up to behave like a very expensive and large two port switch and a virtual adapter is created to bind the two. That is, after these steps, all traffic will pass between the two physical cards and the virtual adapter, referred to as the bridging adapter. The bridging adapter will have an IP address, so it can 'see' the network, but addresses are removed from the two Ethernet cards so that traffic can flow without restriction.

Concepts of

Bridging is usually set-up in environment where distance restrictions are circumvented by having a way point between the two points on the network. The bridging device, usually a piece of hardware, will connect two pieces of Ethernet cabling together allowing two sites that are distanced from each other to 'see' each other and still work within the cable length limits of each cable running to and from the bridge. By nature, this device is transparent and can be a switch with other ports occupied or a smaller device designed only for one far points accessibility. Further, the device needs to be transparent or it will have to route traffic, so usually, a bridge has no IP information on either of it's ports or it will not receive traffic not destined directly for itself.

TCPIP setup

Linux is a very powerful operating system with many customisable parts. Here, we will change the way it networks to achieve a bridge between it's two ports. This part will be specific to SuSE, but the Internet will provide many different ways of doing this on many different platforms. Windows is particularly weak in this area at the time of writing.

In Linux in general, all the servers configuration data is in /etc. Further to this, SuSE have designed their Linux offering to have a /etc/sysconfig folder with more specific server configuration information. In /etc/sysconfig/network is all the data pertaining to the network interface and interoperability set-up.

Go there and modify the settings as such:

1. At the console login as root.
   • This step will disconnect the server enough to stop a remote session, so this has to be done at the server.
2. Install Brigde-Utils first:
   • yast -i bridge-utils
     • You may be asked for the install media.
3. Go to the network folder:
   • cd /etc/sysconfig/network
     • Note that interface set-up information is in configuration files that are preceded by 'ifcfg-<interface name>' and will be auto loaded based on that naming standard.
4. Backup the network configuration by making a folder and copying the contents there:
   • mkdir ../network.old
   • cp -Rv * ../network.old
5. Remove the existing configuration:
   • rm ifcfg-*
     • This will not kill the network connection till the service is reloaded and the configuration files are re-read
6. Create a new configuration file as such:
   • touch ifcfg-br0
     • This creates a blank file of given name.
   • gedit ifcfg-br0
     • At a terminal session from inside Gnome (the desktop) this starts the text editor. Create a file that looks like this:
       
       STARTMODE=auto
       BOOTPROTO=static
       IPADDR=YourValidIPAddress
       NETMASK=YourValidNetMask
       BRIDGE='yes'
       BRIDGE_PORTS='eth0 eth1'
       BRIDGE_AGEINGTIME='300'
       BRIDGE_FORWARDDELAY='0'
       BRIDGE_HELLOTIME='2'
       BRIDGE_MAXAGE='20'
       BRIDGE_PATHCOSTS='19'
       BRIDGE_STP='on'
       
       
7. Restart networking:
   • /etc/init.d/network restart
     • If there were no typos in the file, things should be up and running and the address above can be reached across the LAN.
8. Check output from ifconfig:
   • ifconfig
     • You should have something like this:

br0       Link encap:Ethernet  HWaddr MAC Address
          inet addr:YourValidIPAddress  Bcast:YourValidBroadcastAddress  Mask:YourValidNetMask
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:408 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:49183 (48.0 Kb)  TX bytes:5264 (5.1 Kb)

eth0      Link encap:Ethernet  HWaddr MAC Address
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:422 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:57369 (56.0 Kb)  TX bytes:4922 (4.8 Kb)
          Interrupt:177

eth1      Link encap:Ethernet  HWaddr MAc Address
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:185

• The important thing here is the new interface - br0 - is up and has an IP address and the two sides of the bridge - ETH0 and ETH1 are without IP addresses. the next section discusses testing the configuration. Any problem, check the ifcfg-br0 file and compare the variables used against what is in the ifcfg.template file in the same folder.

• Also, there may be a situation that your two Ethernet adapters are not called ETH0 and ETH1, in which case, the y will be listed in the above commands output.

• I have built this server several times to test this document and once, the server lost it's routing table and lookup information. Change this back using Yast from the command line (yast actual runs when entered as lower case).

Testing

Obviously you are this far which means the OS works and you can ping or SSH to the bridged address. To test the bridge, and making sure you have not activated the firewall, earlier, plug a PC into one of the Ethernet ports on the server using a crossover cable and have the other port of the server connected to the network. You should be able to browse the network and access other services as if the server just built was not there.

First check the firewall is not up, it will restrict all traffic over the bridge, but not to the bridge, so the first sign will be if you can ping the bridge address, from a valid IP address on the same subnet through either Ethernet port. Disable the firewall at the console using Yast. Look under the Security tab, disable and confirm it is. Disable for now and set to manual, we will re-enable later.

Installing Webmin

Webmin is a cross-platform web based administration utility. By default, it can manage almost anything and includes modules for this. However, if a corresponding application is not installed, the module will produce an error when accessed. You will install the full utility, then remove unneeded modules if required.

Acquiring Webmin and Prerequisites

Webmin is a package on it's own, but also requires SSL for secure authentication. This is delivered via Perl scripting and this requires the SSL module as such:

1. Open a PuTTY or any SSH session and login as root.
2. Install Perl-SSL
   • yast -i perl-Net_SSLeay
     • Case is important and you will be prompted for the install media.
3. No errors indicate the component is installed and this piece of work is complete.

Installing application

The best way to install Webmin is from the install binaries rather than pre compiled packages. In this case, this is not very difficult.

1. Open a PuTTY or any SSH session and login as root.
2. Place yourself at the /root home so that all binaries are downloaded there:
   • cd
3. Download the installer from the command line:
   • wget http://kent.dl.sourceforge.net/sourceforge/webadmin/webmin-1.370.tar.gz

Setup

1. Extract the installer:
   • tar -xvzf webmin-1.370.tar.gz
2. Enter the installer folder:
   • cd webmin-1.370
3. Run the installer:
   • ./setup.sh
     • Use defaults for everything except the admin user name and password for which I have used root.
     • When asked to use SSL, say yes.
     • When complete, you will be given the new URL, which can be accessed and logged into.
4. If successful, no errors will be displayed.

If you cannot login, try the IPAddress, but I am sure you already thought that since this project requires a static IP address, you have already created a DNS record in your DNS server?

Testing

At the end of the installation, all things going to plan, you will be able to login to the URL given with the password used above.

Installing Squid

Fortunately SuSE ships with a tested and stable Squid install on the media, have the media handy for this.

Install application

1. Open a PuTTY or any SSH session and login as root.
2. Add the binary (install the application):
   • yast -i squid
     • You will be prompted for the install media.
3. No errors means the application is installed and ready for configuration.

Setup

Before we get to the GUI though, we have made a special volume just for caching and this needs to be set-up with permissions for the squid user.

1. Open a PuTTY or any SSH session and login as root.
2. Check ownership of the cache volume:
   • cd /
   • ls -al
     • This lists all the folders in the root of the filesystem, you should see that /cache has ownership of root:root:
     • drwxr-xr-x 3 root root 4096 2007-10-09 14:22 cache
3. Change the ownership of the cache volume which is mounted as /cache to squid user and root group objects:
   • chown squid:root /cache
     • Confirm with another ls -al and see the ownership has changed:
     • drwxr-xr-x 3 squid root 4096 2007-10-09 14:22 cache
4. Change the permissions to allow all reads and writes to the cache volume:
   • chmod 777 /cache/

Now we use Webmin to set-up the caching server instead of playing around in configuration scripts from the command line editor and risking failure due to a minor typo.

1. Open a web browser and login to Webmin as above.
2. Expand the Servers tab on the left and select Squid Proxy Server from the menu, if you looked before, it is different now.
3. First thing is to change the cache location and initialise it:
   • Open the Cache Options screen.
   • On the first radio button 'Default' change that to 'Listed...'
   • In the Directory dialogue, enter '/cache' leave the file type as 'UFS' change the size to 4096 and directories to 64 and 256 respectively,
   • Click the 'Save' hyper link at the bottom of the screen to be returned to Squid Proxy Server page.
4. On the Squid Proxy Server page, click the button at the top which initializes the cache.
   • For the curious, there will now be 65 folders in the /cache volume. 64 for squid and a 65th for 'lost+found'.
5. Then, as we use local names not FQDNs for our intranets and local web services, we need Squid to append our private domain name space to all single names:
   • Click on Miscellaneous Options and look for 'Default Domain' setting and,
   • Change this setting to '.your.privatedomain.com' not including the quotes (must include leading dot (.))and,
   • Click 'Save' at the bottom of the screen.

Not ready yet. Hold on and keep all appendages in the ride at all times.

Squid will trust nobody by default. For now, we want to run Squid for a few days to see what is going through, so we will open it wide up and let everybody go on as business as usual.

1. While in Webmin and the Squid Proxy Server page, open the Access Control page.
2. On the left, create ACLs for 'Client Address' for each of your IP networks in use, eg 192.168.0.0/16 and,
3. On the right, select 'Add proxy restriction' and add 'Allow' rules for each component then,
4. At the bottom of the screen, click the Return to squid index hyper link and finally,
5. At the bottom of the Squid Proxy Server page, click the Apply Configuration button to have Squid restart with the new settings.

Now the Squid server will allow caching services to any address that starts with either 192.168... or whatever you put...

Rules are from top down, so the new Allow rule should be second last and followed by a Deny All rule to block out any malicious stuff. If this does not allow you to login, or use teh squid service, change the Deny All to Allow All to see if ACL is hurting you.

Finally, Squid must start at boot:

1. Open a PuTTY or any SSH session and login as root.
2. At the console, use chkconfig to add the Squid service to runlevel 3 and 5:
   • chkconfig -a squid

you should now be able to reboot the server and have Squid run properly. Use dmesg to check for startup errors and look at the contents of /var/log/messages for other errors. If you want to check for lines in the very long log file that have the word 'squid' in them, as an example:

• grep squid /var/log/messages

Testing

Once restarted, check this works by setting your browser to have a proxy setting of servername:3128.

If this fails, try the IP address instead of the server name, check the firewall is down, check Squid ACLs are correct.

Installing SARG

SARG is the Squid Analysis Report Generator and will produce reports on utilisation and performance which will help tighten up security and fine tune Squid (and smack hands of users who goof off all day).

Unfortuantely, SARG is not included on the SuSE media and needs to be downloaded and installed manually. Follow these steps:

Download and Install

1. Open a PuTTY or any SSH session and login as root.
2. Change to the root home folder so that we can keep the binaries in a constant place.
   • cd /root
3. Download the binary:
   • wget http://sarg.sourceforge.net/sarg-2.0.7-vhs.i586.rpm
4. Resolve the required dependancies:
   • Sarg requires some other software as documented on the SARG homepage, here, we install these:
     • yast -i apache2
     • yast -i gd
     • Be aware you may be prompted for install media.
5. Install SARG:
   • rpm -Uvh sarg-2.0.7-vhs.i586.rpm
   • You should see a little progress bar and no errors.

Configure and Run

1. Open Webmin as above and login.
2. On the left menu, look for Servers and expand.
3. On the left menu, look for Squid Analysis Report Generator and open
4. You will have an error to the effect that sarg.conf cannot be found.
5. Click the Module Configuration hyper link and change the location to '/etc/squid/sarg/sarg.conf' without quotes, exact case.
6. Click Return to Index and you will have a report generator installed and ready for use.
   • There may be a delay between access and reporting, beware this delay may be a few minutes.
   • Feel free to experiment with reporting options by surfing for a few hours and then returning to reality and checking that SARG is working as advertised.

Testing

Set your browser to use the new proxy service for a few minutes and open the SARG module. Click 'Generate Report Now' and follow the prompts to view the report. You will have some content listed by date and there are some icons for various views, bar graph, list and report etc.

Installing Webalizer

Is like SARG, but produces various other reports around security and access control to Squid.

Installing Application

1. Open a PuTTY or any SSH session and login as root.
2. Install the binaries form the SuSE media:
   • yast -i webalizer
     • You will be prompted for media.

Setup

Webalizer requires no set-up, just use the application from the module plug-in at the Webmin console.

Testing

After some usage, Squid and Linux will have generated some useful statistics, run a Webalizer report then and compare with the quality of information from SARG. Use whichever works for you, I installed both to show my manager how flexible an open source lifestyle can be.

Bandwidth Monitoring

This appliance will be installed, transparently, then after a few days, the bandwidth will be checked and that information used to set-up the firewall, below.

To set-up bandwidth monitoring:

1. Open a web browser and login to Webmin as above.
2. Expand the Networking tab on the left and select Bandwidth Monitoring.
3. As there is only one interface, now, 'br0' select to set-up the monitoring on this interface for now.
4. As monitoring is done with log files that can grow and grow, add the files to the log rotation schedule:
   • Expand 'System' on the left menu and,
   • Click on 'Log File Rotation' and,
   • Add the folder '/etc/webmin/bandwidth/hours' and,
   • Apply a suitable schedule that is proportional to the amount of disk you have.

Check this every day to see how big the logs get and what your estimate of disk space will be. In Linux-land, if the disk fills up, bad things start to happen.

Also, The above may not work, I got it to 'sort-of' work, sop perhaps this little piece can be edited by a much smarter person than I.

Monitor the used bandwidth and ports frequently, then return to set-up the firewall below:

Firewall

As this unit will be transparent, for now, we will allow all traffic through (also this ensure we are not disconnected during set-up). By default, IPtables is included in SuSE, so only the set-up between Webmin and SuSE need to be applied:

Installation and Setup

1. Open a web browser and login to Webmin as above.
2. Expand the Networking tab on the left and select Linux Firewall.
3. As this is the first set-up and as we have manually stopped the firewall during installation (above), you will be presented with several options, select 'Allow all traffic" on 'br0' and 'Enable at boot time' before clicking 'Setup Firewall'.
   • A default set-up is then presented with no rules, and allow everything kind of set-up.
4. Click 'Apply Configuration'.

In this scenario, the Linux appliance will run for a few days to collect traffic information and then a firewall will be built around the findings of that report, so skip this next bit till you are conffident you will not block something critical and don't set-up firewalls remotely from home, late at night after a few beers, this is asking for trouble.

Traffic Redirection for web based protocols

To negate the need to change any client settings, Squid can instruct the firewall to dynamically redirect passing requests to remote port 80 to the servers port 3128 and thereby supply a proxy service.

1. Open a web browser and login to Webmin as above.
2. Expand the Servers tab on the left and select Squid and,
3. Open the 'Port Redirection' applet and,
4. Select 'Port Redirection is enabled, for clients on network' and '192.168.0.0/16' (or whatever your network is) and 'Apply firewall...' and click 'Save'.
5. When returned to the Squid Proxy Server page, click 'Apply Configuration'.
   • Squid is reloaded with the new settings.
   • For interest, go to the Linux firewall module and change the view to NAT. There is now a new redirection rule.
   • This rule only covers the 192.168.0.0 network element or whatever was entered above, if you have other networks, open the rule and at the bottom of the screen, click 'Clone Rule'. A new screen opens with all the same settings as before. Change the network setting to include your other networks or remote sites. Click okay.

Almost done, as all pasing packets will be addressed to port 80, which is usually web based services, a neat little change needs to be made to Squid configuration file from the command line to prevent Squid looping on any local web based traffic:

1. Open a PuTTY or any SSH session and login as root.
2. Open an editor, either at the GUI or the console and edit the /etc/squid/squid.conf file:
   • Look for the following keys and replace them accordingly:
     
     httpd_accel_host virtual
     httpd_accel_port 80
     httpd_accel_with_proxy on
     httpd_accel_uses_host_header on
     
     
   • These keys already exist, so must be replaced accordingly.
3. Restart the service by using the 'Apply Configuration' button in Webmin or from the command line:
   • /etc/init.d/squid restart

Now, any packets that are passing by and are addressed to another server's port 80 will be redirected to port 3128, Squids port and serviced accordingly. There is no way for a user to bypass this and for this reason, all web requests will be cached. Usually there will be restrictions in place to prevent access, I strongly advise that the service is built with full access, unlimited, and then reviewed. This will allow for changes to the system with supported evidence of abuse or security flaws and will also allow for the system to be installed with few as possible side-effects as a result of the change.

After a week, review usage and review this next section, and adjust accordingly.

Until then, skip to testing, a little further down this page.

Basic rules

To be sure this will work, before we create a 'Deny all' rule, we need to be sure the basic will make it into the server or we will disconnect ourselves.

To be sure, because this server will sit between the ADSL router and the network, allowance has to be made for the following protocols:

Forward:

These are requests that can travel through the firewall (as if it was not there).

DNS Lookups (port 53 UDP) Email (port 25 TCP) HTTP (port 80 TCP) HTTPS (port 443 TCP) ICA (port ?) ICA Web (port 8080 TCP) NTP (port 123 UDP) FTP (ports 20 and 21 TCP)

Outbound:

These are packets that originate from the firewall, (as if it was there).

HTTP (port 80 TCP) DNS (port 53 UDP)

Inbound:

These are packets the firewall would receive, (as if it were there).

HTTP (port 80 TCP) - These are the redirected packets and a rule can be made for packets that are directed at the device and packets that are directed to other sites and need to be redirected). Squid (port 3128 TCP) Webmin (port 10000 TCP) SSH (port 22 TCP)

Create entries for each, experiementing as you go along using the Linux Firewall module in Webmin. Maybe start with the Webmin Rule and SSH rule so that you can reconnect and change something without sitting in a freezing server room. Hopefully somebody else with more time will change this section into a step-by-step, but I found with basic firewall and TCPIP knowledge, the Linux Firewall module in Webmin was self-explanatory.

Testing Functionality

Without firewall restrictions, any packet that passes through the bridge and addressed to port 80 TCP should be picked up and redirected to port 3128 TCP (Squid caching server).

To test this, connect a notebook to the logical network and make sure all functionality exists apart from public Internet browsing. You may need to remove all proxy settings from the notebook. That means, any type of Internet activity will result in server not found or unavailable etc.

Then, set-up the notebook with crossover cable in one port (any port) of the Squid server and have the other port connected to the logical network.

The notebook should get a DHCP address and have all the functionality of being connected directly to the logical network, say, in a switch port. However, the notebook should now be able to browse the web without changes to the proxy settings and those browser activities should be reflected on SARG.

If this is case, the transparent proxy is working as designed and after a week of activity, the server's firewall can be hardened using the firewall settings above.

Installation

This device is designed to be installed between the Internet routing service and the logical network. That is, one of the Ethernet ports needs to go directly to the Internet router and the other joins the local network. Of course, it can seperate any service that runs as HTTP and the clients, even reverse caching or Intranet caching etc.

Considerations

Try to remember that any changes to this device will affect traffic that requires to traverse the device, not traffic that is directed elsewhere. The way switches work is that they direct traffic to ports that have MAC addresses that correlate to IP addresses of devices that are connected. This means, any traffic that requires connectivity to any device that is on the opposite side of the server will traverse the bridge, so placement is important.

Also, remember, when closing up the firewall and making the device more secure, try not to lock yourself out. An easy way to get back in is to open the firewall again, using the menus in Yast at the server console. Also, try to remember that the server will be as secure as it's placement, so hardening the passwords and SSH and possibly removing the ability for root to login from anywhere except the console is a good idea. As this is an appliance, it does not require user names, so it may be an idea to use long and complicated user names as well as strong passwords. Normally, this appliance is set-up, fine tuned and then not addressed except to get reports out.

Troubleshooting

Can't connect - check Firewall. Can't resolve names - check firewall and /etc/resolv.conf - whic is managed through Yast. No Internet - Check path to ADSL and check ADSL without Squid. Can the server get to the Internet from the GUI? I made a small hole in our PIX to allow this server only, unlimited web access.

Absolutely last resort, plug the ADSL router directly into the network. As Squid is transparent and both Ethernet ports are on the logical network, there is no harm or changes required to remove the server and keep the end users happy till the problem can be resolved.

Backup and Restore

Linux backup and restore are simply scripts that collate data and send them some place, tape stream, filer or network location with some fiddling with scripts and dependencies etc. Given the build time and the assumption this document will be updated, it will be easier and quicker to build the server from scratch using this document than to try to restore. Make changes with care and be sure to document all alterations (edit this document directly with specifics).

As this server is an HP server and SuSE is supported under Insight Manager, use HPSIM to monitor the hardware.

Hope this helps get a Linux box into your Enterprise, there are no equivalent Windows based services that cost as little as this appliance and you can build it on little bits of hardware, so there is likley to be a server in your computer room that is not being used for anything or is ready to go to the tip.

Please don't email your support requests, look at some forums, try and look at similar instructions for other builds or distributions and get involved and help perpetuate open source software or risk the big monopolies implanting chips in your soul. Then, when you find the answer, edit this document and make the information free, so you can get free information elsewhere.

Moving Forward

I thought it would be great to have this server also scan email and files for viruses and other nasties and remove our need to have ISA or WebMarshal or MailMarshal.

Bringing Open source to Your Business

If you are in the IT industry and this is your first Linux server and you are trying to get buy-in from managment to allow more Linux into your environment, then try to start simple. Business is run by managers who went to business school, so don't ask them how and why, create a business case that proves your ideas and initiatives are worth the time taken to try them out.

Microsoft Word has a great Business Case template for just this thing;-)