Tomcat HOWTO

From openSUSE

This document contains information on how to configure and set up apache and tomcat using mod_jk.

Abstract

This document describes a setup of a hello world webapp using apache as a web server serving from a virtual host using mod_deflate to compress the response streams, multiple tomcat instances servicing requests in load balancing fashion with sticky sessions, and mod_jk configured to route requests from apache to tomcat app server. The configuration attempts to make minimal invasive changes to the stock out of the box installation of the OS. The only impact should be init configuration file links in /etc/init.d.

General configuration

Operating environment:

• OpenSuse 10.2

Runtime environment:

• YaST package: apache2 (apache2-2.2.2.3-20) - web server front-end, excellent performance, SSL support, virtual host, redirecting, rewrite rules, on the fly compression, etc.
• YaST package: tomcat5 (tomcat5-5.0.30-53) - supports Servlet Specifications 2.4 and JSP Specifications 2.0.
• remove YaST package: apache2-mod_jk (apache2-mod_jk-4.1.30-13) - this appears to be a weird version of the tomcat connector that is both broken and badly malfunctioning.
• custom module: mod_jk (1.2.20 or later) - download source from Jakarta Tomcat Connectors.

Development environment

• YaST package: apache2-devel (apache2-devel-2.2.3-20) - provides apache run time compilation tool.
• YaST package: java-1_5_0-sun (java-1_5_0-sun-1.5.0_update10-2.1) - Java(TM) 5 Runtime Environment
• YaST package: eclipse (eclipse-3.2.1-24) - Eclipse Platform and Java IDE
• YaST package: eclipse-jdt (eclipse-jdt-3.2.1-24) - Eclipse Java development tools

Networking environment

• alias ethernet network interface as 192.168.1.2
• update /etc/hosts to resolve www.dogverse.com in browser

Production directory hierarchy

- /hosted/admin/install/sysconfig - configuration files for service
- /hosted/admin/install/init.d - service init files
- /hosted/admin/install/build - place to build the mod_jk module
- /hosted/tomcat5/app1 - Tomcat instance files (e.g. server.xml)
- /hosted/tomcat5/app2 - Tomcat instance files (e.g. server.xml)
- /hosted/httpd/jk - JK configuration files for the whole web server (some are global, some are host specific)
- /hosted/httpd/virtual/dogverse.com - configuration files for the virtual host

Software Installation

All packages are installed through standard YaST repositories. The only custom built module is mod_jk. The mod_jk included in opensuse 10.2 is badly broken. It is weird version (4.1.30-13) whereas the current version on Apache's site is 1.2.20. It also has serious lack of support of the current mod_jk feature (e.g. status). Tomcat5 runs on JRE 5.0 and embeds Eclipse JDT compiler for JSP compilation.

• download the mod_jk source into /hosted/admin/download directory.
• extract file

tesla:/hosted/admin/install/build # tar -xzf ../download/tomcat-connectors-1.2.20-src.tar.gz
tesla:/hosted/admin/install/build # cd tomcat-connectors-1.2.20-src/native

• configure, build, and install

tesla:/hosted/admin/install/build/tomcat-connectors-1.2.20-src/native # ./configure --with-apxs=/usr/sbin/apxs2
tesla:/hosted/admin/install/build/tomcat-connectors-1.2.20-src/native # make
tesla:/hosted/admin/install/build/tomcat-connectors-1.2.20-src/native # make install

• check the installation

tesla:/hosted/admin/install/build/tomcat-connectors-1.2.20-src/native # ls -l /usr/lib/apache2/mod_jk.so
-rwxr-xr-x 1 root root 619225 Mar  1 15:11 /usr/lib/apache2/mod_jk.so

Configure Networking

• Configure alias IP 192.168.1.2

- configure interface via NetworkManager (YaST -> Network Devices -> NetworkCard)
- select "ifup" (NetworkManager can't deal with alias IPs)
- Select the card and click on edit
- Advanced -> Additional Addresses
- click Add
- Enter Alias Name: 0
- Enter IP Address: 192.168.1.2
- Enter Netmask: 255.255.255.0
- Click OK, OK, Next, Finish

• edit /etc/hosts and add a line:

192.168.1.2 www.dogverse.com

Configure Apache and mod_jk connector

• create jk common configuration folder hierarchy

/hosted/httpd/jk/conf
/hosted/httpd/jk/logs

• create jk common configuration files

/hosted/httpd/jk/conf/jk.conf

<IfModule mod_jk.c>
 JkWorkersFile /hosted/httpd/jk/conf/workers.properties
 JkLogFile /hosted/httpd/jk/logs/mod_jk.log

 # Log level to be used by mod_jk
 JkLogLevel error
 JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
 JkRequestLogFormat "%w %V %T"
 JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

 JkShmFile /hosted/httpd/jk/logs/jk.shm
</IfModule>

/hosted/httpd/jk/conf/workers.properties

worker.list=router,status

#app1
worker.app1.type=ajp13
worker.app1.host=localhost
worker.app1.port=8109
worker.app1.lbfactor=50
worker.app1.cachesize=10
worker.app1.cache_timeout=600
worker.app1.socket_keepalive=1
worker.app1.recycle_timeout=300
worker.app1.domain=app1

#app2
worker.app2.type=ajp13
worker.app2.host=localhost
worker.app2.port=8209
worker.app2.lbfactor=50
worker.app2.cachesize=10
worker.app2.cache_timeout=600
worker.app2.socket_keepalive=1
worker.app2.recycle_timeout=300
worker.app2.domain=app2

#router
worker.router.type=lb
worker.router.balance_workers=app1,app2
worker.router.sticky_session=1

#status
worker.status.type=status

• create virtual host directory structure

/hosted/httpd/virtual/dogverse.com/conf
/hosted/httpd/virtual/dogverse.com/logs
/hosted/httpd/virtual/dogverse.com/html

• create virtual host configuration

/hosted/httpd/virtual/dogverse.com/conf/httpd.conf

Listen 192.168.1.2:80

<VirtualHost 192.168.1.2:80>
 ServerAdmin webmaster@dogverse.com
 ServerName www.dogverse.com:80

 DocumentRoot /hosted/httpd/virtual/dogverse.com/html

 ErrorLog /hosted/httpd/virtual/dogverse.com/logs/error_log
 TransferLog /hosted/httpd/virtual/dogverse.com/logs/access_log

 <Directory "/hosted/httpd/virtual/dogverse.com/html">
   Options FollowSymLinks
   AllowOverride None
   order allow,deny
   allow from all
 </Directory>

 DefaultType text/html

 <IfModule mod_deflate.c>
   AddOutputFilterByType DEFLATE text/xml text/html text/css
   AddOutputFilterByType DEFLATE application/x-javascript
 </IfModule>

 RewriteEngine on

 RewriteCond %{REQUEST_URI} ^/$ [NC]
 RewriteCond %{QUERY_STRING} ^c=(.*)$ [NC]
 RewriteRule ^/$ /dv/a/home? [R=permanent,L]

 RewriteRule ^/$ /dv/a/home [R,L]

 <IfModule mod_jk.c>
   Include /hosted/httpd/virtual/dogverse.com/conf/jk.conf
 </IfModule>
</VirtualHost>

/hosted/httpd/virtual/dogverse.com/conf/jk.conf

<IfModule mod_jk.c>
 JkMount /dv/* router

 <Location /status>
   JkMount status
   Order deny,allow
   Deny from all
   Allow from 127.0.0.1
   Allow from 192.168.1.2
 </Location>
</IfModule>

• WARNING: you must enter *all* IP addresses your machine ethernet interface holds in the "Allow" section above. Or you can just hack your way through and do "Allow from all" (do not do this if your box services internet traffic).

• instruct apache to load custom configuration and mod_jk modules. edit /etc/sysconfig/apache2 file and append content to the following existing environment variables:

APACHE_CONF_INCLUDE_FILES="/hosted/httpd/virtual/dogverse.com/conf/httpd.conf /hosted/httpd/jk/conf/jk.conf"
APACHE_MODULES="jk rewrite deflate authz_host"

• restart apache2

service apache2 restart

• verify jk is running: http://www.dogverse.com/status and you should get a status image. If you get 403 see WARNING line above. If you get 404 then you pooped somewhere. Double check your spilling.

Tomcat configuration overview

• place two instances into directory /hosted/tomcat5/app1 and /hosted/tomcat5/app2
• instance app1 uses 8109 ajp13 port and instance app2 uses 8209 ajp13 port
• WARNING: server.xml jvmRoute attribute has to match workers.properties .domain property. Other things have to match - host alias (www.dogverse.com), ajp13 port (8109), and make sure you have unique shutdown port (8005).

/hosted/httpd/jk/conf/workers.properties

worker.app1.domain=app1

/hosted/tomcat5/app1/conf/server.xml

<Server port="8105" shutdown="SHUTDOWN">
 <Service name="Catalina">
   <Connector port="8109" protocol="AJP/1.3" enableLookups="false" />

   <Engine name="Catalina" defaultHost="localhost" jvmRoute="app1">
     <Host name="localhost" appBase="webapps" autoDeploy="true">
       <Alias>www.dogverse.com</Alias>
     </Host>
   </Engine>

 </Service>
</Server>

Detailed Tomcat configuration

• directory structure

/hosted/admin/init.d
/hosted/admin/sysconfig

• copy init.d files

tesla:/etc/init.d # cp /etc/init.d/tomcat5 /hosted/admin/init.d/tomcat5-app1
tesla:/etc/init.d # cp /etc/init.d/tomcat5 /hosted/admin/init.d/tomcat5-app2

• edit /hosted/admin/init.d/tomcat5-app1, replace these two lines

# Provides: tomcat5
TOMCAT_CONFIG=/etc/sysconfig/j2ee

with these

# Provides: tomcat5-app1
TOMCAT_CONFIG=/hosted/admin/sysconfig/tomcat5-`basename $0 | sed -e 's/tomcat5-//'`

• edit /hosted/admin/init.d/tomcat5-app2, replace these two lines

# Provides: tomcat5
TOMCAT_CONFIG=/etc/sysconfig/j2ee

with these

# Provides: tomcat5-app2
TOMCAT_CONFIG=/hosted/admin/sysconfig/tomcat5-`basename $0 | sed -e 's/tomcat5-//'`

• create sysconfig files:

/hosted/admin/sysconfig/tomcat5-app1

export JAVA_HOME="/usr/lib/jvm/java-1.5.0-sun-1.5.0_update10/jre"
export INSTANCE_NAME=app1
export INSTANCE_PORT=8109
export CATALINA_HOME=/hosted/tomcat5/$INSTANCE_NAME
export CATALINA_BASE=$CATALINA_HOME
export CATALINA_TMPDIR=$CATALINA_BASE/temp
export CATALINA_LOCK=$CATALINA_BASE/logs/tomcat5-$INSTANCE_NAME.lock
export CATALINA_PID=$CATALINA_BASE/logs/tomcat5-$INSTANCE_NAME.pid
export CATALINA_OPTS=" -Xmx256M"
export JPDA_TRANSPORT="dt_socket"
export JPDA_ADDRESS="8001"
export JSSE_HOME=""
export TOMCAT_USER="tomcat"

/hosted/admin/sysconfig/tomcat5-app2

export JAVA_HOME="/usr/lib/jvm/java-1.5.0-sun-1.5.0_update10/jre"
export INSTANCE_NAME=app2
export INSTANCE_PORT=8209
export CATALINA_HOME=/hosted/tomcat5/$INSTANCE_NAME
export CATALINA_BASE=$CATALINA_HOME
export CATALINA_TMPDIR=$CATALINA_BASE/temp
export CATALINA_LOCK=$CATALINA_BASE/logs/tomcat5-$INSTANCE_NAME.lock
export CATALINA_PID=$CATALINA_BASE/logs/tomcat5-$INSTANCE_NAME.pid
export CATALINA_OPTS=" -Xmx256M"
export JPDA_TRANSPORT="dt_socket"
export JPDA_ADDRESS="8002"
export JSSE_HOME=""
export TOMCAT_USER="tomcat"

• MORON WARNING: make sure you specify unique JPDA_ADDRESS.
• WARNING: the INSTANCE_PORT has to match the server.xml configured ajp13 ports
• WARNING: the INSTANCE_NAME has to match the directory structure /hosted/tomcat5/app1 and /hosted/tomcat5/app2 and also has to match the init.d file name (since we derive instance name based on the file)

• instance directory structures

/hosted/tomcat5/app1/conf
/hosted/tomcat5/app1/webapps
/hosted/tomcat5/app2/conf
/hosted/tomcat5/app2/webapps

• link common directories

tesla:/etc/init.d # cd /hosted/tomcat5/app1
tesla:/hosted/tomcat5/app1 # ln -s /usr/share/tomcat5/bin
tesla:/hosted/tomcat5/app1 # ln -s /usr/share/tomcat5/common
tesla:/hosted/tomcat5/app1 # ln -s /var/log/tomcat5/base logs
tesla:/hosted/tomcat5/app1 # ln -s /usr/share/tomcat5/server
tesla:/hosted/tomcat5/app1 # ln -s /var/cache/tomcat5/base/temp
tesla:/hosted/tomcat5/app1 # ln -s /var/cache/tomcat5/base/work

tesla:/hosted/tomcat5/app1 # cd /hosted/tomcat5/app2
tesla:/hosted/tomcat5/app2 # ln -s /usr/share/tomcat5/bin
tesla:/hosted/tomcat5/app2 # ln -s /usr/share/tomcat5/common
tesla:/hosted/tomcat5/app2 # ln -s /var/log/tomcat5/base logs
tesla:/hosted/tomcat5/app2 # ln -s /usr/share/tomcat5/server
tesla:/hosted/tomcat5/app2 # ln -s /var/cache/tomcat5/base/temp
tesla:/hosted/tomcat5/app2 # ln -s /var/cache/tomcat5/base/work

• copy select config files:

tesla:/hosted/tomcat5/app2 # cd /hosted/tomcat5/app1/conf
tesla:/hosted/tomcat5/app1/conf # cp /srv/www/tomcat5/base/conf/web.xml .
tesla:/hosted/tomcat5/app1/conf # cp /srv/www/tomcat5/base/conf/catalina.properties .
tesla:/hosted/tomcat5/app1/conf # cp /srv/www/tomcat5/base/conf/catalina.policy .

tesla:/hosted/tomcat5/app1/conf # cd /hosted/tomcat5/app2/conf
tesla:/hosted/tomcat5/app2/conf # cp /srv/www/tomcat5/base/conf/web.xml .
tesla:/hosted/tomcat5/app2/conf # cp /srv/www/tomcat5/base/conf/catalina.properties .
tesla:/hosted/tomcat5/app2/conf # cp /srv/www/tomcat5/base/conf/catalina.policy .

• create server.xml

/hosted/tomcat5/app1/conf/server.xml

<Server port="8105" shutdown="SHUTDOWN">

 <Service name="Catalina">
   <Connector port="8109" protocol="AJP/1.3" enableLookups="false" />

   <Engine name="Catalina" defaultHost="localhost" jvmRoute="app1">
     <Host name="localhost" appBase="webapps" autoDeploy="true">
       <Alias>www.dogverse.com</Alias>
     </Host>
   </Engine>

 </Service>
</Server>

/hosted/tomcat5/app2/conf/server.xml

<Server port="8205" shutdown="SHUTDOWN">

 <Service name="Catalina">
   <Connector port="8209" protocol="AJP/1.3" enableLookups="false" />

   <Engine name="Catalina" defaultHost="localhost" jvmRoute="app2">
     <Host name="localhost" appBase="webapps" autoDeploy="true">
       <Alias>www.dogverse.com</Alias>
     </Host>
   </Engine>

 </Service>
</Server>

• link the /hosted/admin/init.d files to /etc/init.d

tesla:/etc/init.d # cd /etc/init.d
tesla:/etc/init.d # ln -s /hosted/admin/init.d/tomcat5-app1
tesla:/etc/init.d # ln -s /hosted/admin/init.d/tomcat5-app2
tesla:/etc/init.d # chkconfig tomcat5-app1 on
tesla:/etc/init.d # chkconfig tomcat5-app2 on

• start service

tesla:/etc/init.d # service tomcat5-app1 start
tesla:/etc/init.d # service tomcat5-app2 start

Deploy webap

• build dogverse webapp using eclipse. Bind a HelloWorld servlet to /a uri and stream back the output "Hello World"
• Package the web app as dv.war
• deploy by dropping to /hosted/tomcat5/app1/webapps and /hosted/tomcat5/app2/webapps. It will autodeploy.

Enjoy the fruits of your labor

• open firefox and point to http://www.dogverse.com and you should be redirected to http://www.dogverse.com/dv/a/home. This should render "Hello World"

Future considerations

There are number of things that can be done:

• add templating technology (clearsilver)
• add database driven capability (mysql)
• add logging service (log4j, java service wrapper)

Templating considerations

The Java's templating technologies are not the brightest knife in the drawer. They bloat memory, create lazy developers who abuse the session state mechanisms, or are hardcoded with their custom state machines (e.g. struts). My favorite memory-light response is clearsilver.

• download, unpack, configure, build, install clearsilver jar and JNI library
• install the .jar file by linking to it from /usr/share/tomcat5/common/lib (the $CATALINA_HOME/common links to /usr/share/tomcat5/common in our instance)
• set LD_LIBRARY_PATH to /usr/lib in the sysconfig java can load the JNI library
• read tutorial on how to use the template files to generate HTML content

Database considerations

• add directories

/hosted/tomcat5/app1/conf/Catalina/localhost
/hosted/tomcat5/app2/conf/Catalina/localhost

• add your app server resources

/hosted/tomcat/app1/conf/Catalina/localhost/dv.xml

<Context path="/dv" reloadable="true" crossContext="true">
 <Resource
   name="jdbc/db"
   auth="Container"
   type="javax.sql.DataSource"
   maxActive="10"
   maxIdle="10"
   maxWait="10000"
   username="dvapp"
   password="**********"
   driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/dvapp?autoReconnect=true" />
</Context>

/hosted/tomcat/app2/conf/Catalina/localhost/dv.xml

<Context path="/dv" reloadable="true" crossContext="true">
 <Resource
   name="jdbc/db"
   auth="Container"
   type="javax.sql.DataSource"
   maxActive="10"
   maxIdle="10"
   maxWait="10000"
   username="dvapp"
   password="**********"
   driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/dvapp?autoReconnect=true" />
</Context>

• remember to create MySql resources as appropriate (e.g. user dvapp, database dvapp)
• To access MySql from java you need mysql java connector
• install the connector .jar file by linking to it from /usr/share/tomcat5/common/lib (the $CATALINA_HOME/common links to /usr/share/tomcat5/common in our instance)

Logging considerations

Multiple tomcat instances have a nasty habit of creating race conditions when writing to the same log4j configured file (which in turn lives in the webapp itself, and I hate creating custom .war files for each deployment just so I can change the logging location). This is why we are thankful for existence of SocketService, java service wrapper, and a human brain.

• download java service wrapper
• configure in /hosted/log4jss
• add /hosted/init.d/log4jss and /hosted/sysconfig/log4jss and appropriate link in /etc/init.d

What a complicated mess. Just use System properties in your log4j.xml or log4j.properties file to make your log file names unique. Every application server has environmental variables to uniquely identify each server instance. This has the added benefit that you don't see log records from servers you are not interested in. I've never had a need to see logging entries from multiple servers in the same log file (all app servers keep user Sessions routed to a single server instance).