Load-balancing Tomcat with Apache

February 2008

Discussion

Introduction

Tomcat is a popular application server used to host web applications. Apache is a popular web server which provides services like https encryption and decryption, URL rewriting etc. Apache can also be used a load balancer to balance load between several Tomcat application servers.

This article briefly discusses some alternatives for load balancing an application server. It discusses implementation details for setting up load balancing with Apache using ‘mod_proxy’ module. It also looks at some of the features provided by apache such as ‘server affinity’ and safe removal of node.

Downloadable web application is provided which can be used to test load balancing with Apache. Jmeter script is also provided for load testing Apache.

Setting up Apache and load balancing is the role of the System Administrator. Unless a java developer works in a small team or is setting up a test environment, he wont get involved in setting up load balancing. However, it is good to understand the principles behind load balancing. The knowledge might help a developer to fix issues in live environment.

This article discusses load balancing in the context of a web application. The application is accessed using https, requires a user to login and some user specific information is stored in session.

Background knowledge

It is assumed that the reader is familiar with the following concepts and technologies:

• Apache (Version 2.2)
• Tomcat (version 5.5.23)
• Jmeter for load testing web applications

Some important terms

Load balancing – user requests are processed by more than one server with all servers sharing the load ‘equally’
Server affinity (sticky session) – With server affinity, multiple requests from a user are processed by the same server. This is required for non clustered servers as the user session data is held on one server only and all requests from that user have to go to the server which has the session data for the user.
Transparent failover – User is not aware of a server crash. Transparent failover can be request level or session level. Transparent failover can be achieved by clustering application servers. With clustering, all the servers are the same and so the loss of a server does not interrupt the service. With load balancing alone, the user has to login again when server crashes.
Server Cluster – A group of servers which appear to be a single server to the user. Loss of a server is transparent to the user. User data is held on all servers in the cluster group. Any server can process any user request and loss of any server does not lead to interruption of service. The user does not have to login after a server failure. Since the user session data is replicated over the network to more than one server, there is a performance overhead. So clustering should be avoided unless transparent failover is required.
Scalability – measure of the ability of a system to handle increasing load without reducing response time
Response time – time taken to process a user request
Real workers – Term used by apache to refer to servers which are components of a load balanced system. The real workers do the actual work and the real worker is usually a remote host.
Virtual worker – In Apache, the load balancer is referred to as the virtual worker which delegates processing to real workers.

Load balancing algorithms

• Round robin – requests are Reduced likelihood of version conflicts
• Weighted round robin – servers of different capacity are assigned requests in proportion to their capacity (as defined by a load factor). Apache as 2 versions of this:
• Request counting algorithm – requests are delegated in round robin manner irrespective of the nature of the request
• Weighted traffic counting algorithm – Apache delegates traffic to real worker based on the number of bytes in the request

Compare Load balancing with Clustering

Both Load balancing and clustering aim to improve scalability by spreading load over more than one server. They both aim to provide horizontal scalability.

Load balancing	Clustering
User has to login after server crash	User does not have to login after server crash. So failover is transparent to the user
Load balancing is done by the web server or using DNS or using hardware load balancer or using Tomcat balancer web application	Clustering capability is provided by the application server. Clustering also requires load balancing
The application servers (e.g. Tomcat) do not communicate with each other	The application servers (e.g. Tomcat) communicate with each other.
There is minimal effect on response time when moving up from a single server to load balanced servers under the same load	Response times could deteriorate when moving to a clustered system from a single server as the session data is now replicated over the network to other servers. More the session data, more is the deterioration in performance compared to a single server. Response time also depends on number of nodes in the cluster. More the number of nodes, more is the deterioration in performance as data is replicated using TCP to every single node in cluster. This can be reduced by using UDP to replicate session data. With UDP, session data could be lost during session replication. So a user might have to login again if a server crashes
Load balancing can be used independently of clustering	Clustering also requires load balancing but makes ‘server affinity’ redundant. It provides ‘transparent failover’ capability over load balancing at cost of decreased response time and more complex configuration
Usually no changes are required to move an application from a single server to a load balanced set of servers	Application must meet certain criteria for it to work in a clustered environment. The user variables stored in the session must be ‘serializable’. To get good response time, only small objects must be stored in the session.

Choices for implementing Load balancing

Hardware based load balancing

• Pros
  • Fast
• Cons
• Expensive
• Proprietary
• Less flexible

Software based load balancing (e.g. Apache or Tomcat balancer)

• Pros
  • Open source and free to implement with Apache and Tomcat balancer application
  • Easy to configure
  • More flexib
• Cons
  • Lower performance compared to hardware based solution

Alternatives for software based load balancing

• Apache ‘mod_proxy’ module or ‘mod_jk’ module. However ‘mod_proxy’ is easier to configure and newer than ‘mod-jk’ module.
• Using Tomcat balancer application
• Linux virtual server
• Using DNS for load balancing

The rest of this article discusses load balancing using Apache ‘mod_proxy’ module.

Load balancing with server affinity

A simple load balanced setup which does not provide ‘server affinity’ is not suitable for stateful web applications. In stateful web applications, user state (session data) is held on one server. All further requests from that user must be processed by the same server. Hence server affinity (sticky sessions) is necessary for stateful web applications which don’t use clustering. The minimum load balanced setup is with 2 application servers and one web server (load balancer). If https decryption is required, then the same Apache server can also be used for https decryption.In a load balanced system with server affinity, all requests from user 1 go to Tomcat instance 1. This is shown below:

Apache implements server affinity by rewriting the ‘jsessionid’ sent by Tomcat to the browser. The Tomcat worker name is added to the end of ‘jsessionid’ before the ‘jsessionid’ is sent to the browser. In the next request from the same user, the Tomcat worker name is read from the ‘jsessionid’ and the request delegated to this Tomcat real worker.The ‘jsessionid’ stored as a cookie in the browser has the tomcat worker name as shown in screenshot below:

The Apache configuration (in httpd.conf) to setup Apache as a load balancer for 2 application servers is shown below:

ProxyPass /apache-load-balancing-1.0 balancer://mycluster stickysession=JSESSIONID


 BalancerMember ajp://tomcat1:8009/apache-load-balancing-1.0 route=tomcat1 loadfactor=50
 BalancerMember ajp://tomcat2:8009/apache-load-balancing-1.0 route=tomcat2 loadfactor=50

The above setup requires ‘mod_proxy’ module to be loaded.The first line sets up a reverse proxy for request ‘/apache-load-balancing-1.0’ and delegates all requests to load balancer (virtual worker) with name ‘mycluster’. Sticky sessions are enabled and implemented using cookie ‘JSESSIONID’.The ‘Proxy’ section lists all the servers (real workers) which the load balancer can use. For each participant in load balancing, we define the url (using http, ftp or ajp protocol) and give it a name which matches the name of the Tomcat engine defined in the Tomcat ‘server.xml’. The ‘loadfactor’ can be a number between 1 and 100. Set it to 50 so it can be increased or decreased dynamically later on.The tomcat ‘server.xml’ in ‘TOMCAT_FOLDER/conf’ folder should have this configuration:

....

...

With 2 browser sessions, the Tomcat server console and the server response show that all requests from the same user are processed by the same server.

Load balancing manager and safe removal of server node

Apache load balancing manager Apache includes a ‘balancer manager’ which can be used to check the status of the servers used for load balancing and to prepare for safe removal of a node. To enable balancer manager, add this section to Apache ‘httpd.conf’ file:

 SetHandler balancer-manager

Balancer manager requires ‘mod_proxy’ and ‘mod_proxy_balancer’ modules to be loaded.

Balancer manager will now be accessible at the url ‘/balancer-manager’.

Screenshot of this is shown below:

Safe removal of a server node When an application server (real worker) has to be taken offline for maintenance, set a very low load factor for the server due to be taken offline. Few new users will be served by that server. All existing users ‘sticking’ to this real worker will continue to be processed by this real worker. This will help reduce the number of users who will have to login again when the real worker is taken offline.After some time, the real worker is disabled and taken offline. Once it is ready to come online, the worker is enabled again in balancer manager.

Tests

About the attached web application and JMeter scriptThe attached web application contains a single servlet and 3 JSP pages. The servlet has hard coded usernames and passwords for 10 users and is used to authenticate the user and store the username in the session. The servlet also prints the username and server name of server. The login JSP (index.jsp) is used to login and the second JSP (greeting.jsp) is used to print a greeting after the user logs in. This JSP also prints the username and server name in the response. The third JSP (user_details.jsp) is used to print user details and the server name and IP address of the server name used to process the request. This JSP also prints the username and server name of the server to the console. The name of the server is setup as a context parameter in the web.xml. Change it before deploying it so that the 2 web applications have different names to make it easier to identify them

 
  serverName
  Tomcat instance 1
 

The attached JMeter script sets up 10 threads with each thread being used to login 1 user and request user details. Login is done once per thread/user and all subsequent requests from the user are requests to get user details.Manual test with 2 browser windows to show server affinityOpen 2 browser windows and login using different usernames. The server consoles in Tomcat will show that requests from one user will always go to a particular Tomcat instance demonstrating server affinity. See screenshot below:

Load testing with JMeterThe attached JMeter script is used to simulate a load of 10 users. The attached JMeter script has been setup to record the response. Comparing the response from the server and the server console, we can see that the load of 10 users is shared equally between 2 servers and all requests from one user are processed by the same server.

Sudden loss of one server and then restoring the serverThis can be simulated with either shutting down tomcat to simulate a server crash. This is best simulated using JMeter as a client to simulate a load of 10 users continuously requesting pages from the servers.Run the attached JMeter script and once the load test is running, take one server down. Any subsequent requests to the offline server will be redirected to the second server. When one server goes down, the user session data is lost and so all the users who have ‘affinity’ to that server will have to login again.

Reducing single point of failure

Load balancer can become the single point of failure. This can be reduced by using round robin DNS to delegate user requests to more than one load balancer. The load balancer delegates requests to more than one application server. In this scenario, if the load balancer and/or the application server goes down, the other load balancer and application servers can still provide some level of service. This is illustrated in the diagram below

:

Conclusion

Apache can be used to load balance Tomcat servers. This setup provides other useful features such as ‘Server affinity’ and safe removal of nodes for scheduled maintenance. Load balancing is recommended if transparent failover is not required. It is easy to setup load balancing and ‘server affinity’ with Apache.JMeter can be used to load test the configuration and to test the behaviour in case of a server crash.The load balancer can become the single point of failure. This can be reduced by using 2 load balancer and using round robin DNS to delegate request to more than one server.

Source Files

web application.zip
apache load balance load test script.jmx

Biography

Avneet Mangat 6 years experience in Java/J2EE. Currently working as Lead developer at Active Health Partners ( www.ahp.co.uk ). Bachelors degree in Software Engineering, Sun Certified Web developer and Java programmer, Adobe certified Flash Designer and Prince2 certified (foundation). Lead developer of open source tool DBBrowser, please see http://databasebrowser.sourceforge.net/ Outside interests include photography and travelling. Please contact me at avneet.mangat@ahp.co.uk or avneet.mangat@gmail.com for more information.