When launching a web application, you want to ensure that it will be able to handle the load of users accessing the site. We can get an idea of how it will perform by using a benchmarking tool. While there are many popular tools such as ab (Apache HTTP server benchmarking tool) used by developers today, I want to introduce you to Siege.
Siege is an HTTP load testing and benchmarking utility. It has a similar interface to ab, which will make transitioning to the tool almost seamless. Also, instead of testing against a single URL, Siege allows you to test against multiple. This allows for a more real-world simulation of how a user would use your system.
If you are running Mac OS X, you can install Siege via homebrew by running the following:
brew install siege
To benchmark URLs in Siege, set the
-b option. This will run your tests with no delay for throughput benchmarking. Without the
-b benchmark option, Siege adds a one second delay between requests for each simulated user.
You can have this benchmark run forever, or you can specify a specific period of time. To set the duration of the test, use the
-t NUMm option. The NUMm format is a representation of time. Here are some time format examples:
- -t60S (60 seconds)
- -t1H (1 hour)
- -t120M (120 minutes)
To demonstrate running a benchmark, let's run Siege against this blog:
siege -b -t60S http://blog.remarkablelabs.com
Running the above command will output the following performance statistics:
** SIEGE 2.72 ** Preparing 15 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 4066 hits Availability: 100.00 % Elapsed time: 60.00 secs Data transferred: 13.73 MB Response time: 0.22 secs Transaction rate: 67.77 trans/sec Throughput: 0.23 MB/sec Concurrency: 14.95 Successful transactions: 4066 Failed transactions: 0 Longest transaction: 3.28
Based on the results, the server was always available and had an average response time of 230ms. The average amount of requests per second was 67.77.
Combined with a tool like New Relic, Siege's load testing features will give you insight into your application's weak spots. When creating a load test, you should set the
-c NUM options. The
-d NUM option sets a random interval between 0 and NUM that each "user" will sleep for between requests. While the
-c NUM option sets the concurrent number of simulated users.
Here is an example of a load test of a Heroku application:
$ siege -c50 -d10 -t3M http://some.application.com ** SIEGE 2.72 ** Preparing 50 concurrent users for battle. The server is now under siege...[alert] socket: 100081664 select timed out: Operation timed out Lifting the server siege... done. Transactions: 1398 hits Availability: 99.93 % Elapsed time: 179.94 secs Data transferred: 4.70 MB Response time: 1.21 secs Transaction rate: 7.77 trans/sec Throughput: 0.03 MB/sec Concurrency: 9.40 Successful transactions: 1398 Failed transactions: 1 Longest transaction: 29.47 Shortest transaction: 0.10
During that test, 50 concurrent connections bombarding the server resulted in an average response time of 1.21 requests per seconds and a single failed transaction. The Longest transaction statistic was 29.47, which allows me to assume the failure was due to a Heroku timeout.
To really simulate an experience a user may take, you can provide a file of URLs separated by line to the
-f option. Another recommendation is to set the
-i option that tells Siege to randomly select URLs from the text file, resulting in a more real-world test.
siege -c50 -d10 -i -f site.txt