[OxLUG] Tomcat DoS crashing Amazon EC2 VM

Top Page
Attachments:
Message as email
+ (text/plain)
Delete this message
Reply to this message
Author: Tim Pizey
Date:  
To: oxlug
Subject: [OxLUG] Tomcat DoS crashing Amazon EC2 VM
Hi,

I have been having fun beating the bounds, or limits checking, our current project.

I grabbed some old code written in 2000 and re-purposed it to
steadily increase the hit rate on a URL, recording the response time, until a limit was hit.
http://code.google.com/p/dsn-chassis/source/browse/trunk/manta/src/test/java/org/cggh/chassis/manta/loadtest/LoadTest.java


This resulted in some pretty graphics:

Straight Apache serving HTML:
https://spreadsheets.google.com/oimg?key=0AibFqy9K5V7OdFZQcVRraDFEWG1kOWRaQjNOVzlQQ3c&oid=2&zx=c4tvsk-kz59qc

A complex Orbeon form, with rests between batches of hits:
https://spreadsheets.google.com/oimg?key=0AibFqy9K5V7OdG1yTTc0UElTQW5qVE12T29MSjJTaWc&oid=2&zx=2set0q-8env7u
with a reduced rest:
https://spreadsheets.google.com/oimg?key=0AibFqy9K5V7OdENUckpEZm15SXY4RnZCQzUwZXgtSHc&oid=2&zx=530dw3-wpa7pc

With no recovery period, ie with a steadily increasing hit rate:
https://spreadsheets.google.com/oimg?key=0AibFqy9K5V7OdFNsdkFhN3ZpSzJZb1N6Tk40cjlRZFE&oid=2&zx=wi916a-op1jf9
above a hit rate of 10 hits per second the performance regrades rapidly, leading to no response.

Orbeon/tomcat becoming wildly erratic above its working limits:
https://spreadsheets.google.com/oimg?key=0AibFqy9K5V7OdC05MXN2bUc5eHZDd2RIbUtEcmpPbXc&oid=2&zx=pxqb66-2n3qgz


The above are all taken from localhost.

I pointed the code at a 1gig Amazon EC2 instance with a batch size of 50.
That is I was hitting the machine 50 times for each rate from 1 hit per second through to 192 hps.
I crashed the machine at 192 hps.
I mean really crashed, no ssh access.

This is the first time I have set out to discover the limits of a site, and it certainly seems like a better approach than just keeping
my fingers crossed, but having found that a site will not withstand hit rates above n how do I go about ensuring these limits are not breached?

cheers
Tim




--
We are in dialogue.