I have been doing some benchmarking in the wake of my TileCache installation ‘endeavor’ of last week (part 1 – part 2). In a series – well, probably two – articles, I will try to provide some insight into the performance of the TileCache – Python – Apache ensemble.
Let me lay out first how I went about getting the results below. I used ab on my MacBook Pro to generate the HTTP requests. ab – or ApacheBench – is an extremely easy to use benchmarking tool, and don’t be fooled by the name, it will benchmark your lighttpd / iis whatever-based server just as happily. I sent the requests to my desktop PC at the office, on which I danced the TileCache / mod_python setup dance I described in earlier posts. The connection was from my home through a VPN tunnel to the office, so it’s sort of remote – at least physically. I set ab to run for one minute for each session.
I wrote (ahem) a short shell script to do various ab sessions, with the following variables:
- using CGI or using mod_python
- the number of concurrent requests
The former is relevant because I want to show how much faster the mod_python solution actually is. I could not find any good numbers on this specifically related to TileCache. The latter is relevant because a tile server will typically receive many concurrent requests, even from a single client. This specifically applies when TileCache is employed as a caching layer between OpenLayers and a WMS server and OpenLayers is configured to cheat your server and fetch multiple tiles at once, as described here.
So now on to the results. I’ll show you some graphs which are visualizations of ab’s CSV output. They show a devision of the total number of requests in percentiles by time (in ms) needed to serve that percentile of the requests.
Note that the graphs are logarithmic (10logn). I did that to ensure optimal visibility of the results – the differences can be small for most of the percentile range, but will spike at the end of the graph.
With 5 concurrent requests:
…10 concurrent requests…
…and finally even 100 — which is already quite a heavy load, especially for my poor desktop with a single consumer SATA disk:
I’ll write up my thoughts about these results, as well as ideas for further optimization, in a next post shortly. For now, I leave you to think about this yourselves 😉 and – just because I like Excel so much – feed you two more graphs to summarize the mod_python and CGI results:
Still to come:
- Analysis of the results shown
- Further optimization techniques (ideas welcome – I’m new at this!)
- Check the log files – were all requests actually served?
- Longer running benchmarks, how does TileCache hold up in the long run?