For many of you running CGI Proxy, speed and scalability issues have always reared their ugly head, threatening to slow down your service to a crawl. I have been dealing with these issues running Vtunnel.com since 2005, and have come up with some good ways to reduce the load. The first big set of changes to make were described in our previous article, the Freeproxies guide to performance tuning apache for mod perl and cgi proxy. Today we’re going to take that a step further and introduce to you the benefits of using the open source squid cache in the cgi proxy environment.
First I would like to say that without great inspiration from the book “Scalable internet architectures“, it never would have occurred to me the great performance benefits that could be obtained using squid. Anyone dealing with environments that need to scale beyond a single server would benefit from reading this book.
In order to understand how squid can help you, first you have to understand how the various technologies you are using interact, and where the bottlenecks are.
So lets first describe a cgi proxy transaction without squid.
1) The user requests a page from your server, and apache receives this request. If there is a free apache slot available, it is now allotted to this request. You need a slot for each active request you’d like to process, and each slot uses a significant amount of ram.
2) Apache now will load up cgi proxy into the apache slot if it is not already running, and run that perl script. In the process of running, cgi proxy will most likely need to resolve a domain name, connect to the remote site, download some content, and reprocess it to send to the end user. All of these things take time, during which your expensive apache slot is being held up. This process is also the most cpu intensive of the things that are done during the life of the query.
3) After CGI Proxy in your apache slot has fetched what it was looking for and modified it suitably, it is time to send it to the end user. The apache slot is still being used, and ram wasted, even though very little processing is now going on. This is what we call ’spoon feeding’ the client their data. If network conditions between your server and the end user are poor, the time you spend spoon feeding the client will be significant.
In this case we are assuming we have plenty of bandwidth available, as our solution does not significantly help with that particular issue.
So we can see there are two bottlenecks:
1) The amount of CPU power available to process requests
2) The amount of apache slots you have available (roughly equivalent to total ram)
Now, many people may have thought of putting a squid cache on the ‘backend’, i.e., between your server and the websites it is requesting from. This will only save on incoming bandwidth, and, because the servers you are downloading from are usually pretty fast, the benefits of caching here are minimal. If you are being metered on both incoming and outgoing bandwidth, this may save you a bit of incoming bandwidth, but that’s about it.
However, we are going to take a totally different approach, and put squid on the ‘frontend’, between your clients and the apache server. We can help with both cpu usage and ram usage using squid caching in this way.
First, the obvious. Any request coming in that is cachable by your frontend squid is a request that does not have to be processed by Apache or CGI Proxy at all. This increases speed, reduces cpu usage, and reduces ram usage (since we didn’t need an apache slot). This by itself is probably good enough to consider using squid, but it is not where the most benefit lies.
The biggest performance benefit is gained from the fact that apache now has to talk to squid locally, rather than to your remote clients.
In order to see how this is important, lets go back to our description of a transaction with your proxy:
1-> User Requests Something From Server
Apache picks a slot for the user (if available. if not available, queue the request)
Apache does a handshake negotiation with the remote user
2-> Apache starts CGI Proxy / mod perl if it is not running in the apache slot (this is incredibly cpu intensive)
CGI Proxy figures out what to do with the request
CGI Proxy does a DNS Lookup of the remote server
CGI Proxy connects to the remote server (and does a handshake)
CGI Proxy downloads some stuff from the website
CGI Proxy processes the result (the majority of cpu usage typically happens here)
3-> CGI Proxy spoon feeds the result to the client. The farther away the client is (in ping times) and the slower their internet connection, the longer this will take.
Now, let’s compare that to with using squid:
1-> User requests something from server
Squid does a handshake with the user
Squid checks to see if the item is in the cache already (if it is, skip to step 3)
2-> Squid connects to apache and does a handshake locally (i.e. with zero latency). Chances are good this can be skipped entirely due to keepalives
Squid requests the item from apache
Apache assigns a slot to the request
Apache loads up cgi proxy / mod perl if it is not running in the slot (a cpu intensive thing to do). Note this is less likely to happen than before because squid is more likely to make good use of keepalives than the end user.
CGI Proxy figures out what to do with the request
CGI Proxy does a DNS lookup of the remote server
CGI Proxy connects to the remote server (and does a handshake)
CGI Proxy downloads some stuff from the website
CGI Proxy processes the result (again, the majority of cpu usage happens here)
CGI Proxy blasts the result over to squid. This takes very little time since squid is located on the same server. The apache slot is now free for additional requests.
3-> Squid spoon feeds to result to the client. This can take a long time, but, squid does not significantly degrade its performance or increase its resource usage when additional clients remain connected.
As you can see, with caching, a percentage of the time the most of the work (step 2) can be skipped altogether. Also, even when this is not possible, the amount of time that your apache slots need to be taken up is significantly reduced. Squid, which does not use significantly more ram when more users remain connected, is now responsible for the bulk of the time consuming work, whereas CGI Proxy and mod perl are now responsible primarily for actual processing only.
For those of you wanting a more concrete example, here are equations that relate the performance of your service:
Requests per second = Seconds to process each request * Number of simultaneous requests that can be processed
Number of simultaneous requests that can be processed = Total Ram available / Ram usage per apache slot
Seconds to process each request = (Total CPU Power / CPU time to process each request) + wait time overhead
When you relate these three equations, you can see the primary limiting factors of your system. By implementing squid on the front end, you reduce cpu time per request when the requests are cachable. You also get a small benefit on cpu by cycling your apache connections less often. More importantly, you significantly reduce wait time overhead by having apache talk to squid instead of to a remote client.
All of these factors significantly reduce seconds to process each request, putting less pressure on you to increase the available maximum connections. This is good because the only way to increase the number of simultaneous connections with apache is to get more ram (which costs money), or use less ram per slot (which you’ve already done if you followed our previous article).
Just for completeness, I will mention two other factors affecting overhead wait. One is the quality of the bandwidth at your web host. The other is the speed of your dns resolution. If your bandwidth quality is low, your server will spend a lot of time trying to download pages from the internet, tying up your apache slots. If your DNS resolution is slow, CGI Proxy will sit trying to resolve server addresses for a long time. Both of these things undermine the purpose of the optimizations we’ve just made. It is beyond the scope of this article to discuss improving these areas, but it is something you should keep in mind. It is worth mentioning that the best way to improve dns response times is to set up a local DNS resolver on your server.
Now that you know why you should use squid caching with cgi proxy, the obvious question is, “how do I do it?”.
Because the answer to that question is complicated and has some pitfalls, it will be the topic of our next article here on the freeproxies blog. Stay tuned.