Converting caffeine to code, shooting pics, and cruising the roads of life.

Apr 16th, 2006  Apr 16th, 2006

High Traffic Web Development

Over the past week and a half, I have been in the process of reading up on developing for high traffic sites. It is very interesting that it only requires small optimizations here and there.

One thing I’ve found in common with all large sites and that should be pretty obvious, is a PHP compiler cache and some type of caching system. Some of the more popular ones are APC, Bware Afterburner, Turck MMCache, and the Zend Accelerator. PHP caching systems are easy to write and can speed up template calls a ton.

You can speed up your site by using output buffering. What this does is use writev() instead of write(). The write() calls were sent to apache as 4kb buffers, where as the writev() calls aren’t.

Another speed up can be accomplished by setting Apache’s SendBufferSize to PageSize. This allows the page to be handed over to the kernel, to be sent, without blocking.

To reduce bandwidth, you may want to look into gzipping the contents of your page. I have seen this shrink 80gb backups down by almost a third the size, except for when most of the contents were images. So you should gain a lot for this if you can afford the overhead of the operations to compress your data.

There are also some simple thinks you can do to speed up your code. To speed up database calls, only query for that data you need. No need to SELECT * when you only need to SELECT id. You should also only query a table once. Get the data you need from it and store it, don’t query it again.

If you are using PHP5 or higher, you may also want to use MySQLi. This new MySQL API is much faster than the old API and includes both a functional and OOP interface. Most applications can be easily converted to use it. Plus, you get support for prepared statements and bind statements.

Optimization can also be made during the design of your database. Make sure you realize the differences between MyISAM and the InnoDB storage engines. MyISAM is very efficient for either very high volume writes or reads, but has table level locking. InnoDB has non-locking reads and row level updates, plus high concurrency.

You may also want to cache query results if they are not expected to update as often. An example I often use for this is CMS systems. Why query all of the blocks you want on one page, when you can cache the results of the original query for 6 hours and have them quickly accessible? Plus, if the owner of the site changes them, just clear the cache…

Now, back to what I’ve been up to. Besides reading up on the above, I have also been messing with my new project for the last two weeks straight. It is amazing how many hacks IE5-IE6 requires. I know I’ve used at least four to get my new site to display correctly. While the thing worked in Firefox, Safari, Epiphany, Konqeror, Opera, and IE7 the _whole_ time.

I hope most people will like the design though. I have shown it to around 20 people on IRC, and only one person hasn’t liked it a lot. So I would say I’ve done pretty good considering this was the first site I had designed entirely in Photoshop and then converted it to CSS/XHTML by hand. The core work for this new project starts this week and will probably continue for the next few months to a year.

Reference for high volume PHP: http://www.oreillynet.com/onlamp/blog/2006/04/digg_phps_scalability_and_perf.html

11 Responses to “High Traffic Web Development”

  1. Hubert Roksor says:

    Well, thanks for this nice article but I think you should reconsider your tip about “include_once/require_once”.

    First of all, what files are best included only once? files containing classes and/or functions. So you’d better check for their existence with class_exists()/function_exists() instead of including them blindly.

    Second and most importantly, include_once/require_once are a lot slower than their counterparts (assuming you use an opcode cache). See http://talks.lerdorf.com/show/nz2/16 for a better explanation :)

  2. Cody Mays says:

    Thanks, I’ll update the article. I can see why now. :thumbup_tb:

  3. I see, its quite useful. tips are really useful

  4. Hacks says:

    Great quality stuff.

  5. Levitra. says:

    Levitra….

    Levitra. Levitra…..

  6. Why are electronic cigarettes not available in regular shops?

  7. I have been running an internet company for years now at http://www.greatbusinessreviews.com and I have to compliment you on this good piece you have written. You really know your stuff. Would you like to guest write on a few of my sites? Anyways, good job and keep up the good work.

  8. Geraldine says:

    Good day very cool web site!! Guy .. Excellent .. Amazing .
    . I will bookmark your websitе anԁ take the feeds alsо?
    I’m glad to search out numerous useful info here in the publish, we need work out more techniques on this regard, thank you for sharing. . . . . .

  9. I’ve been surfing online more than 3 hours today, but I by no means discovered any interesting article like yours. It’s lovely value sufficient for me. Personally, if all webmasters and bloggers made just right content as you probably did, the web will probably be a lot more helpful than ever before.

  10. Hauth says:

    You truly allow it to be look so simple with all your presentation however in locating this matter to be seriously an element that I am I will never understand. It kind of feels way too tricky and also broad in my situation. I am looking forward for your following publish, I will try and get the hang on from it!

  11. tiffany dunn says:

    Thank you for sharing excellent informations. Your web-site is very cool. I am impressed by the details that youˇve on this website. It reveals how nicely you perceive this subject. Bookmarked this web page, will come back for extra articles. You, my friend, ROCK! I found simply the info I already searched everywhere and simply couldn’t come across. What a great site.

Leave a Reply