Yahoo Developer Network has featured a article on speeding up your website by following 13 simple rules. Seems to be very simple and effective.

   1. Make Fewer HTTP Requests
   2. Use a Content Delivery Network
   3. Add an Expires Header
   4. Gzip Components
   5. Put CSS at the Top
   6. Move Scripts to the Bottom
   7. Avoid CSS Expressions
   8. Make JavaScript and CSS External
   9. Reduce DNS Lookups
  10. Minify JavaScript
  11. Avoid Redirects
  12. Remove Duplicate Scripts
  13. Configure ETags

Read the full article here.