17 floggle toggles died making VB's post...

http://www.youtube.com/watch?v=PtuqjFf7-N4

Hit upon an nginx configuration that I think does it (reverse proxy cache).

This is for the static subdomain londonfgss.com and should only server static files.

nginx.conf
[code]
user www-data www-data;
worker_processes 2;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {

    worker_connections  1024;
    use epoll;

}

http {

    # allow long server names
    server_names_hash_bucket_size 64;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    server_tokens       off;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;

    keepalive_timeout   65;

    include /etc/nginx/conf.d/*.conf;
    include             /etc/nginx/sites-enabled/*;

}
[/code]

conf.d/gzip.conf
[code]

Compression

gzip on;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_proxied any;
gzip_min_length 1100;
gzip_buffers 16 8k;
gzip_types text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript image/gif image/jpeg image/png;

Some version of IE 6 don't handle compression well on some mime-types, so just disable for them

gzip_disable "MSIE [1-6].(?!.*SV1)";

Set a vary header so downstream proxies don't send cached gzipped content to IE6

gzip_vary on;
[/code]

conf.d/proxy.conf
[code]
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
client_header_buffer_size 64k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 16k;
proxy_buffers 32 16k;
proxy_busy_buffers_size 64k;
[/code]

sites-available/default
[code]
server {

    listen       80;
    server_name  static.londonfgss.com;

    proxy_store_access   user:rw  group:rw  all:r;

    location ~* ^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js)$ {
            root /var/www/cache;

            proxy_store $document_root${request_uri};

            if (!-f $document_root${request_uri}) {
                    proxy_pass         http://www.londonfgss.com:80;
            }

            if (-f $document_root${request_uri}) {
                    expires max;
            }
    }

    location / {break;}

}
[/code]

Oh. And to use nginx gzip, I've had to disable mod_deflate and use gzip in php for html. Otherwise we'd be gzip'ing twice.

I've just turned that on btw... so if anyone notices weird effects with avatars, images, stylesheets or jaavscript, let me know.

I can't seem to be able to post in any thread, no matter how hard I try.

Oh, wait...

BTW, we're now #2 in the world of cycling forums:
http://www.alexa.com/topsites/category/Top/Sports/Cycling/Chats_and_Forums

Etags don't work with CDN's. And if you examine the site using Firebug you'll see that caching is pretty rock solid.

The DB is indexed thoroughly and correctly. Only slow indexes are the fulltext search. We're using MyISAM for those tables (posts, threads, pms) and InnoDB everywhere else.

MySQL Query cache is enabled, and common queries have been normalised to use a DAL to ensure that the same query can use the cache. We get about 60% hits on those, it does fine.

PHP5 is op-code compiled using XCache.

All common and heavily re-used fragments of data (such as styles, templates, configuration) are stored in XCache.

XCache is running entirely from RAM.

We now have separate DB and Web servers, and the DB server doesn't break a sweat, it's all on the web server, and it's all about Apache2 with XCache (opcode and key/value pairs).

Oh, and I've unloaded all unncessary and unused PHP modules so that only what is needed it loaded by Apache.

Hmm, sounds like everything's done right. I am not sure how much performance you can actually squeeze out of vbulletin. Do you consider tweaking the board itself? Would be interesting to profile the PHP scripts themselves, and find out where the bottlenecks are.

Edit: Also, if you need some PHP stuff done, give me a shout, and I see what I can do.

Also, did you consider using lighttpd instead of apache2? From what I understand PHP will perform very well on that, but then again I am no sysadmin.

I think some of the brains round here can be measured in yards!!!

What about APC instead of XCache?

http://blog.digitalstruct.com/2007/12/23/php-accelerators-apc-vs-zend-vs-xcache-with-zend-framework/

I used to use APC, I didn't find it performed as well as XCache did.

I find comparison sites very misleading, you need to know the CPU specs, the configuration used, the load testing used, etc... otherwise you're not really making fair comparisons.

One strange thing I've seen on this site is that the prefork version of Apache2 outperforms the workversion. Prefork is single-threaded, and worker is multi-threaded... you'd assume that the multi-threaded version would be better... and you'd be correct for static files (but nginx handles those anyway), but the issue isn't Apache here it's PHP.

PHP is not thread-safe so any server that is multi-threaded cannot actually take advantage of multi-threading and so it goes slower as it wasn't optimised for single-threading.

Of course it's feasible to configure Apache to use CGI, and if you used Lighttpd you'd use a CGI mode for PHP. This does allow greater throughput, but because you're basically spinning up a process to generate a page and then destroying it (a light and easy task) you lose the ability to have a memory resident cache in the process for op-code compiling and memcached-like storage of frequently used values (the site configuration).

So to use an op-code compiler, and to use a memory cache, both of which provide greater throughput than working with CGI in a lightweight process would, I need to have a permanent process available that is optimised for single-threaded use (PHP5). Hence... Apache2 in apache handler mode rather than CGI.

Something I might try is nginx and memcached instead of apache2 and xcache... but I'd want to increase my knowledge of nginx to a very intimate level before I use that as the main work horse.

BTW, we're now #2 in the world of cycling forums:
http://www.alexa.com/topsites/category/Top/Sports/Cycling/Chats_and_Forums

In your face Rickshaw forum!

^^ We should enter some of the posts on here for the Bad Sex awards. ;)

(No offence intended, by the way, Velocio and pascalo et al. I just absolutely adore jargon.)

So is anyone here an expert or power user with nginx, php-fpm, using nginx to load balance, etc?

The issue: 1 server gave us 4,000 users per day... 2 servers has given us 8,000 users per day... however the thing still crashes most days. Adding a server only brought us a bump in numbers, and not more resilience.

So... I've bought another server... saturn.

Saturn is running nginx, APC, memcache, php-fpm... instead of Apache2, XCache and mod_php.

I want to load balance the two (Apache and nginx servers) using nginx, until such a time that I can turn off Apache and just use nginx.

Any skilled volunteers?

Oh, go on then.

I can load balance a Windows box. This is of no help to you whatsoever. And thus continues the saga of my life..

VB, there's some sparky bods on this forum that might have the expertise....

***Technology
***Velocio - web design and admin
steff - Syadmin/IT - Linux/BSD, a little Solaris, Macs, and Windows if required
woffle - web design and development / plsql development and oracle, mysql etc database admin

Obviously you won't be asking yourself for help....that would be silly. ;)

It looks like I might not need help... this thing appears to be working now... I'm posting from Saturn.

Anyhow, for my own notes:
[code]
sudo apt-get install make bison flex gcc patch autoconf subversion locate
sudo apt-get install libxml2-dev libbz2-dev libpcre3-dev libssl-dev zlib1g-dev libmcrypt-dev libmhash-dev libmhash2 libcurl4-openssl-dev libpq-dev libpq5 libsyck0-dev
[/code]

[code]
cd /usr/local/src/
sudo wget http://us.php.net/get/php-5.2.8.tar.gz/from/this/mirror
sudo tar zvxf php-5.2.8.tar.gz
sudo wget http://php-fpm.anight.org/downloads/head/php-5.2.8-fpm-0.5.10.diff.gz
sudo gzip -cd php-5.2.8-fpm-0.5.10.diff.gz | sudo patch -d php-5.2.8 -p1
cd php-5.2.8
sudo apt-get install defoma fontconfig-config libfontconfig1 libfreetype6 libgd2-xpm libjpeg62 libpng12-0 libt1-5 libxpm4 ttf-dejavu ttf-dejavu-core ttf-dejavu-extra ucf libttf-dev libfreetype6-dev freetype1-tools libpng-dev libjpeg-dev
sudo ./configure \
--disable-debug \
--disable-pdo \
--disable-rpath \
--enable-fastcgi \
--enable-fpm \
--enable-gd-native-ttf \
--enable-inline-optimization \
--enable-mbregex \
--enable-mbstring \
--enable-pcntl \
--enable-sockets \
--enable-sysvsem \
--enable-sysvshm \
--enable-zip \
--with-bz2 \
--with-curl \
--with-curlwrappers \
--with-freetype-dir \
--with-gd \
--with-jpeg-dir \
--with-mcrypt \
--with-mhash \
--with-mysqli \
--with-pcre-regex \
--with-pic \
--with-png-dir \
--with-zlib
sudo make all install
sudo strip /usr/local/bin/php-cgi
[/code]

[code]
sudo pecl install memcache
sudo pecl install apc
[/code]

[code]
sudo cp /usr/local/src/php-5.2.8/php.ini-recommended /usr/local/lib/php.ini
sudo mkdir /etc/php/
sudo ln -s /usr/local/lib/php.ini /etc/php/php.ini
sudo ln -s /usr/local/etc/php-fpm.conf /etc/php/php-fpm.conf
[/code]

[code]
cd ..
sudo wget http://sysoev.ru/nginx/nginx-0.6.35.tar.gz
sudo tar zxvf nginx-0.6.35.tar.gz
sudo rm -f nginx-0.6.35.tar.gz
cd nginx-0.6.35
sudo ./configure --sbin-path=/usr/local/sbin --with-http_ssl_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module
sudo make && sudo make install
sudo ln -s /usr/local/nginx/conf /etc/nginx
sudo php-fm start
sudo /etc/init.d/nginx start
[/code]

Inspired by:
http://interfacelab.com/nginx-php-fpm-apc-awesome/
http://www.softwareprojects.com/resources/programming/t-how-to-install-php-fpm-spawn-fcgi-replacement-1602.html

It looks like I might not need help... this thing appears to be working now... I'm posting from Saturn.

[URL="http://www.softwareprojects.com/resources/programming/t-how-to-install-php-fpm-spawn-fcgi-replacement-1602.html"][/URL]

Boast post! :-)

Did you get there on the Bob?

Hehe... if anyone wants to have some fun on the new (much quieter) server...

Then you need to edit your hosts file and insert this:
67.207.133.51 www.londonfgss.com

NOTE: This will only last for a while, I'm going to load balance the web servers. But for today and this weekend if things are busy this is what will speed you up.

NOTE 2: This doesn't yet have SSL installed on it, so non https:// only.

It looks like I might not need help... this thing appears to be working now... I'm posting from Saturn.

Anyhow, for my own notes:
[code]
sudo apt-get install make bison flex gcc patch autoconf subversion locate
sudo apt-get install libxml2-dev libbz2-dev libpcre3-dev libssl-dev zlib1g-dev libmcrypt-dev libmhash-dev libmhash2 libcurl4-openssl-dev libpq-dev libpq5 libsyck0-dev
[/code][code]
cd /usr/local/src/
sudo wget http://us.php.net/get/php-5.2.8.tar.gz/from/this/mirror
sudo tar zvxf php-5.2.8.tar.gz
sudo wget http://php-fpm.anight.org/downloads/head/php-5.2.8-fpm-0.5.10.diff.gz
sudo gzip -cd php-5.2.8-fpm-0.5.10.diff.gz | sudo patch -d php-5.2.8 -p1
cd php-5.2.8
sudo apt-get install defoma fontconfig-config libfontconfig1 libfreetype6 libgd2-xpm libjpeg62 libpng12-0 libt1-5 libxpm4 ttf-dejavu ttf-dejavu-core ttf-dejavu-extra ucf libttf-dev libfreetype6-dev freetype1-tools libpng-dev libjpeg-dev
sudo ./configure \
--disable-debug \
--disable-pdo \
--disable-rpath \
--enable-fastcgi \
--enable-fpm \
--enable-gd-native-ttf \
--enable-inline-optimization \
--enable-mbregex \
--enable-mbstring \
--enable-pcntl \
--enable-sockets \
--enable-sysvsem \
--enable-sysvshm \
--enable-zip \
--with-bz2 \
--with-curl \
--with-curlwrappers \
--with-freetype-dir \
--with-gd \
--with-jpeg-dir \
--with-mcrypt \
--with-mhash \
--with-mysqli \
--with-pcre-regex \
--with-pic \
--with-png-dir \
--with-zlib
sudo make all install
sudo strip /usr/local/bin/php-cgi
[/code][code]
sudo pecl install memcache
sudo pecl install apc
[/code][code]
sudo cp /usr/local/src/php-5.2.8/php.ini-recommended /usr/local/lib/php.ini
sudo mkdir /etc/php/
sudo ln -s /usr/local/lib/php.ini /etc/php/php.ini
sudo ln -s /usr/local/etc/php-fpm.conf /etc/php/php-fpm.conf
[/code][code]
cd ..
sudo wget http://sysoev.ru/nginx/nginx-0.6.35.tar.gz
sudo tar zxvf nginx-0.6.35.tar.gz
sudo rm -f nginx-0.6.35.tar.gz
cd nginx-0.6.35
sudo ./configure --sbin-path=/usr/local/sbin --with-http_ssl_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module
sudo make && sudo make install
sudo ln -s /usr/local/nginx/conf /etc/nginx
sudo php-fm start
sudo /etc/init.d/nginx start
[/code]Inspired by:
http://interfacelab.com/nginx-php-fpm-apc-awesome/
http://www.softwareprojects.com/resources/programming/t-how-to-install-php-fpm-spawn-fcgi-replacement-1602.html

Repost

[code]
sudo apt-get install defoma fontconfig-config libfontconfig1 libfreetype6 libgd2-xpm libjpeg62 libpng12-0 libt1-5 libxpm4 ttf-dejavu ttf-dejavu-core ttf-dejavu-extra ucf libttf-dev libfreetype6-dev freetype1-tools libpng-dev libjpeg-dev
[/code]

May I ask why you had to install all those font packages on a server?

May I ask why you had to install all those font packages on a server?

Captcha fun :)

The packages = greater complexity in the captcha.

A fair number of those are the result of dependencies between the packages.

Have you tried the new server yet? It's screaming along.