ModPerl

Pointers

• ApacheCon

my notes from ApacheCon 2005

• WE04 - From CGI to mod_perl 2.0, Fast!
• presentation by Philippe M. Chiasson
• gozer@ectoplasm.org
• http://gozer.ectoplasm.org/talk/

Requirements

• Apache 2.x
• mod_cgi
• perl

CGI

• 2x fork, 2x exec
• startup
• teardown

mod_perl

• Embedded Perl
• one time startup
• time time teardown
• perl locates subroutine and invokes

mod_cgi setup

    LoadModule cgi_modudule libexec/mod_cgi.so
    ScriptAlias /cgi-bin /var/www/cgi-bin

benchmarking

     ab -c1 -n50 http://.../x.pl

mod_perl setup

• wget http://perl.apache.org/download/mod_perl-2.0-current.tar.gz
• /usr/bin/perl Makefile.PL MP_APXS=/usr/sbin/apxs

    LoadModule perl_module modules/mod_perl.so
    Alias /perlrun /var/www/cgi-bin

    <Location /perlrun>
      SetHandler perl-script
      Option +ExecCGI
      PerlHandler ModPerl::PerlRun
    </Location>

• apachectl configtest
• apachectl restart
• if it works: $ENV{MOD_PERL} = mod_perl/2.0.2-dev

ModPerl::PerlRun

• as close to CGI as possible
• load, BEGIN, executed, END, destroyed
• 15x faster in demo

ModPerl::Registry

• try this first - faster, may break some cgis
• PerlHandler ModPerl::PerlRun
• faster, but CGI emulation is lacking
• 30x faster in demo
• load from cache if possible, BEGIN, execute, END

ModPerl

• Simple perl modules - pretending to be CGI
• Derives from ModPerl::RegistryCooker
• new ones can be made pretty easily in a pinch

CGI emulation

• avoid globals unless read-only
  • will stick around forever
• use "my"
• use strict and use warnings
• don't use globals for subroutines - pass args around explicitly
  • mod_perl turns entire scripts into subroutine
  • ModPerl::ROOT::ModPerl::Registry::var_www_cgi_2dbin_hello_2epl::handler
  • can create PerlClosures without telling you - man perlref
• look at the error log!
  • very verbose
  • "will not stay shared" - closure created
• use fully qualified paths in "use lib" and "require"
  • current working directory is "/"
  • doesn't fork or chdir
  • cwd is a process attribute - chdir isn't thread safe for apache
  • if your MPM is Prefork - ModPerl::RegistryPrefork or ModPerl::PerlRunPrefork

my notes from ApacheCon2005

• WE08 - mod_perl for Speed Freaks!
• presentation by Philippe M. Chiasson
• gozer@ectoplasm.org
• http://gozer.ectoplasm.org/talk/

intelligently compare speeds

• metrics - benchmarking
• comparison points
• exactly how fast are you now
• benchmark early and often
• ab - comes with apache
  • kilobytes per second - more interesting

mod_status

• overview of server's health

    LoadModule status_module modules/mod_status.so
    ExtendedStatus On

    <Location /server-status>
      SetHandler server-status
    </Location>

• http://localhost/server-status
• http://localhost/server-status?auto - machine parsable

Apache2::Status

• mod_status for mod perl
• comes with mod perl
• paths from which modules are loaded
• how much memory each module is taking

    <Location /perl-status>
      SetHandler perl-script
      PerlHandler Apache2::Status
      PerlSet Var StatusOptionsAll On
    </Location>

• http://localhost/status/perl?CGI

memory usage

• total physical RAM is the limit
• SWAP doesn't count - shouldn't use it
• Wired counts
• Shared, Resident, Total
• GTop - perl module - "use GTop"
• use BSD::Resource - "getrusage()"

shared memory

• identical memory shared by more than one process
• get as much as possible
• copy on write

module preloading

• load modules being used all the time into shared memory
• Apache2::Status - ones that take up the most space should be preloaded
• modules not preloaded will be reloaded on every request
• PerlModule CGI
• PerlModule DBI
• PerlModule DBD::mysql
• PerlModule Apache2::RequestReq
• PerlModule Apache2::ServerRec
• module initialization
  • DBI->install_driver("mysql");
  • CGI->compile(':all');
  • Check docs of the module you use

Registry Preloading

• do something like this for each perl file on disk

    #startup.pl
    use ModPerl::RegistryLoader ();
    my $rl = ModPerl::RegistryLoader->new(
        package => ¡ÆModPerl::Registry¡Ç,
        debug   => 1,
    );
    $rl->handler($url, $filename);

use Foo vs. use Foo ()

• by default, don't import anything
• use Foo > importing default EXPORTS - use Foo() > importing nothing by default

    # saves 400K
    use POSIX ();

Apache2::Const

    use Apache2::Const qw(OK DECLINDED);
    use Apache2::Const -compile => qw(OK DECLINED);

    # now use something like this?
    return $Apache2::Const::OK

Other differences

• perl script
  • STDIN/STDOUT tied
  • %ENV, @INC saved/restored
  • %ENV changes propagated
• modperl
  • none of this happens

PerlOptions - might want to turn off

• PerlOptions -Autoload = forces you to preload everything
• PerlOptions -GlobalRequest = forces passing request object around
• PerlOptions -ParseHeader = don't scan output for headers
• PerlOptions -SetupEnv = don't set up special environment vars

KeepAlive

• an HTTP feature
• generally a good idea
• mod_perl - not so much
  • servers pick up extra RAM
  • turn it off
  • event mpm might change that
• alternately - proxy requests and do keep-alives on proxy

memory leaks

• OS memory usually not reclaimed
• Perl designed for run fast, then terminate
• mod_perl is a long running process
• some perl optimizations don't apply
• all sorts of things can and will leak
• once memory is used, it isn't thrown away
  • e.g. slurping
    • don't read a whole file into a variable
    • iterate over and read one line at a time

Apache2::SizeLimit

• if limits exceeded, apache will dump spawn a new child

    #startup.pl
    use Apache2::SizeLimit;
    $Apache2::SizeLimit::MAX_PROCESS_SIZE   = 12000;
    $Apache2::SizeLimit::MIN_SHARE_SIZE     =  6000;
    $Apache2::SizeLimit::MAX_UNSHARED_SIZE  =  5000;
    $Apache2::SizeLimit::CHECK_EVERY_N_REQUESTS = 4;

    #httpd.conf
    PerlCleanupHandler Apache2::SizeLimit

profiling

• find bottlenecks in perl code
• don't guess
• mod_perl - not all profilers will work

Apache::DProf

- PerlModule Apache::DProf

• makes web server really really slow
• cd /var/httpde/logs/prof/12345
• dprofpp = read database and generate reports
  • tons of options

Devel::Profiler::Apache

• preferred replacment
• drop in replacement for Apache::DProf
• dprofpp compatible log files
• pure perl implementation

Odd Speedups

• ApacheCompression

Apache::DBI

• datbase is an extremely common back-end
• slow, expensive, bottlenecks
• wrapper to DBI
• Plug and Play, no code change
• persistent DB connections
• not a huge savings for a local mysql - really big deal for oracle

    PerlModule Apache::DBI
    PerlModule DBI

• connect_on_init
  • pre-connects to hot DB
  • once for each httpd child

    Apache::DBI->connect_on_init( $DSN, $user, $pass );

• prepare_cached
  • DB handles are cached
  • caches prepared statements into DB handles
  • compiled SQL stored on the server
  • not all clients have caught up to this

stat

• file information - size, owner, lastmod
• system call - relatively expensive

AllowOverride

• very inefficient
• depends on default config
• .htaccess files are searched for
• AllowOverride None = always use this when possible

PerlTransHandler

• by default, apache will look for a script through all parent directories - lots of stats
• PerlTransHandler Apache2::Const::OK > add after PerlHandler - stops the filesystem walking

print()

• STDOUT connected to client
• might not be buffered
• generally expensive
• group print statements together, e.g. print<<'EOF';
• print a list - very efficient and fast

$|

• $|++;
• do not buffer at all
• buffering causes slower page loads
• use $r->rflush() = an explicit buffer flush