Mongrel: Serving, Deploying, and Extending Your Ruby Applications

Next Page

Book Cover
Mongrel: Serving, Deploying, and Extending Your Ruby Applications
By Matt Pelletier, Zed Shaw
...............................................
Publisher: Addison Wesley Professional
Pub Date: October 18, 2006
Print ISBN-10: 0-321-48350-2
Print ISBN-13: 978-0-321-48350-8
Pages: 106

Table of Contents
Overview

This short cut is an introduction and guide to Mongrel, a fast, versatile RubyWeb server. If you build or manage Web applications, this will be a usefulreference as you set up and use Mongrel in your development and productionenvironments, as well as a handbook for how you can extend Mongrel to suityour own needs.

In addition to covering how to use and extend Mongrel, we also review anumber of topics that we consider "Best Practices" for modern software development,deployment, and performance testing. We discuss these in thecontext of using Mongrel, but they should be considered applicable to anysoftware project. Our own work has always benefited from seeing the techniquesof others, so we hope that sharing the experience, approach, andphilosophy that went into designing and developing Mongrel will be interestingand helpful for your own pursuits.

Section 1: What This Short Cut Covers 4

Section 2: Introduction 6

Section 3: Getting Started 13

Section 4: Configurations 20

Section 5: Production Deployment 37

Section 6: Extending Mongrel 60

Section 7: Debugging 79

Section 8: Performance 90

Section 9: Security 96

Resources 103

Acknowledgments 105

About the Authors 106

Next Page


Table of Contents

Previous Page

Book Cover
Mongrel: Serving, Deploying, and Extending Your Ruby Applications
By Matt Pelletier, Zed Shaw
...............................................
Publisher: Addison Wesley Professional
Pub Date: October 18, 2006
Print ISBN-10: 0-321-48350-2
Print ISBN-13: 978-0-321-48350-8
Pages: 106

Table of Contents

Copyright
Section 1. What This Short Cut Covers
 | Section 1.1. The Format of This Short Cut
 | Section 1.2. Zed Sez
Section 2. Introduction
 | Section 2.1. What Is Mongrel?
 | Section 2.2. How Does Mongrel Work?
 | Section 2.3. What Can Mongrel Do for Me?
Section 3. Getting Started
 | Section 3.1. Installing Mongrel
 | Section 3.2. Using Mongrel
 | Section 3.3. Supported Platforms
 | Section 3.4. Supported Frameworks
Section 4. Configurations
 | Section 4.1. Configuring Mongrel
 | Section 4.2. Common Configurations
 | Section 4.3. Hit the Ground Running: Example Configurations
Section 5. Production Deployment
 | Section 5.1. Requirements
 | Section 5.2. (Too) Many Options
 | Section 5.3. "Fake" Hardware Layout
 | Section 5.4. Software Deployment
 | Section 5.5. Configuration
 | Section 5.6. Care and Feeding
Section 6. Extending Mongrel
 | Section 6.1. Mongrel's Architecture
 | Section 6.2. Handlers
 | Section 6.3. Filters
 | Section 6.4. Distributing Your Plugins as RubyGems
 | Section 6.5. Commands
 | Section 6.6. Handlers as GemPlugins
 | Section 6.7. Advanced Handlers
 | Section 6.8. Finding More Plugins
Section 7. Debugging
 | Section 7.1. Available Tools
 | Section 7.2. Common Scenarios
 | Section 7.3. Other Tools
 | Section 7.4. Reporting Mongrel Bugs
Section 8. Performance
 | Section 8.1. A Simple Process for Tuning Your First Deployment
Section 9. Security
 | Section 9.1. Mongrel's Security Design
Resources
Acknowledgments
About the Authors

Previous Page


Copyright

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this work, and the publisher was aware of a trademark claim, the designations appear with initial capital letters or in all capitals.

The authors and publisher have taken care in the preparation of this work, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.

Visit us on the Web: www.awprofessional.com

All rights reserved. This publication is protected by copyright, and permission must obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permissions, write to:

Pearson Education, Inc.
Rights and Contracts Department
75 Arlington St., Suite 300
Boston, MA 02116
Fax: (617) 848-7047

Copyright © 2007 Pearson Education, Inc.

This product is offered as an Adobe Reader™ PDF file and does not include digital rights management (DRM) software. While you can copy this material to your computer, you are not allowed to share this file with others.

For information regarding permissions, write to:

Pearson Education, Inc.
Rights and Contracts Department
75 Arlington St., Suite 300
Boston, MA 02116
Fax: (617) 848-7047

First release, October 2006

Dedication

To my father Mike, who got me started many years ago.

M.P.

To Mike and Roxanne, for helping a poor boy when he needed it most.

Z.S.


Section 1. What This Short Cut Covers

This short cut is an introduction and guide to Mongrel, a fast, versatile Ruby Web server. If you build or manage Web applications, this will be a useful reference as you set up and use Mongrel in your development and production environments, as well as a handbook for how you can extend Mongrel to suit your own needs.

In addition to covering how to use and extend Mongrel, we also review a number of topics that we consider "Best Practices" for modern software development, deployment, and performance testing. We discuss these in the context of using Mongrel, but they should be considered applicable to any software project. Our own work has always benefited from seeing the techniques of others, so we hope that sharing the experience, approach, and philosophy that went into designing and developing Mongrel will be interesting and helpful for your own pursuits.


Section 1.1. The Format of This Short Cut

1.1. The Format of This Short Cut

This short cut uses the common formatting conventions to represent code, filenames, and so on. If you have never read a book on software before, the following description will prove essential.

Indented blocks of code or shell commands will appear in a fixed-width font:

uri '/', :handler => DirHandler.new('/var/www/opinions/')

A reference to a file, function, or class within a sentencelike HttpHandler, mongrel.log, or some_function(param)will be in italics.


Section 1.2. Zed Sez

1.2. Zed Sez

We have created a special feature in this book called Zed Sez. Mongrel is certifiably Opinionated Software, and Zed Shaw, the author of Mongrel and coauthor of this short cut, is an Opinionated Developer. Mongrel represents somewhat of a "Best Hits" of the practices and techniques he has refined over time in his work. He wields his no-nonsense, pragmatic approach without fear or hesitation (as anyone on the Mongrel mailing list can attest), so we feel it is only appropriate to make space for some of Zed's rants to give you a real taste of what Mongrel is all about. You will see little sidebars throughout the book that come right from the heart (Zed even drew his own head). You may not agree with everything he says, but you probably should.


Section 2. Introduction

In this section we introduce Mongrel, explain its origins and history, and how it is used. We have found there are some common misconceptions about how it works or what exactly it does, so we will help clear these up so we can start off on the right foot. Even if you think you know what Mongrel is all about, make sure you read this first.


Section 2.1. What Is Mongrel?

2.1. What Is Mongrel?

Mongrel is a small, fast, mostly Ruby Web server.[1] It was created to do only a few things, but to do them really well: Make the development, deployment, and extension of Ruby applications Dead Simple. That's it.

[1] By "mostly" we mean it uses some Ruby C extensions to handle the URI parsing, but otherwise it is all Ruby.

2.1.1. A Little History

Zed Shaw started Mongrel in late December 2005. He wrote it because he was frustrated by the existing solutions for developing and deploying Ruby Web applications. FastCGI was abysmally broken and WEBrick was abysmally slow. He had written SCGI Rails Runner earlier in the year as a replacement for FastCGI, but he quickly hit a ceiling, as it was still just a middleman. Also, he was bored. So he wrote Mongrel to exorcise some demons, and to scratch an itch. It turns out that he was not the only one feeling frustrated, so his solution turned out to be pretty useful for a lot of Ruby developers and sysadmins. Mongrel's success is largely a result of Zed's approach to building software. In the Ruby development world, philosophy and practices are at least as important as the code itself, so Zed has forged Mongrel in his own way. It is a Web server; it is not a universal adapter. It is tight and small and secure, and he will be happy to explain why.

Mongrel is already packaged for distribution for the following flavors of Linux and will be included in Apple's next release of OSX (called Leopard, see http://www.apple.com/server/macosx/leopard/more.html for details).

Zed Sez

When I started on Mongrel, the deployment landscape was nasty. It seemed that only 37signals knew how to deploy large-scale applications in Rails, and I think that's only because they sacrificed chickens to Ninhursag between bouts of Touretts-inspired screaming. I was working on SCGI Rails Runner, and that wasn't much of an improvement. What we needed was just a Web server that ran Rails as fast as FastCGI, but as simply as WEBrick. At every turn I tried to make sure Mongrel was easy to set up, but still flexible so it would fit into as many locations as possible.


Section 2.2. How Does Mongrel Work?

2.2. How Does Mongrel Work?

The basic function of a Web server is to listen for HTTP requests and send back responses. The requests are usually just a URL, some headers, and sometimes a body (like form parameters). If an image was requested, the Web server will usually pick that file off the file system and send it along. If the request is intended to reach a Web application, the Web server will usually pass the request to the application, the application will generate a response in the form of headers and HTML, and the Web server will relay that back to the requesting party. Most Web servers have grown into hulking beasts because they are used as catch-all applications that come with every imaginable option, module, and feature. Mongrel does not wish to replace these kinds of applications, it just wants to serve HTTP requests really well, especially the kind that need a Ruby application. Mongrel has been designed from the start to be Dead Simple: easy to use, easy to extend, and pretty secure.


Section 2.3. What Can Mongrel Do for Me?

2.3. What Can Mongrel Do for Me?

This section describes what you can expect from Mongrel, whether you are a developer, sysadmin or manager.

2.3.1. For the Developer

Mongrel is great for local development.

If you are doing Ruby development, then using Mongrel is a no-brainer. It's much faster than WEBrick and can handle heavy development work (with all the logging and reloading turned on) easily. It installs with one command and configures with another. You can be up and running in less than a minute.

It integrates nicely with Rails (and other Ruby frameworks).

Mongrel was initially created to simplify developing and running Rails applications, so most Mongrel users are Rails developers. They have contributed some excellent plugins to handle common tasks like clustering, file uploads, and DRb process handling, and they are a great place to learn how to write your own. In June 2006 the Rails Core team integrated Mongrel with script/server, so if that is your comfort zone, you only need to have Mongrel installed.

Extend your application easily.

Mongrel has been designed to make extending it (or your Web application) easy. You can think of Mongrel as a tight Ruby library that deals with the sticky mechanics of HTTP, gives you simple access to the request, and otherwise stays out of your way. The interfaces for extending it are clean and easy to understand, so even if you shy away from Ruby outside of Rails-world, you should take a look. Frequently you can speed up your application by offloading certain tasks and having Mongrel handle them instead. If you have your own Ruby library that is not one of the supported frameworks (see Section 3.4), you can write a handler so Mongrel can load it with all the special options you need. See Section 6 for details.

Mongrel includes excellent debugging tools.

Mongrel includes thorough tools to help you debug your application. You can watch the entire request and response process too, and write your own handlers and plugins to dig as deeply as you need to test your application and environment.

Mongrel is the same in production.

As you build, test, and deploy your application, you can rest assured that Mongrel will work exactly the same when in the production environment. This comes in handy when you need to run your tests, integrate with other systems and services, work with caching, etc. See Section 4 to see how Mongrel is used on both local development boxes and production environments.

2.3.2. For the System/Network Administrator

If you are wondering how Mongrel would fit into your system architecture, you shouldn't feel alarmed. Zed was a sysadmin for years, and spent so much time dealing with poorly designed packages that he knew what it would take to make Mongrel play nice with common server configurations.

Mongrel is a real Web server.

Mongrel is a fully HTTP-compliant server (more compliant that most), and can speak HTTP directly with all the network devices that common hardware and software configurations use. This saves you from the need to translate requests into CGI or use buggy or slow modules (like mod_ruby or FastCGI) to connect to your Ruby application.

Mongrel is a Ruby application.

Since Mongrel is written in Ruby, it can load and run any Ruby code natively, just like Tomcat does with Java Applications (except it doesn't need a 120MB footprint!). This is why you won't need something like FCGI, SCGI Rails Runner, or modules anymore to pass the HTTP Request to the Application Framework.

Mongrel can be clustered and deployed easily.

Mongrel has been designed to be Dead Simple to set up in production environments. This means that major concerns such as clustering and deployment have been addressed and best practices are already in place. See Section 4.2 to learn more about mongrel_cluster and Section 5 to see how folks are setting it up with Capistrano.

2.3.3. For the Manager

Discriminating minds choose Mongrel.

Mongrel has been widely seen as the clear choice among the available options for developing and running a Rails application. The same could be said for developing and running any Ruby application, but so far the biggest group of Ruby developers are using Rails to build their sites and Mongrel to run them. Mongrel is not a whole lot of code, and it hasn't even been around for a year. Yet for some reason, within just six months it became the preferred way to develop and serve production Web applications.

Mongrel is pretty damned secure.

Claiming that one's software is "secure" is usually asking for it, so we'll say that Mongrel is "pretty damned secure." Zed explains why in Zed Sez in Section 9.

Mongrel's license is capitalist-friendly.

Mongrel and its sub-project GemPlugin use the Ruby license, which you can read here at http://mongrel.rubyforge.org/license.html. Some folks are scared of using open source software because they think it will force them to open-source their code (which is actually true in certain cases with GPL and LGPL, for example). Ruby's license is less restrictive, and was good enough for Apple to include Mongrel in the next version of OSX.


Section 3. Getting Started

Now that you know what Mongrel is all about, let's start using it. You can install it with a single command and configure it with just one more. This section will walk you through the basic usage. We'll dig into configurations in Section 4.

These examples assume you will be using Mongrel to serve a Rails application. If that's not the case, and you're using one of the other supported frameworks or plan to use Mongrel for your own custom application, you'll still want to follow along and pay close attention. Even though Mongrel is framework-agnostic, most of the community currently uses it to run Rails applications, so the best-tested and most mature commands, handlers, filters, and plugins are all Rails-related. The other supported frameworks follow similar patterns to get Mongrel running. If you're planning to extend Mongrel yourself (it's easy!) see Section 6.

In our examples we also assume you are using a flavor of Unix. Mongrel is fully supported on Windows, so in most cases the examples will work without modification. Please see Section 3.3 and the Mongrel Web site (http://mongrel.rubyforge.org) for details on Win32-specific limitations or differences.


Section 3.1. Installing Mongrel

3.1. Installing Mongrel

Installing Mongrel is a snap. The only prerequisites are Ruby 1.8.4 or greater and RubyGems. If you don't have these already, you'll have a hard time running Ruby at all. For help with installing these on your system, please refer to the Ruby Web site (http://www.ruby-lang.org/en/), but in most cases you will want to install it.

You can install Mongrel in one step from your command line:

$ sudo gem install mongrel --include-dependencies

This will install the Mongrel gem and its dependencies, which are gem_plugin and, depending on your platform, either daemons or win32-service. If you're not on Windows you'll see Mongrel build the C extension during the gem install. Check the mailing list and Web site if you run into any issues, as it's probably caused by a funky Ruby installation.

3.1.1. Mongrel Pre-Releases

If you are the risk-taking type (you are using Ruby after all, aren't you?), then you can install and help test Pre-Release gems from Mongrel's Pre-Release repository. Like most pre-release code, these gems typically include new features and/or bugfixes that need some platform testing, and overall they're pretty stable. Ruby tends to introduce some interesting system-level differences, so testing on different platforms is extra important.

$ sudo gem install mongrel --source mongrel.rubyforge.org/releases


Section 3.2. Using Mongrel

3.2. Using Mongrel

Now that you've installed Mongrel (and probably spent twice as long reading about it), you can take a look at what it gives you. Mongrel uses commands to do business with your application. You can see which commands come with Mongrel out of the box:

$ mongrel_rails
Usage: mongrel_rails <command> [options]
Available commands are:
 - restart
 - start
 - stop
Each command takes -h as an option to get help.

You'll notice we're calling the mongrel_rails executable. This is just a Ruby script that comes with Mongrel and includes the three basic commands you need to run a Rails application: start, stop and restart. Each command takes optional configuration parameters, which default to values that will be familiar to Rails developers and are similar to what you'll see available in Rails' script/server command. We'll get into the details of how you can use these options in Section 4. For now, let's just see what the options for the start command are.

$ mongrel_rails start -h
Usage: mongrel_rails <command> [options]
    -e, --environment ENV            Rails environment to run as
    -d, --daemonize                  Whether to run in the background or not
    -p, --port PORT                  Which port to bind to
    -a, --address ADDR               Address to bind to

    -l, --log FILE                   Where to write log messages
    -P, --pid FILE                   Where to write the PID
    -n, --num-procs INT              Number of processors active before clients denied
    -t, --timeout TIME               Timeout all requests after 100th seconds time
    -m, --mime PATH                  A YAML file that lists additional MIME types
    -c, --chdir PATH                 Change to dir before starting (will be expanded)
    -r, --root PATH                  Set the document root (default 'public')
    -B, --debug                      Enable debugging mode
    -C, --config PATH                Use a config file
    -S, --script PATH                Load the given file as an extra config script.
    -G, --generate CONFIG            Generate a config file for -C
        --user USER                  User to run as
        --group GROUP                Group to run as
        --prefix PATH                URL prefix for Rails app
    -h, --help                       Show this message
        --version                    Show version

We want to just get our vanilla Rails application running on Mongrel, so all we need to do is run:

$ cd ~/code/examples/
$ mongrel_rails start
** Starting Mongrel listening at 0.0.0.0:3000
** Starting Rails with development environment...
** Rails loaded.
** Loading any Rails specific GemPlugins
** Signals ready. TERM => stop. USR2 => restart. INT => stop (no restart).
** Rails signals registered. HUP => reload (without restart). It might not work well.
** Mongrel available at 0.0.0.0:3000
** Use CTRL-C to stop.

You can see Mongrel loads Rails in the following way:

  1. Start the HttpServer, listening on port 3000

  2. Load the Rails-specific configuration values and plugins, then mount the Rails application at "/"

  3. Set up the signals

A common practice is to daemonize Mongrel so it runs in the background, and use that same terminal to tail the Rails development log. You can stop it using the stop command. Once you get Mongrel running, you can configure it to suit your particular needs.

3.2.1. Where Mongrel Puts Files

When running a Rails app, Mongrel will store its log and PID file in RAILS_ROOT/log by default. Mongrel's log file mongrel.log stores Mongrel-related messages only. It is not intended to replace a request log (e.g., apache.log) or the Rails' logs (e.g., development.log). See Section 7 for more info about enabling robust logging features.


Section 3.3. Supported Platforms

3.3. Supported Platforms

Here you will find a description of the support and limitations of the various operating systems Mongrel runs on.

3.3.1. Unix

Mongrel runs on most flavors of Unix. Debian and SuSE have packages, and FreeBSD has ports. Other flavors of linux, OSX, OSX Server, and Solaris can install and run it. If your OS is not listed, you should try to install it anyway.

3.3.2. Windows

Mongrel is supported on Windows and with a few exceptions will work the same as it does on Unix. The major differences to note are:

  • Mongrel will not run daemonized (mongrel_start -d). You can run it as a service using the mongrel_service plugin.

  • Stop and restart commands are fickle.

  • Mongrel is not as fast as it is on Unix. This is because Ruby itself is much slower on Windows, but if you are familiar with the speed difference already, then Mongrel will be a huge step forward.

For details on Mongrel on Windows, see the Mongrel Web site (http://mongrel.rubyforge.org).


Section 3.4. Supported Frameworks

3.4. Supported Frameworks

If you are using one of the popular Ruby application frameworks, Mongrel may already support it. More developers are writing handlers to load their own frameworks, so if you use a framework or library that is not listed here, take a look at how handlers work in Section 6.

3.4.1. Ruby on Rails

Mongrel has excellent Ruby on Rails support. Most Mongrel users are Rails developers, so the best plugins, handlers, and documentation are all for Rails applications and deployments. See the Mongrel Web site (http://mongrel.rubyforge.org) for details on any limitations or caveats with Rails.

3.4.2. Iowa

Iowa is supported on Mongrel. See the Iowa Web site (http://enigo.com/projects/iowa/index.html) for details.

3.4.3. Camping

Camping, the micro-framework for Ruby Web applications, is supported (and is fed fresh salmon weekly) on Mongrel. See the Camping Web site (http://camping.rubyforge.org) for more details.

3.4.4. Og/Nitro

Nitro is supported on Mongrel. See the Nitro Web site (http://www.nitroproject.org/) for more details.


Section 4. Configurations

This section will show you how Mongrel can be configured for your development or deployment purposes. Mongrel will even generate configuration files for you. We will walk through each of the configuration options you saw earlier in detail.


Section 4.1. Configuring Mongrel

4.1. Configuring Mongrel

Mongrel lets you configure and extend it through Yaml configuration files. You can write these by hand or pass configuration options into Mongrel, and it will generate one for you that you can later pass as a parameter.

[Pages 22 - 23]

Option | Config File Param | Description | Default
-e, environment ENV | :environment | Configures your Rails environment to what you need. | development
-d, daemonize | :daemon | If given (no options) then Mongrel will run in the background (not on Windows). | false
-p, port PORT | :port | Port to bind to when listening for connections. | 3000
-a, address ADDR | :host | Address to bind to when listening for connections. | 0.0.0.0 (every interface)
-l, log | :log_file | Where to dump log messages in daemon mode. Use an absolute path (not on Windows). | $PWD/log/mongrel.log
-P, pid | :pid_file | Where to write the PID file so start and stop commands know the Process ID. Use absolute paths. | $PWD/log/mongrel.pid
-n, num-procs INT | :num_processors | Maximum number of concurrent processing threads before Mongrel starts denying connections and trying to kill old threads. | 1,024
-t, timeout TIME | :timeout | Time to pause between accepting clients. Used as a throttle mechanism. | 0
-m, mime PATH | :mime_map | A YAML file that maps from file extensions to MIME types for static files. It's important that if you are using page caching or you have a different language setting like UTF8, then you have to configure this. Read more below. | not set
-c, chdir PATH | :chdir | Directory to change prior to starting Mongrel. cwd means "change working directory." | . (current directory)
-r, root PATH | :root | Document root where Mongrel should serve files from. If you are putting Mongrel under a different base URI, and you want it to serve files out of a different directory, then you need to set this. | public
-B, debug | :debug | Turns on a debugging mode, which traces objects, threads, files request parameters, and logs accesses, writing them to log/ mongrel_debug. This option makes Mongrel very slow. | false
-C, config FILE | :config_script | Takes whatever options you've set for Mongrel, plus the current defaults, and then writes them to a YAML file suitable for use with the -C option. | N/A
-S, script PATH | N/A | A special Ruby file that is run after Rails is configured to give you the ability to change the configuration with Ruby. This would be where you can load customer Mongrel handlers, extra libraries, or set up additional Ruby code. This option is fairly advanced, so use with caution. | not set
-G, generate PATH | N/A | Takes whatever options you've set for Mongrel, plus the current defaults, and then writes them to a YAML file suitable for use with the -C option. | not set
user USER | :user | Changes to this user if you are starting Mongrel as root and need extra security. | not set
group GROUP | :group | Must be combined with user as it changes the group as well. | not set
prefix URI | :prefix | The URI prefix for all generated URLs from your Rails application, must start with / and not end in /. | not set

In most cases, during development you won't need to change the defaults. A common practice is to daemonize the Mongrel process, and when it's necessary to run multiple Rails applications at once, specify the port, like so:

$ mongrel_rails -d -p 3001

This can become tedious if you have to type it frequently, so in keeping with the Don't Repeat Yourself principle, Mongrel will take in a config file (see the -C parameter) in YAML format that sets these parameters for you. It even comes with a Generator parameter (see the -G parameter) that will generate this config file for you. You can tell Mongrel to put the file wherever you want, and include all the parameters you want to set. In this example we put it in Rails' config directory.

$ mongrel_rails start -d -p 3001 -G config/bam.yml
** Writing config to config/bam.yml
** Finished. Re-run without -G and WITH -C using your new config file.

The content of your config file should look like this:

---
:config_script:
:debug: false
:pid_file: log/mongrel.pid
:num_processors: 1024
:docroot: public
:user:
:host: 0.0.0.0
:mime_map:
:prefix:
:group:
:port: '3001'
:daemon: true
:cwd: /Users/matt/examples/bam
:config_file:
:includes:
- mongrel
:environment: development
:timeout: 0
:log_file: log/mongrel.log

We can now run Mongrel without having to type the config in every time:

$ mongrel_rails start -C config/bam.yml

4.1.1. Mongrel Scripts

If you want to mount a handler yourself, you can tell mongrel_rails to run a script file. It's just a Ruby file, so you can require or include whatever code you need. However, don't name the file with an .rb extension, because Rails will automatically load it if it is in the load path. We usually use .conf, though it doesn't really matter.

Mongrel has a DirHandler that will list the contents of a directory, so we'll mount the /drawing prefix to point to a directory on our system that we want to let users browse. You could mount a directory from anywhere in your file system. Let's create a file drawings.conf with the following content:

uri '/your_face', :handler => DirHandler.new('/Users/matt/examples/your_face')

If you are using a config file (like we just used above), you'll need to regenerate it with all the correct options, or you can just edit the existing config manually. Mongrel will give precedence to the parameter values in the config file over those in the command line. Since the -G parameter writes out all the parameters (even the empty ones, like :config_script in the above example), if you tried to run the following command, Mongrel would NOT load the script file.

$ mongrel_rails start -C config/bam.yml -S config/drawings.conf

Instead you should change the bam.yml file's existing :config_script parameter to config/drawings.yml, and it will work like a charm.


Section 4.2. Common Configurations

4.2. Common Configurations

Now that you can see how easily Mongrel can be configured, let's see what people are doing with it. Most of the configuration options will change from defaults because you want to run Mongrel in a cluster or in a production environment where you want to sit it behind a static HTTP server like Apache.

4.2.1. Mongrel Standalone

As a real HTTP server, Mongrel will work just fine as a front-facing Web server. After all, this is how it works in your development environment. Typically small Web sites will run just fine with a single Mongrel process. If you need SSL, you can run Mongrel behind a reverse-proxy like Pen. If you have request hogs like file uploads, you'll probably want to run more than one Mongrel, which means you'll need Mongrel Cluster and a reverse-proxy server.

4.2.2. A Pack of Mongrels (mongrel_cluster)

Depending on what kind of Web application you're running and how much traffic you expect, you might want to have multiple Mongrel processes running in a cluster. If you are accustomed to running multiple FCGI processes, then you will want to run a cluster of Mongrel processes (a "cluster" in this case does not mean you need separate machines, just separate processes running on a single machine). To help figure out how many Mongrels you'll need, you should read "How Many Mongrels" (http://rubyforge.org/pipermail/mongrel-users/2006-May/000200.html). If you need at least two, then you'll want to run a cluster. You do this with the mongrel_cluster gem, written by Bradley Taylor from RailsMachine. Detailed instructions for setup and use are available on the Mongrel Web site (http://mongrel.rubyforge.org).

4.2.3. Mongrel Behind a "Static" Web Server

A common configuration for running a Rails application these days is to have a fast, compiled Web server like Apache field incoming requests, and have it reverse-proxy to a cluster of Mongrels. We call this a "static" server setup because the front-facing server will be good at serving static content like cached pages, CSS, Javascript, and images fast. Mongrel can serve static files quickly, but Ruby will never run as fast as a compiled binary, so use that to your advantage. The static Web server is usually configured with its mod_rewrite to only pass appropriate requests to the Mongrel cluster. If you are running a fully cached site, for example, this kind of setup will never even need to call Rails.

4.2.3.1. Static Server Options

Apache has been frowned upon by many Rails developers because of its flaky support for FastCGI, which was previously the least-horrible way to run a Rails application. FastCGI was pretty buggy, requiring scripts like script/spawner and script/reaper to kill and start FastCGI processes on the fly. Since Mongrel eliminates this problem, Apache has become a preferred option among those running Rails sites. We provide an example configuration with Apache in Section 5.

4.2.3.2. What about LightTPD?

When FastCGI was the best option out there, LightTPD (aka Lighty) was considered a top choice for a static server, sometimes preferred over Apache. At the moment Lighty's mod_proxy module has a bug that will not properly connect to Mongrel processes if they go down and come back. This problem will be fixed in Lighty 1.5.0 with the mod_proxy_core module. Keep an eye out for Lighty announcements. Once this issue is resolved, Lighty will likely return to the shortlist.


Section 4.3. Hit the Ground Running: Example Configurations

4.3. Hit the Ground Running: Example Configurations

In this section we will cover two example configurations that will let you run a cluster of Mongrels behind a fast static Web server.

4.3.1. Apache Has Returned: Apache + mod_proxy_balancer + Mongrel

With the advent of Mongrel, Apache has risen from its ashes to once again become a preferred configuration option among those using Mongrel in production. The 2.2 version of Apache includes a new module called mod_proxy_balancer that will allow you to reverse-proxy, something that mod_proxy does not allow.

Apache is a jungle of configuration, so in this example we will use a simple approach to avoid confusion and minimize heartburn. If you already use Apache and have a preferred method for organizing your files, you should know how to adjust the following examples.

4.3.1.1. Install Apache 2.2

We're going to download and compile Apache from source. If you have trouble getting Apache to run and compile, you will find many excellent resources online. We are going to include some common modules.

$ curl -O http://www.ip97.com/apache.org/httpd/httpd-2.2.3.tar.gz
$ tar zxvf httpd-2.2.3.tar.gz
$ cd httpd-2.2.3
$ ./configure \
  --enable-proxy \

  --enable-proxy-balancer \
  --enable-proxy-http \
  --enable-rewrite \
  --enable-cache \
  --enable-headers \
  --enable-ssl
$ make
$ sudo make install
$ sudo /usr/local/apache2/bin/apachectl -k start

4.3.1.2. Set Up Your Mongrel Cluster Config

We're going to use a simple cluster of two Mongrels, using mongrel_cluster. Here we simply run the cluster::configure command, then start the cluster.

$ cd ~/code/examples/duck
$ mongrel_rails cluster::configure -p 3000 -e production -a 127.0.0.1 -N 3
Writing configuration file to config/mongrel_cluster.yml.
$ mongrel_rails cluster::start
Starting 3 Mongrel servers...

4.3.1.3. Modify the Default Apache Config

We will add one line to the default Apache config http.conf (look in /usr/local/apache2/conf)

# Include an external config for our examples
Include /usr/local/apache2/conf/duck.conf

For the purpose of this example, we're going to put all new configuration details in a single file called duck.conf in the default Apache configuration directory (/usr/local/apache2/conf/).

NameVirtualHost *:80
# Setup the cluster
<Proxy balancer://duck_cluster>
  BalancerMember http://127.0.0.1:3000
  BalancerMember http://127.0.0.1:3001
  BalancerMember http://127.0.0.1:3002
</Proxy>
# Setup the VirtualHost for your Rails application
<VirtualHost *:80>
  ServerAdmin duck@example.com
  ServerName localhost
  ServerAlias localhost
  DocumentRoot /Users/matt/code/examples/duck/public
  <Directory '/Users/matt/code/examples/duck/public'>
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>

  ProxyPass / balancer://duck_cluster/
  ProxyPassReverse / balancer://duck_cluster/

  # Setup your Rewrite rules here
  RewriteEngine On

  # Rewrite index to check for static
  RewriteRule ^/$ /index.html [QSA]

  # Send all requests that are not found as existing files to the cluster
  RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} ! -f
  RewriteRule ^/(.*)$ balancer://duck_cluster%{REQUEST_URI} [P,QSA,L]

  # Error logs
  ErrorLog /Users/matt/code/examples/duck/log/apache_error_log
  CustomLog /Users/matt/code/examples/duck/log/apache_access_log combined
</VirtualHost>

4.3.2. From Russia, with Love: Nginx[2]

[2] This configuration is based on work by Ezra Zygmunowicz and has been used with permission.

One of the chieftains of the Rails deployment world, Ezra Zygmunowicz, found a Russian Web server that happens to be mature, feature-complete, and fast called Nginx (think "Engine X"). It has a low memory footprint, and the proxying is faster than Apache's mod_proxy_balancer. He worked with some fellow developers and managed to get the configuration worked out. There is a wiki for Nginx English Documentation (http://wiki.codemongers.com/Nginx) to help guide you through any changes you'd like to make for your own setup. You'll want to set up mongrel_cluster the same way we did in the previous section for Apache.

4.3.2.1. Install Nginx

Installing Nginx is straightforward. This will install it into /usr/local/nginx.

$ curl -O http://sysoev.ru/nginx/nginx-0.4.0.tar.gz
$ tar zxvf nginx-0.4.0.tar.gz
$ cd nginx-0.4.0
$ ./configure --sbin-path=/usr/local/sbin --with-http_ssl_module
$ make
$ sudo make install

4.3.2.2. Set Up the Nginx Config

Now we can use a nice configuration that Ezra has honed to a sharp edge. In this example we are saving this in our Rails application's config directory as nginx.conf.

# User and group to run as
user  matt matt;
# Number of nginx workers
worker_processes  2;
# pid of nginx master process
pid        logs/nginx.pid;
# Number of worker connections. 1024 is a good default
events {
  worker_connections  1024;
}
# start the http module where we config http access.
http {
  # Pull in mime-types. You can break out your config

  # into as many includes as you want to make it cleaner
  include       conf/mime.types;
  # set a default type for the rare situation that
  # nothing matches from the mime-type include
  default_type  application/octet-stream;
  # configure log format
  log_format  main  '$remote_addr - $remote_user [$time_local] $status '
                    '"$request" $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "http_x_forwarded_for" ';
  # main access log
  access_log  logs/access.log  main;
  # main error log
  error_log  logs/error.log debug;
  # OSX does not support sendfile.
  # Uncomment this if you're on linux or bsd
  #sendfile        on;
  # These are good default values.
  tcp_nopush         on;
  keepalive_timeout  65;
  tcp_nodelay        on;
  # this is where you define your mongrel clusters.
  # you need one of these blocks for each cluster
  # and each one needs its own name to refer to it later.
  upstream mongrel {
      server 127.0.0.1:3001;
      server 127.0.0.1:3002;

      server 127.0.0.1:3003;
  }
  # output compression saves bandwidth
  gzip on;
  gzip_min_length  1100;
  gzip_buffers     4 8k;
  gzip_types       text/plain text/html text/css text/js;
  # the server directive is nginx's virtual host directive.
  server {
    # port to listen on. Can also be set to an IP:PORT
    listen       80;
        # sets the domain[s] that this vhost server requests for
    #server_name  example.com www.example.com;
      # Set the max size for file uploads to 50Mb
    client_max_body_size 50M;
    # doc root
    root /Users/matt/code/examples/duck/public;
        # vhost specific access log
    access_log  logs/host.access.log  main;
    # If you have a 'maintenance.html' in your doc root, all
    # requests will be forwarded to that automatically.
    # This is part of Capistrano's 'disable_web' task.
    if (-f $document_root/maintenance.html){
      rewrite  ^(.*)$  /maintenance.html last;
      break;
    }

    location / {
      # needed to forward user's IP address to rails
      proxy_set_header  X-Real-IP  $remote_addr;
      # needed for HTTPS
      proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect false;
      # Basic Caching
      # Add /index.html to the request and check to see if that
      # file exists. If it does, continue to the next config.
      if (-f $request_filename/index.html) {
        rewrite (.*) $1/index.html break;
      }
      # Rails Caching rules
      # Add .html to the end of the request, then check for a file
      # with that name. If the file exists, continue to the next
      # config. Otherwise set the headers and proxy the request to
      # the mongrel cluster.
      if (-f $request_filename.html) {
        rewrite (.*) $1.html break;
      }
      if (! -f $request_filename) {
        proxy_pass http://mongrel;
        break;
      }
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   html;
    }
  }
}

You can test the configuration like so:

$ sudo /usr/local/sbin/nginx -t -c config/nginx.conf

Starting is the same as the testing command, without the -t option.

$ cd ~/code/examples/duck
$ sudo /usr/local/sbin/nginx -c config/nginx.conf

Stop using kill

$ kill -15 pid

4.3.2.3. Other Static Server Configurations

There are other Web server configurations and tools that will work well with Mongrel, including Pen, Pound, Litespeed, and others. You can find more details at the Mongrel Web site (http://mongrel.rubyforge.org).


Section 5. Production Deployment

It's nearly impossible to tell you how to deploy a real production deployment. Each organization has its own peculiar brand of software, hardware, policies, procedures, people, quirks, and just plain dumb stuff they do on a daily basis. Trying to cover every possible thing you'd run into could (and does) fill up entire bookshelves.

In this section we'll go through one typical production deployment. We'll make it platform-agnostic and mention the places where you'll have to know your stuff about your system. This production deployment will not cover Windows since that's a whole book in itself and it's very rare for people to do anything but small- to medium-sized Ruby deployments on Windows.[3]

[3] This isn't a dig on Windows, it's just a reality of Ruby and how it runs on Windows.

What we'll do is walk through a choice of hardware, software, where the software goes, how best to configure it at first, and how to test each component in stages. After that you'll have to monitor your deployment to make sure it continues to run and start planning for your next big move should you grow out of this installation. We will leave this part of the configuration to you and other larger books you can find.


Section 5.1. Requirements

5.1. Requirements

You will not do this in a day. If you are expecting to code until 1 minute before your deadline and then simply point and click and have an instant server then you need to take some kind of medication because you are violently hallucinating. You will need at least a week of 8 hours days to make sure your first deployment works and to have the time to do it right.

Yes, that's right, 40 hours minimum. If you have no Unix system administration experience and no network administration experience then hire someone on a consulting basis and expect them to still take this long. Why would an expert take as long as a beginner? Because they don't know the particular brand of dumbass at your company. Harsh, but it's true. If they're good, they might do it in 20 hours, and if you're good, then an expert can do it in three. If you've done your first deployment in less than that, then you must be forgetting all the time you spent fixing it later.

Because of the wildly differing nature of all the various operating systems possible, we have to gloss over the actual installation of software. We will list any gotchas we or our reviewers found for the various systems, but it's up to you to use your system's package installation method.

5.1.1. Required System Access

This is important enough to merit its own subsection. If you do not have sudo or root access to all of these machines, then don't bother continuing. It is possible to install everything to an unprivileged directory, but that is a major piece of work and isn't worth the trouble.

5.1.2. Best Practices Rubric

A best practice is most easily defined as "Whatever is working consistently for everyone else." Your first production deployment is not the time to become clever. What you want to do is adopt some common best practices, but since this short cut doesn't have the space to tell you what those are, we'll give you a little rubric to see if you're ready.

  1. Does your application have full unit testing?

  2. Are you doing automated daily deployments to a staging server?

  3. Does your planned production deployment have agreement from everyone that it should go out?

  4. Do you have either a QA team that can test it or automated "interaction tests" with a tool such as Watir or Selenium?

  5. Do you already have the hardware installed, configured with your chosen OS, and networked properly?

  6. Have you tested that the network is configured correctly and fast enough for your deployment?

  7. Do you have the proper amount of RAM and disk for your deployment?

  8. Is Ruby installed on all machines such that it can build Mongrel properly?

  9. Is there a dedicated person whose sole responsibility is building and maintaining the production deployment?

  10. Does this person have a dedicated developer to help or is he already a developer familiar with the code?

  11. Do you have a shared location where you can document the deployment, such as a Wiki or CMS?

  12. Do you know how to use httperf or ab and know what the statistics mean?

  13. Are you using Rails migrations for all of your database changes and have scripted anything that migrations can't do?

For each question you answer with "NO," add 10 hours to your time estimate for completion. This may seem unrealistic, since saying "NO" to everything means it'll take 190 hours (about one month), but this estimate is actually low according to most first deployment experiences. What we've found is that without all these things in place, you end up doing a poor deployment and spend the next month fixing it up.

If you're just starting out and can't answer these questions properly, then consider these as instructions for what you need to do for a good development and staging setup. Go through the pains of getting to "YES" for each of these questions in your staging setup and then do your production deployment.

5.1.3. Worst Practices

Zed Sez

Programmers recently had a major shift in their quality focus with the introduction of best practices from the Agile methodologies and the Capability Maturity Model. These days if you aren't doing continuous integration, writing solid unit tests, performing personal process improvement, frequently communicating with your team, and automating everything, then you are easily labeled an amateur developer.

Nothing like this has happened for system administrators. I'm not quite sure how this half of the IT industry gets away with it, but my experience has been that system administrators are far behind in their quality practices compared to developers. They frequently do things by hand, never measure the quality of their systems, are constantly making unilateral changes that impact the company, and frequently make policy decisions in the name of "security." Usually these "security" policies only make their unautomated job easier rather than actually improve security.[4]

[4] I worked at a government agency that said HTTPS+XML was not secure, but that SOAP was, and used telnet to manage important records based on system administrators setting policies. If the irony is not obvious (lucky you, you've never had to deal with SOAP), SOAP is HTTP(S)+XML, and telnet sends in cleartext.

Sure, developers screw up too, but today developers who haven't adopted many quality practices are ridiculed and laughed at by their peers. Nobody questions why a developer can build, test, deploy, and validate an entire product with one script, yet a system administrator has to spend two hours typing to do simple things like change passwords.

If you're a system administrator, the best advice I can give is "Automate or die." I firmly believe that 99% of what system administrators do can be automated since I've done just that everywhere I've worked. Watch out because there's money in wiping you out with automation. It just takes one good book written by the Kent Beck of system administration and you're out of a job.

If you're a system administrator and you're offended by this, then answer this question: "Could you not run ssh for a week?" No, I don't mean write a script named "flabber" that just runs ssh. I mean is your administration automated to the point that you don't need to ssh manually into your machines for weeks on end? If you log in hourly or even daily, then you are part of the problem. You have no right to be offended.

The following list of "worst practices" is for both developers and system administrators, but it's more for system administrators since they'll be doing the deployments. I've done all of these at some point, but I don't do them anymore.

  1. Creating and maintaining unversioned files.

  2. Doing config changes on production, manually rather than through the automated deploy process

  3. Not telling everyone you're making changes.

  4. Deploying without telling anyone.

  5. No strict development, staging, production-automated maintenance process.

  6. No production rollout-approval process.

  7. Not indicating which changes are going into production.

  8. Not following best practices (you are not smarter than I am).

  9. Not following the same process for dev, staging, and prod every time.

  10. Letting anyone deploy to production.

  11. Not letting anyone deploy to development.

  12. Preventing developers from controlling staging and development.

  13. Not collaborating on the staging deployment so they know what's going on.

  14. Not documenting their systems and deployment methods.

  15. Not tracking quality of service to identify production problems and proactively prevent them.

  16. Not automating everything done to any production system.

  17. Using substandard tools because they are either free or commercial (people should use the best tool, not the one with the most CYA[5] factor).

    [5] Cover Your Ass

If you do any of this, then stop now. You are not helping, and it's just going to add even more problems in the future. I would say if you do any of the above practices, add another 10 hours for each practice.


Section 5.2. (Too) Many Options

5.2. (Too) Many Options

A common complaint about deploying Mongrel is the large number of available deployment options. You can use Pound, Pen, Balance, Apache, IIS, Nginx, LightTPD, Litespeed, and probably servers we've never even heard of before. You can put Mongrel behind firewalls and hardware load balancers. It's very daunting for the beginner.

What we're going to cover is a good, solid best-practice deployment involving Apache, Mongrel, and MySQL on separate servers. You could also combine these three pieces onto one machine, but to make things interesting, let's configure three machines.


Section 5.3. "Fake" Hardware Layout

5.3. "Fake" Hardware Layout

What we want is three machines that will serve one purpose each. One machine (web1) will run Apache and be the only Internet facing piece of hardware (and should go in a DMZ as shown in Figure 5.1). The next machine (app1) will run Mongrel and service Rails requests to Apache. The last machine (db1) will run MySQL to maintain the database.

Figure 5.1. Sample Network Configuration

[View full size image]

In reality you'll probably have to test this configuration and mix it up a bit. You might need a couple of Mongrel machines or you might be able to use just one machine for everything. For education purposes, though, we'll just stick with this as if it were real.

Additionally, you'll want to avoid configuring a firewall setup like in Figure 5.1 until you have a fully functioning and tested deployment and you know its performance. Firewalls are great for security, but when you're first setting up a deployment, they can complicate things unnecessarily. We've seen many deployments go wrong because of poorly configured firewalls, and knowing that the configuration worked before adding the firewalls is a big help. Reduce lurking variables (http://en.wikipedia.org/wiki/Lurking_variable) so you can focus on problems in isolation.


Section 5.4. Software Deployment

5.4. Software Deployment

Every operating system has different ways to deploy software, even different versions of the same operating system. Some have a choice between source build and package management. Commercial systems have restrictions on what is an approved software package and version and installing anything else means you don't get support.

Because of this, it's not really possible for us to explain how to install the software on your system. We'll give you the main components that belong on each machine, and leave it to you to make sure they get installed correctly.[6] We'll also include some common problems people run into when they install each component.

[6] One interesting thing to try in a heterogenous environment is the NetBSD pkgsrc. Their source builds work on nearly any POSIX system and don't interfere with your package system. You can get the exact same software installed on all systems even if they are different, and most of the software is current.

5.4.1. web1 (Apache)

  • Apache 2.2.x You need this more recent version of Apache so you can get the new faster and more capable mod_proxy_balancer. There was a defect in the 2.2.2 version of Apache that caused the proxy code to fail in some cases, so make sure your version is either higher numbered or doesn't have this problem. Our example in Section 4 uses 2.2.3.

  • Apache 2.0.x (alternative) Some operations departments do not allow the latest bleeding edge. The problem is that Apache versions before 2.2.x had a slow mod_proxy, which didn't support multiple backends. The alternative is to install Apache 2.0.x on the web1 machine, and then Balance on the app1 machine.

Once Apache is installed you'll want to make sure it's part of your system's boot process, that you can start/stop it, and that it's accessible on port 80. If you need SSL, then take the time to learn how to configure this and get your certificate as well. We won't be covering SSL configuration in this short cut.

5.4.2. app1 (Mongrel)

  • Ruby Make sure you include all tools necessary to build C extensions. For example, Debian requires build-essentials and several Ruby-related libraries.

  • RubyGems Some systems are now including this, but most people seem to install RubyGems from source download. You can get it from RubyForge http://www.rubyforge.org/projects/rubygems/.

  • Rails This is pretty easy with RubyGems, but systems are now also installing it through the package management.

  • Mongrel You of course need Mongrel, and there are also some systems that package it, so you may be in luck.

  • libmysqlclient You'll need to install whatever your OS calls the MySQL client libraries. Many times it's easier to just install MySQL (disk is cheap).

  • mysql gem You'll need the Ruby library from RubyGems to talk to MySQL, which also means you'll need the client libraries on the app1 box. This is best installed using gem install mysql rather than your package system.

  • balance (alternative) If you are forced to use Apache 2.0.x, then you'll need to put balance on this machine. Balance simply takes TCP/IP connections to one port and proxies them to a set of backend ports. It's fast and relatively easy to set up.

This is in addition to any software you need to get your Rails application working. My recommendation is that you install only the bare minimum to get a simple test Rails application working. Only after everything is configured and running well should you install your main application.

We typically install Ruby using the OS package system and then RubyGems if it is available. After that we install everything else using gem install. Depending on your purity you may try to install everything through your system's packages or mix it up like we do.

5.4.3. db1 (MySQL)

Obviously you'll want to install MySQL on the db1 machine and configure it properly. Hopefully you've been using MySQL this whole time and doing an install is second nature to you. It might be difficult to configure MySQL for production without some prior knowledge, but ask around for a consultant to help you out. There are also a few good books you can use as a reference.

We have to confess, though, that we usually install Webmin or phpMyAdmin to manage the MySQL server. System administrators who are die-hard script junkies hate these tools, but they encapsulate many system management best practices and can help someone who has limited system administration experience. Consider installing Webmin if you need to administer a machine and don't have much time to monkey around with configuration files. An additional advantage of Webmin is that you can cluster your management and manage a fairly large group of machines from one primary machine.

5.4.4. All Boxes

  • Ruby If you are looking to do Capistrano management of all three boxes, then you'll need Ruby on all three. Capistrano is super good stuff, so you'll want to make this leap very soon.


Section 5.5. Configuration

5.5. Configuration

5.5.1. Mongrel[7]

[7] Much of this section is based on documentation donated by Austin Godber to the Mongrel project.

We're going to use mongrel_cluster to manage your small set of Mongrel applications. We'll need the mongrel_cluster gem, and we'll first just set up the testapp so you can see how it's done. This will lay the groundwork for your real production application.

5.5.2. A Simple Test Rails Application

There's a bit of a chicken and eggs problem with this kind of setup. In order to test that your web1 machine works with your app1 machine, you need to have a working application on app1. But you can't really get your application working without web1 and db1 working. The best thing to do is to create a small "test" application and serve it manually until you're ready for the big time:

$ cd /var/www/apps
$ rails testapp
$ cd testapp
$ mongrel_rails start

You should then be able to visit http://localhost:3000/ and see the Rails test page. That means your Mongrel setup is working, but you'll want something that also shows Rails is working. Stop Mongrel with CTRL-C and do:

$ script/generate controller test

Edit app/controllers/test_controller.rb and add this to the TestController class:

def index
  render :text => "test"
end

Then start up Mongrel again and go to http://localhost:3000/test to see your little tester Rails application.

5.5.3. mongrel_cluster

If you haven't yet, stop your Mongrel instance with CTRL-C and install the mongrel_cluster gem:

$ gem install mongrel_cluster

Next we'll want to configure our testapp instance so that it is running three instances of mongrel starting at port 8000:

$ sudo mongrel_rails cluster::configure -e production \
   -p 8000 -N 3 -c /var/www/apps/testapp -a 127.0.0.1 \
   --user mongrel --group mongrel

This creates a configuration file mongrel_cluster.yml and puts it in /var/www/apps/testapp/config. Now you can start this cluster using:

$ cd /var/www/apps/testapp
$ mongrel_rails cluster::start

And of course the command cluster::stop will stop it. You should now (with the cluster still running) try to hit each port (8000, 8001, 8002) with your browser and make sure the test pages are present.

We're going to leave this running in this for now while we set up the web1 Apache server. Then we'll come back and finalize the install by putting your app on, configuring it, and putting it into the boot process for app1.

5.5.4. Apache

Refer to Section 4 to learn how to get your Apache 2.2.x install up and running. If you're using virtual hosts then make sure they are configured correctly and serving the files for your domain name.

Once you have everything running you'll need to change your configuration to point your virtual host at the app1 servers:

<Proxy balancer://mongrel_cluster>
   BalancerMember http://app1:8000
   BalancerMember http://app1:8001
   BalancerMember http://app1:8002
</Proxy>

Change app1 to be either your real hostname or the IP address. Restart your Apache instance according to your system's process restart method and hit http://web1/ to see if your application comes up. If not, double-check the following:

  1. Can you access ports 8000, 8001, 8002 from web1 to app1? Use curl http://app1:8000/ to test it quickly.

  2. Are you using fancy rewrite rules? Disable or remove your rewrite rules for now. They aren't doing you any good at this stage since the app1 server is in a different location and they only add complexity.

  3. Can you even access http://web1/ normally?

  4. Is it plugged in? No, seriously, are all the machines turned on and networked properly? Hopefully you aren't trying to do this in the midst of a firewall configuration, so try to ssh to each machine and make sure they are accessible.

  5. Did you check the Apache server logs for errors?

  6. Did you check the Mongrel server mongrel.log and production.log files? There are usually clues in there.

  7. If you get really desperate, shut down your cluster and refer to Dash-Bee Logging in Section 7, then shut down your testapp cluster. With the cluster down you can start up one Mongrel instance on port 8000 with: mongrel_rails start -e production -p 8000 -B. Run this and then try accessing your Apache server again. See if you get any information in log/mongrel_debug/rails.log about the requests.

5.5.5. MySQL

You now have your web1 server balancing against your app1 Mongrel cluster and you're ready to get your database configured on db1. First thing, though, is you should have been using Rails migrations this whole time. It was mentioned in Section 5.1 as something you should do, but it's pretty much a requirement at this point. Later you'll have to deploy your schema to MySQL and then manage versioning of the schema, so if you want to be a hero and do them manually, make sure your SQL scripts are set up properly.

If you don't have migrations, you're on your own (and probably very much in trouble). You'll have to go through the MySQL configuration and just make it up as you go.

What you need to do is first make sure MySQL is running and configured properly and that you can use the mysql tool to connect to localhost on db1. Once you can do that, follow these steps:

  1. Set up the MySQL user you plan to use for your production deployment on db1.

  2. From app1 use the mysql tool to try connecting to db1 and run a few queries. You may have to install this tool on app1 if you went the purist's route and only installed the mysql client libraries.

  3. Once these two machines are talking mysql protocol, you're ready to install your production application and get it working.

Until you can connect manually from app1 to db1, you shouldn't continue. Permissioning and user management in MySQL is rather weird and painful (well, with any DBMS), so we usually break down and install phpMyAdmin or webmin to manage it. Yeah, we're evil.

5.5.6. Last Step: Your Application in Production

These instructions are intended to get you up and running quick with a basic best practice deployment, but also teach you how everything is installed and managed. In reality everyone who's half smart about anything uses Capistrano to do their deployments. Capistrano is excellent software that uses Rake and ssh to automate application deployments. It can manage fairly large installations, restart servers, roll back failed deployments, and provide status as well.

The problem with Capistrano is that until you've done one deployment manually it's difficult to use Capistrano to automate your installation. We usually do the first deployment by hand in a new location, and then we step back and plan the automation with Capistrano. As you gain experience doing this you'll be able to start off right away with Capistrano, but for now we're going to do this first deployment by hand.

It is a sin of the highest degree to continue doing deployments manually once you've done your first one by hand. Capistrano is a fantastic tool that is easy to understand if you know even a small amount about build tools (Rake, Make, SCons, nmake). Taking the time after this deployment to automate your work will save you much pain and anger in the future.

5.5.6.1. Get Your Application On

You've got two approaches to follow to get your code onto the app1 server:

  1. From your development system just copy it over to app1:/var/www/apps/myapp using scp or rsync. You should really only do this if your app1 machine cannot access your version control repository because of firewall issues.

  2. Log onto app1 and checkout your application from version control directly into the /var/www/apps/myapp directory. This is the preferred way since this is how Capistrano does things. In fact, only do option #1 in desperation since you'll have to change things later to accommodate Capistrano.

Let us assume that you've got your code in Subversion and you're going to set up your application just like your testapp:

$ cd /var/www/apps
$ svn co http://mysvn/repository/myapp
$ sudo mongrel_rails cluster::configure -e production \
   -p 8000 -N 3 -c /var/www/apps/myapp -a 127.0.0.1 \
   --user mongrel --group mongrel
$ cd testapp; mongrel_rails cluster::stop
$ cd ../myapp; mongrel_rails cluster::start

Your application should start up and you should see tons of errors. What you must do now is configure this deployment so that it uses your db1 database, runs your rake db:migrate task, and completes any configurations your application specifically needs. It might be a good idea during this stage to not use mongrel_cluster but to just run the application manually with mongrel_rails start -p 8000 and test against http://app1:8000/ until the application works.

Halt! Before you start to change the files you just checked out, take a step back. You have a revision control tool that helps you keep track of your source changes, but if you edit files outside of this process then you'll run into conflicts later. Since you've checked the source out, you have three choices that make sense:

  1. Consider your checkout a "development action" and plan on checking your changes in when you are done. This means you have to coordinate with the other developers and make sure your changes don't break their build.

  2. Consider this a "deployment action" and that it should remain untouched. In this case you would do development on another machine, check the changes in, and then svn update on the app1 machine.

  3. Don't do a svn co and instead do an svn export to get a copy of the source that can't be changed. This is the safest approach but also kind of a pain.

Our general process is to do #1 for the first deployment since it's quicker and more direct, then when we automate with Capistrano we use #3 to keep things sane and safe.[8]

[8] When you svn co your source, it leaves the .svn directories lying around where your Web server can serve them to the world for proper viewing. This can lead to security problems, especially if you do silly things like put passwords into your source or have other sensitive items viewable.

5.5.6.2. Systemize Your Deployment

The mongrel_cluster gem comes with an /etc/init.d startup script that works on most linux systems. There's a small process you can do so that your newly minted and functioning application will be started whenever the machine reboots.

$ mkdir /etc/mongrel_cluster
$ ln -s /var/www/apps/testapp/config/mongrel_cluster.yml \
  /etc/mongrel_cluster/testapp.yml
$ cp /path/to/mongrel_cluster_gem/resources/mongrel_cluster \
  /etc/init.d/
$ chmod +x /etc/init.d/mongrel_cluster

Then you'll need to configure your system to run /etc/init.d/mongrel_cluster on start/stop and you're set. You should try rebooting your machine to make sure the application actually does start properly.

5.5.6.3. The Final Get-Together

The only remaining task is to actually make sure all the pieces work. It's a good idea to test each component in order from back to front, and then test them as one whole. If you were smart you'd also automate this test process and include it in your post-production deployment process.

5.5.6.4. Cheap Simple Caching

Normally you'll need to set up Apache mod_rewrite rules of varying complexity levels to take advantage of Rails' page caching. Since you have a front-facing web1 server you'll have to do some research to find a way to share disk with app1. This is fairly complex but there is a quick and dirty way around it for many people's applications: mod_cache.

<IfModule mod_cache.c>
  LoadModule mem_cache_module modules/mod_mem_cache.so
  CacheEnable mem /
  MCacheSize 4096
  MCacheMaxObjectCount 100
  MCacheMinObjectSize 1
  MCacheMaxObjectSize 2048
</IfModule>

This configuration is only a sample taken from the official mod_cache documentation, but it demonstrates the trick. This will cache the majority of your content and will work as long as your content doesn't require immediate updating. There are ways to also exclude some locations, to cache to disk, and to control the cache organization. Using the memory cache often works well, as you can just bounce the server after a deployment to reset it, but the details of your application will determine if it is good fit for you.

Another trick you should investigate is using an asset server in Ruby on Rails. You make this setting in config/environments/production.rb:

config.action_controller.asset_host = "http://assets.web1"

This tells Rails to write URIs for assets (Javascript, images, stylesheets, etc.) so that they point to a server named "assets.web1." If you then set up a virtual host for web1 to map this, you will be able to put a majority of your assets on that server and have them served directly rather than from app1.


Section 5.6. Care and Feeding

5.6. Care and Feeding

5.6.1. Monitoring

You'll want to install some form of monitoring for all of your systems. Section 5.6.2 lists several good open-source monitoring tools for both host-based and network-based monitoring. You should also take a look at monit as a means of watching your systems internally (meaning not from an external server the way nagios would do it). Others prefer to use runit to manage their processes reliably, and it should work well with monit.

5.6.2. Security

You can install a few small pieces of software right away and get a massive security improvement.

  • samhain A fantastic little bit of software that monitors your system for host-based intrusion. It can detect file changes, root kits, logins, logouts, privilege escalation, and use a secure central server or e-mail for reporting. It's also great for finding rogue admins and developers who change things without telling anyone.

  • psad This or portsentry will watch your system for portscans (people trying to find out what your systems are running) and then block them actively. This is a very effective way of stopping most attackers since the first thing they do is scan your machine.

  • mod_security This is a module for Apache that lets you do very capable security controls at the HTTP level. You can sometimes use this to protect against new exploits found for Rails without having to upgrade right away.

  • snort Another fantastic tool that watches your network for potential attacks. You can put it on a separate little machine (running OpenBSD if you're ultra hardcore) and it'll detect network level intrusions.

  • syslog-ng A great way to centralize your system logging. A centralized log infrastructure makes it easier to recover from intrusions, find out how they were done, and makes it easier to monitor your systems.


Section 6. Extending Mongrel

It is surprisingly trivial to extend Mongrel to suit your own needs. You can write your own commands, request handlers, filters, and quickly configure them as needed. Unlike other systems, though, you don't have to know anything mentioned in this section to use Mongrel. All of this information is "behind the dirty curtain" and only for the very adventurous types who need it.

It's possible to have your first handler up and running in about ten minutes, and fairly extensive plugins can be written within hours or days. It's not as nice as Rails, Nitro, or Camping, where you can create whole applications in days. When you extend Mongrel, you're working at the bare-metal death's edge and have to do more work to keep things straight. The advantage is that you have full control over what happens and you can squeeze even more speed out of the beast.


Section 6.1. Mongrel's Architecture

6.1. Mongrel's Architecture

What makes Mongrel so extensible is its simplistic architecture that gets the job of cooking and shuttling HTTP requests around the system done. Mongrel would probably make the Enterprisey folks scream since it is obviously too simple to scale. Ironically this simplicity is exactly why Mongrel is fast and scales, but it's also your secret weapon when extending and embedding to your own crazy ideas.

Zed Sez

Simpler always wins when you are writing software. I didn't say "simplistic" but "simpler," meaning when compared to another solution. I find that too often programmers get too caught up in building grand palaces of crystal and gold with incredibly intricate inner workings. They love impressing their friends with the equivalent of the "SUV" of software architectures and the more boxes, lines, and color in the diagram the better. Even better if it has to go on multiple walls.

What we have now is a crisis where the coders of the world value complexity and intricacy over simplicity and directness. They would rather build a "Rube Goldberg Architecture"[9] full of indirect puzzles that interact in a chain to accomplish nearly nothing. They then wonder why their software is buggy, crashes, has security holes, and requires a coterie of 10 developers to gather before it works.

[9] (Pitagora Suiichi) for the Japanese readers (http://en.wikipedia.org/wiki/Pitagora_Suicchi).

I've always had a different aesthetic sense when I write my software. I value simplicity and directness and try to write software that follows this approach. I jokingly call it the Shibumi School of Software Structure. All I do is apply this rule: When given two possible designs with equal end results, pick the simpler one. I then ruthlessly strip the solution down to its finest elements, but no more.

Keep this in mind when you've got an architecture or solution designed that you feel is fantastic. Are you actually impressed with the solution or its complexity? Are you sure there's not a simpler way to do the same thing, even if it seems "ugly"? Is it possible that all the people who told you this is how software is designed are wrong?

These are the types of questions developers should be asking. Until they do, we're just going to get larger and larger architectures that don't really do anything. Rube would be proud.

Mongrel's architecture is composed of the following key players:

  • HttpServer The core center of Mongrel that processes incoming HTTP requests and coordinates all the other classes to complete them.

  • HttpRequest Created by the HttpServer after the incoming HTTP headers and body are processed, it contains all the information your HttpHandler needs to do its work.

  • HttpResponse This is crafted and then handed to the HttpHandler as the way the handler builds its response to the client.

  • HttpHandler The class responsible for processing cooked HttpRequest and HttpResponse objects in order to complete a request. This is where you put your logic.

  • URIClassifier You don't really need to use this, but it is responsible for mapping request paths to chains of HttpHandlers for processing. This is what your handler gets registered with.

That makes up the main classes being used. Yes, that's all of them. There are a few others that help setting up new handlers, help package them as plugins, etc. but you can get by with just these five, and actually you don't have to know how HttpServer and URIClassifier work.

How all of these classes work together is best illustrated in Figure 6.1.

Figure 6.1. Mongrel Request Processing

The only thing missing from this diagram is that HttpServer does this for chains of handlers that are registered at different URI paths. To make sure you understand the diagram, we'll go through a typical request.

  1. HttpServer accepts a new client and parses their headers. At this point the server will reject clients if their headers do not parse.[10]

    [10] This is the origin of the "BAD CLIENT" error message and is also the reason Mongrel can resist many attacks and still stay alive.

  2. The results of parsing are passed to a new HttpRequest, which becomes the HttpRequest#params variable used by HttpHandler to process forms and queries.

  3. HttpRequest continues processing the client in order to build up the body of the request. The body is always stored in an IO object, but it might be a temporary file or a StringIO depending on the size of the body.

  4. Once the HttpRequest is constructed, it and a HttpResponse object are passed to the first registered HttpHandler to do the processing.

  5. The HttpHandler#process does its magic and builds the response using the HttpResponse#start method call to pass back headers and the body content. Remember that you are at bare metal so you must explicitly specify most headers manually.

  6. Finally, when the HttpHandler#process method is done, the server checks the HttpRequest it passed and determines if the handler chain is done. If not, then it loads the next handler and continues until there are no more.

The only missing piece is how the HttpServer knows which handler gets which requests. As mentioned in the list of important classes, URIClassifier keeps a mapping of all paths and the HttpHandler chains that are responsible for processing them. HttpServer just looks them up in URIClassifier like a hash map and it's done. You actually won't work with this directly except for simple registration calls through Configurator#uri.

6.1.1. Documentation

Now that you understand Mongrel's deceptively simple architecture, you should take a quick break to browse through the Mongrel Web site's RDoc documentation. Much effort was put into documenting every class in Mongrel so that developers can be self-sufficient. You can find Mongrel's documentation at the Mongrel RubyForge RDoc pages (http://mongrel.rubyforge.org/rdoc/).


Section 6.2. Handlers

6.2. Handlers

Let's jump right into things and make a very simple handler that only reflects its parameters back to the Web browser. Put this code into a file named mongrel.conf:

class ParamDumper < Mongrel::HttpHandler
  def process(req, resp)
    resp.start do |head, body|
      head['Content-Type'] = 'text/plain'
      body.write(req.params.inspect)
    end
  end
end
uri "/dumped", :handler => ParamDumper.new

Simple enough. We just say ParamDumper inherits HttpHandler, create a process method, and use the HttpResponse object to dump the HttpRequest#params variable via inspect. The last line actually calls Configurator#uri, but there's a bit of voodoo going on that we'll explain later. For now just understand that's how you attach a handler to a URI path. Notice also that we set the "Content-Type" to "text/plain" using the head variable.

Loading this for now will be as simple as:

mongrel_rails start -e production -S mongrel.conf

The -S option (which we first covered in Section 4) basically means "run this file as a configure script." A configure script is a Ruby script, but it is run within the context of an already configured Configurator object. This means that all the functions in Configurator like uri can be called as if they were global functions. The Configurator#uri function is the one you'll be calling the most.

When you get this running and it is working you should be able to go to http://localhost:3000/dumped with a browser and it'll spew the results to a text page. That's it, you've just written your first HttpHandler.


Section 6.3. Filters

6.3. Filters

Sometimes you don't really want to handle a request, but you want to be able to modify the results coming in or going out of the server dynamically. Mongrel's HttpHandler classes can do this since they by default act in a chain. In this situation you are ostensibly writing a "filter," which is really just a handler that modifies its results.

This brings up another point: handlers don't have to do any processing. It's perfectly legal for them to make a decision and do nothing, allowing the remaining handlers do their thing instead.

For our first filter we're going to write a class that removes any HTTP_USER_AGENT headers coming in, and then another filter that deflate compresses (deflates) its output. First the UserAgentRemover:

class UserAgentRemover < Mongrel::HttpHandler
  def process(req, resp)
    if req.params.delete "HTTP_USER_AGENT"
      STDERR.puts "Removed a user agent"
    end
  end
end
uri "/dumped", :handler => UserAgentRemover.new, :in_front => true

Pretty simple, we just delete it from the params hash and log a simple message. If you need to do simple logging then use STDERR since this is written to the log/mongrel.log file or the console and isn't hijacked by the framework. If you need to do serious logging then use Ruby's Logger.

The above code assumes that you put this in the mongrel.conf after the first uri call. This is why this block of code as a :in_front parameter, which tells the Configurator that you want this handler moved to the front of the chain.

Run your mongrel_rails again and hit the /dumped URI to see that HTTP_USER_AGENT is actually removed.

This ability to modify requests as they come in with simple handler chains means that you can perform security checks, clean up requests, do file processing, and other tasks prior to Rails or another framework getting the results. We'll now write another handler that does nearly the same thing but it takes results and translates them into deflate form. Add this next block to your mongrel.conf file:

[View full width]
 require 'zlib'
 class DeflateFilter < HttpHandler

    def initialize(ops={})
      @options = ops
      @always_deflate = ops[:always_deflate] || false
    end
    def process(request, response)
      accepts = request.params['HTTP_ACCEPT_ENCODING']
      # only process if they support compression
      if @always_deflate or (accepts and (accepts.include? 'deflate' and not response
  .body_sent))
        response.header['Content-Encoding'] = 'deflate'
        # we can't just rewind the body and gzip it since the body could be an attached file
        response.body.rewind
        gzout = StringIO.new(Zlib::Deflate.deflate(response.body.read))
        gzout.rewind
        response.body.close
        response.body = gzout
      end
    end
  end
uri "/dumped", :handler => DeflateFilter.new

Obviously this handler is more involved, but it's a good example of doing something useful with HttpHandlers as filters. Let's walk through it:

  1. It shows how to set up some default options and write your own initialize. HttpHandler is very thin and you can modify the initialize method however you want. In this case we allow a Hash parameter style and let the user configure the handler.

  2. We then peel out the encodings the client accepts so we can see if they accept "deflate."

  3. If they accept it then we set the "Content-Encoding" to "deflate" and we deflate compress the body using Ruby's Zlib::Deflate API. Notice that we have to do some shuffling of the response body. If we were to gzip the response directly, and the previous handler had attached a file to send, then we'd be gzip their files silently.

  4. Instead we deflate their file into a StringIO, detach it and send it on, closing the original.

This filter isn't put on the handler chain with :in_front so it will happen after ParamDumper. If we were to draw a picture of this setup it'd look like Figure 6.2, even though in the mongrel.conf we added them in a different order. This is necessary since, if you need to put a filter in front of say RailsHandler, then :in_front lets you do that.

Figure 6.2. Handler Chain Processing

Previous Page
Next Page


Section 6.4. Distributing Your Plugins as RubyGems

6.4. Distributing Your Plugins as RubyGems

While it is easy to type your handlers into a -S mongrel.conf, this isn't an easy way to distribute your handlers and it'll break down if you need extras to go along with them. The preferred way to package up a Mongrel plugin and distribute it to people is to use the GemPlugin system.

GemPlugin is a dynamic plugin loading system based entirely on RubyGems. It's very simple to use and comes with a full project generator to get you started. GemPlugin makes it possible for people to extend Mongrel without including their source in Mongrel, and to distribute them in a way that's familiar to most Ruby users.

Explaining how GemPlugin works probably isn't going to be helpful since it digs into the RubyGems API. Instead, we'll do a quick demonstration of the PFM[11] in action. Take your install of Mongrel and make sure you remove the mongrel_cluster gem:

[11] Pure F***ing Magic. The best kind of magic (http://en.wikipedia.org/wiki/PFM).

$ sudo gem uninstall mongrel_cluster
$ mongrel_rails
Usage: mongrel_rails <command> [options]
Available commands are:
 - console
 - restart
 - start
 - stop
 - status
Each command takes -h as an option to get help.

Notice the list of commands when you run the mongrel_rails command? Add the mongrel_cluster gem back to your system and run mongrel_rails again and you'll see this list of commands:

- cluster::configure
- cluster::restart
- cluster::start
- cluster::stop
- console
- restart
- start
- stop
- status

Now you get all these new commands like magic. No, like PFM. How's that work? The simplest explanation is that GemPlugin goes through the list of gems installed in your system and tries to find gems that depend on the mongrel gem. When it does, it requires the gem and runs an init.rb file if present.

With this simple system, Mongrel will then autoload any plugins you install as gems. We'll go through two examples of doing a GemPlugin in the next few sections.

6.4.1. Using Rubyforge to Share

When you've crafted a nice GemPlugin and you want to share it with the world, you should pop over to http://www.rubyforge.org/ and register a project. When you do this you can upload your .gem file to their file service and then the RubyGems mirrors will pick it up.


Section 6.5. Commands

6.5. Commands

Our first example of using GemPlugins will be to write a new command for Mongrel to make it quack like a duck. Matt has an affinity for quacks, so it seemed appropriate. Mongrel's commands are implemented as GemPlugins so they can be added easily without modifying Mongrel's source directly.

6.5.1. Generate the Project

GemPlugin comes with a command called gpgen, which generates a skeleton project ready for you to use. You run it like you would the rails command to start a project:

$ gpgen duck

This will print out the files being created, most of which you won't have to edit. The important files are:

  • duck/COPYING Your license or copying restrictions.

  • duck/lib Where you store your source for the plugin to use.

  • duck/Rakefile Builds your stuff.

  • duck/resources A place to put files your plugin might need.

  • duck/lib/duck/init.rb Required to initialize your plugin.

  • duck/resources/defaults.yaml Default configuration options.

We'll be skipping the defaults.yaml and resources for now. They let you package additional configuration information with your gem plugin, which GemPlugin will also load. You should refer to the GemPlugin RDoc[12] for more information.

[12] The easiest way to get to any gem's RDoc is to run gem_server and access it with your browser.

6.5.2. Set Up the Project Files

In order to set up our duck command, we need to tweak the Rakefile and put some code in the init.rb, then simply install the gem and go. First the Rakefile needs some lovin':

version="0.1"
...
setup_gem(name, version) do |spec|
  spec.summary = "The Mongrel Duck Quacker"
  spec.description = "What is the sound of a duck typing?"
  spec.author = "Matt Pelletier"
  spec.add_dependency('gem_plugin', '>= 0.2')
  spec.add_dependency('mongrel', '>= 0.3.13.4')
  spec.files += Dir.glob('resources/*/')
end

This will configure the gem task for Rake so that it is properly attributed, has the right dependencies for GemPlugin (both gem_plugin and mongrel), and includes any resources (we have none). Try to build it to make sure at least this much was done.

$ rake
$ gem install pkg/quack-0.1.gem

You should see some really boring build stuff and then in the pkg directory is the gem. Install and uninstall it to make sure it works.

If you were serious about distributing this gem you'd also want to update the COPYING and LICENSE files (not sure why this needs both) and add any other dependencies you need.

6.5.3. Write the Initialize File

With our build working, we now just need to toss some code into the init.rb file to get things in motion. This is where we take a slight diversion into "Why the Lucky Stiff" land. Mr. Stiff wrote a really cool article about hijacking contexts. Using this, GemPlugin can have a "second name space" that you register your plugins with outside of the usual Ruby Module system. This is handy for finding plugins by a sane category system without trolling through all the registered classes.

The best way to show you how this works is to just show you the Duck command's code and then explain it:

require 'mongrel'
require 'gem_plugin'
class Duck < GemPlugin::Plugin '/commands'
  include Mongrel::Command::Base
    def configure
    options [
             ['-a', '--annoying LEVEL', 'Just how annoying?', :@annoying, 1],
            ]
  end
    def validate

    @annoying = @annoying.to_i
    valid?  @annoying > 0, "--annoying must be an integer"
    return @valid
  end
  def run
    @annoying.times {|i| puts "QUACK! "}
  end
end

We start off by simply requiring the libraries we need, and then we declare the Duck class. See that odd syntax in that GemPlugin::Plugin "/commands" bit of code? That's the weird "register this class under /commands" way of making a GemPlugin. Using this class declaration style (which is not standard Ruby in any way), Mongrel is able to ask GemPlugin to give it a list of all classes registered under "/commands" and then run them as needed based on names. It's quite voodoo but it does have a great usage in this case.

The next step is to include the Mongrel::Command::Base module to give your class all the features of a Mongrel command. This hooks up a bunch of extra code, so you should go read the RDoc for this in the Mongrel documentation.[13]

[13] Remember, easiest way is gem_server.

Once we've got this basic setup going we just need to write three functions: configure, validate, and run. They are run in this order and do basically what they say. The contents of configure are almost always setting up options, but you can do other stuff that needs to work before validation.

The validate function then uses a set of handy valid? methods to check for conditions, reports errors and aborts if anything doesn't match. The return value of this function should be @validate (it just should be, don't ask why).

With that all over with, the run command should be set up, have all its settings validated, and should be ready to go. As you can see, we print out a series of annoying "QUACK!" messages in honor of Matt.

6.5.4. Install and Play

This part is very simple; you just need to run rake again and install:

$ rake
$ gem install pkg/quack-0.1.gem

Now when you run mongrel_rails, you should see your fancy duck command, and you can then do:

$ mongrel_rails duck -a 10

Wow, isn't that annoying? Quack, quack, quack all day long.


Section 6.6. Handlers as GemPlugins

6.6. Handlers as GemPlugins

Handlers can also be GemPlugins, and they are the preferred way to distribute your handlers. If we take our UserAgentRemover handler and turn it into a GemPlugin, the code would be:

class UserAgentRemover < GemPlugin::Plugin "/handlers"
  include Mongrel::HttpHandlerPlugin

  def process(req, resp)
    if req.params.delete "HTTP_USER_AGENT"
      STDERR.puts "Removed a user agent"
    end
  end
end

uri "/dumped", :handler => plugin("/handlers/useragentremover")

The only change made to the code was to do the usual GemPlugin::Plugin voodoo, include the HttpHandlerPlugin module, and then use the Configurator#plugin function to set the :handler => plugin() setting.

6.6.1. More Examples

The Mongrel RubyForge project Files section has several other examples of using the GemPlugin system to create distributions of your handlers and commands. You should take a look at mongrel_upload_progress as a full example of implementing a useful set of handlers as GemPlugins.


Section 6.7. Advanced Handlers

6.7. Advanced Handlers

Handlers have a "notifications" feature that has been in Mongrel since 0.3.13.3, but which will receive a major change in a version to come after 0.3.13.4. Rather than explain how to use the featuresince it will most likely changewe'll instead refer you to the mongrel_upload_progress plugin that is available as a stock Mongrel gem.

The mongrel_upload_progress plugin utilizes the notification callbacks to update a visual progress feedback for users uploading files. The example includes Mongrel code and a small Rails example.


Section 6.8. Finding More Plugins

6.8. Finding More Plugins

Finding Mongrel plugins isn't as organized as the Rails plugin system. Mongrel comes with a few that you can see on the RubyForge project Files section. You should refer to the mailing list (http://rubyforge.org/mailman/listinfo/mongrelusers) and watch for any messages announcing new ones.


Section 7. Debugging

Mongrel is designed to help developers get things done. While it also makes deploying Ruby Web applications easier, it is the debugging features that developers love the most. Most folks are not accustomed to having this level of debugging from their Web server, though, so this section will cover all the various things you can find out about your Rails application using Mongrel.

The debugging capabilities in Mongrel were born out of frustration and necessity. It's frustrating tracking down certain types of defects in a Rails application, and when people have these problems they blame Mongrel. It's necessary to have strong debugging in order to determine if the problem is caused by Mongrel or by other code.

Using these techniques makes it easier to find memory leaks, open file leaks, various thread issues, and problems with proxy configurations. The tools aren't the greatest but they'll help quite a bit in a pinch.


Section 7.1. Available Tools

7.1. Available Tools

The primary debugging method in Mongrel is logging. By enabling different options or setting up special handlers you can get Mongrel to log helpful data to files found in the log/mongrel_debug directory. Since Mongrel is easily extensible, you can also use the current handlers as the basis for your own specialized debugging filters.

7.1.1. "Dash-Bee" Logging

The main logging people use is "Dash-Bee" or just -B for the mongrel_rails option that enables all of the available logging filters. It's very easy:

mongrel_rails start -e production -B

Once you run your Rails application this way, you'll get tons of information from the log/mongrel_debug directory. Notice that we started in production mode. This is done because in debuggin mode Mongrel is painfully slow, and also Rails development mode does quite a few nasty things to Ruby, which can skew your debugging output.

Right away Mongrel will start logging the following:

  • accesses Apache-style access logs statements get printed to the console for you to see what requests are coming in.

  • objects.log Used for debugging memory leaks. For each request a listing of every class currently active in object space, the number of objects of that class, the delta when compared with the last run, and a bit of information on the average lengths of objects that answer to the length method.

  • files.log Used for finding file leaks, this log contains a list of each currently open file and how many times it's been opened without a close. File leaks are very crippling to Mongrel since Ruby 1.8.x can only handle about 1,024 depending on the system.

  • rails.log Many times what you think Rails is receiving is different from what your HTTP client is sending. This log file will tell you exactly what parameters Rails is receiving for each request.

  • threads.log Library authors love messing with the internals of Threads (specifically the Thread.current[ ] hash). This can cause unexpected behavior and memory leaks. Mongrel will dump the active threads and their Thread.current contents.

Each of these log files will be written to the mongrel_debug directory for each request. There isn't any locking or coordination done on this logging so you should avoid really thrashing Mongrel in -B mode to get better results.

7.1.2. USR1 Logging

While -B can help you do deeper debugging "back at the lab," it is not viable for many production situations since it's slow and does voodoo hackery on Ruby's internals. To help with two common problems while in production mode, Mongrel has a lighter type of debug logging called USR1 Logging. This mode is a toggled mode that simply flips a global variable to tell Mongrel to log more information when there's an exception, and to report on a few more key elements.

First, in USR1 mode Mongrel will dump the full request of any client that violates the HTTP specification. You can then use this to see if you should block this client, fix the client, or request that Mongrel's parser get an update.

Second, it logs at 10-second intervals the number of requests waiting for different Rails actions. This helps you spot the Rails actions and controllers that are blocking your Mongrel processes from continuing. It is a very light logging that will only go to your mongrel.log file (or console if not in daemon mode).

Using USR1 logging is very simple:

killall -USR1 mongrel_rails

On POSIX systems this will tell all your Mongrel processes to start doing USR1 logging. It doesn't do a restart, stop any active connections, or anything that should impact your running application. It's also not a performance hit (maybe 1% or so).

Once you've gathered some information you can use, then you simply run the killall command again and Mongrel will toggle the logging off.

Zed Sez

Have you ever worked with that one programmer who insists that his code is absolute perfection handed from Gaiai in little baskets woven from gold? You know the guy. He's the one that's constantly using big fancy words like "efferent coupling" and "semantic inference," and claims Haskell/Scheme/Lisp/ML is the greatest language ever.

You probably also have experienced when Super-Software-Architect-Master-Coder writes crap-buggy code yet defends it to his death. He'll insist that everyone else change to make his junk work. Quite often these so-called "master coders" are actually fairly incompetent and hide behind extensive education to cover their constant failures. I've worked with several who would rather quit than admit that they were wrong. Very unprofessional.

These people's problem is they suffer from Potpourri Turd Syndromea belief that their you-know-what don't stink and smells like fine dew on freshly cut grass. Whenever there's a bug, they go running like kids in a candy store to other people's code trying to find fault and just assume that it's nothing they wrote.

The developers that I consider good always assume that what they write is the first suspect when a bug comes through the door. They will spend quite a bit of time building evidence to rule out possible causes, try to find causes of defects, and have no problem admitting when their designs are flawed.

When you run into a problem with your application, always assume it's your fault first. Mongrel's not perfect, but its code is minuscule compared to the size of Rails and most likely even your own application code. Mongrel also powers many large and medium deployments without any problems. If there's an error, the evidence already says it's in your code, so bite the bullet and start investigating it as if it's your problem.

7.1.3. Customized Log Configuration

A more advanced usage of the logging capabilities in Mongrel is to use a mongrel.conf file and the -S option to configure just one of the debugging handlers. This lets you log only the part of the request you need to determine a problem while letting Mongrel run a bit quicker. All you have to do is create a mongrel.conf file with the appropriate Configurator#debug function call.

As an example, we'll configure just access logging and rails.log parameter logging:

debug "/", what = [:access, :rails]

Place this line in a mongrel.conf and then tell Mongrel to run this file with the -S option:

mongrel_rails start -e production -S mongrel.conf

You could configure your own custom handlers in a special mongrel_debug.conf that you run only in desperation to get special information.


Section 7.2. Common Scenarios

7.2. Common Scenarios

There are quite a few places where your code can go wrong, and since you wrote it, it's your problem, but here are the most common problems that Rails applications run into. If these don't help you solve your problem, remember to check out the Mongrel mailing list and the FAQ on the Mongrel Web site (http://rubyforge.org/mailman/listinfo/mongrel-users and http://mongrel.rubyforge.org/faq.html).

  • Leaking Files Leaking files in Rails is common for people who come from other languages where open files will be closed by the GC. Ruby's GC isn't quite that smart, so many new Rails programmers leak files by accident. Using the files.log debugging will show you right away what you're doing. Best way to avoid it is to always open your files and use them in a block:

    open('thefile.txt') {|f| f.read }

  • Leaking Objects Leaking in a language like Ruby basically means that you're storing objects in places where the Garbage Collector (GC) can't get to them or can't release them. The object.log will help you spot objects that are getting leaked if you simply look for any class whose delta goes up but never down. You can avoid leaks of this kind by utilizing WeakRef in any internal caching you do, and by not storing things in class variables (@@likethis).

  • Nobody Likes Mongrel Or, everything's configured right, but requests just aren't getting to Mongrel. The problem is difficult to debug without actually looking at the network traffic, but looking at the rails.log will tell you if requests are coming in and being formed correctly. If that doesn't work then you'll need to use a packet sniffer to really find out what's going on.

  • Jammed Requests This is also known as "stopped mongrel" or "stuck mongrel" and happens when you have a Rails action that seems to never finished. Common causes are having multiple Mongrel processes all trying to use common resources without proper locking. Turning on USR1 logging temporarily in production will help you figure out which action is causing you problems.

  • EMFILE If you open too many files for either Ruby to handle or for your system's restrictions then you'll get a "too many open files" exception in your Rails codes or even at other random locations. Mongrel will log these in the mongrel.log as attempts to kill Threads that are "too old," but really the problem is you have a Rails action that's causing the damage. What you want to do is turn on USR1 logging and -B logging to get the files.log. Then look at which request opens the most files and find a way to tone it down.


Section 7.3. Other Tools

7.3. Other Tools

When trying to find out what is going on with your server, many times you have to use an external tool. There are tools to help you debug from the client's perspective, from the server's perspective, and finally from the network's perspective.

7.3.1. Client Debugging

When you are not sure if the problem is in the client or the server, simply using one of these tools can really help you rule out the client.

  • FireBug A very nice tool that lets you view nearly everything about the requests in FireFox.

  • curl A good command-line tool that you can use to snap headers and data from a server and try out quick parameters to see if what you're getting is really what a browser should see. Try -I to see the returned headers (a nifty way to verify which Web server is getting hit, Mongrel or your front-facing one).

  • RFuzz An HTTP client with fuzzing[14] and client testing capabilities that Zed built to help test Mongrel (and also to see just what kinds of crap well-known HTTP servers and Web sites would accept. You'd be amazed). It is an excellent tool for thrashing your own application to find holes, and it happens to be a tight little HTTP client should you want one. It is as compliant an HTTP 1.1 client as Zed could build.

    [14] Fuzzing is a technique where you try to break applications by handing them mountains of random data. It works amazingly well.

  • httperf As well as being able to do performance testing, httperf can help you spot stability problems while under heavy loads, and simulate user sessions.

7.3.2. Server Debugging

If you know the problem is with the server, you may still not know what component is responsible. It could be Mongrel, the fronting Web server, load balancers, the operating system, or a bad network configuration. These tools help you test servers in various ways.

  • Lsof "list of open files" tells you which processes have which files, sockets, domain sockets, etc. open on your server. It will also tell you what state each socket is in and which process owns it. It's kind of slow and only really works right when run as root, but it's a great way to find out who's leaving files open and what files those are. The main benefit of lsof over something like netstat is that lsof works the same everywhere.

  • netstat Although usually very specific to each particular OS and even different OS versions, netstat will dump tons of different information about your system's network configuration, including routing, packet transfer statistics, open ports, and who's currently connected to which processes.

  • strace You gotta be desperate to turn this on, but if you see a process that's hung or eating all of your CPU, then running strace (also known as truss on some systems) against it will print out the system calls the program is making. Many times this is all you need to figure out what's causing the problem.

7.3.3. Network Debugging

When you have no evidence that the client is causing the problem, and you are convinced the server is configured correctly, then you either blame green Pygmy elves or you break out various network debugging tools.

  • traceroute It's an old one but good one. Don't waste hours or weeks trying to find out why a server is slow before you run this tool and make sure the pipes are all running well.

  • mtr A nicer little graphical version of traceroute is "Matt's Traceroute".[15] Works as a GUI tool and a curses interface. It also will calculate better statistics to show you where links in the routing chain are most likely slow or dropping packets.

    [15] Not Matt Pelletier, a different Matt (there are a few of us around).

  • tcpdump If you're not sure about the traffic going to your various servers then you can have tcpdump print out information about packets being transmitted on the network. This is a quick way to spot problems with routing, connections, etc. since tcpdump only logs what it sees.

  • Wireshark (Ethereal) The absolute best network analysis tool out there. It's changed names, but Wireshark is an incredibly good graphical tool for analyzing network traffic. Think of it as a super fantastic tcpdump. You can capture packet traces, save them to files, use filters to reduce the information, follow TCP streams to see full conversations between peers, and decode entire protocols.


Section 7.4. Reporting Mongrel Bugs

7.4. Reporting Mongrel Bugs

Sometimes Mongrel has defects that you would like to report. This is difficult, though, because many problems aren't only Mongrel's, but usually with Mongrel and some Web framework being hosted. Add to this the potential that your code may be broken and debugging gets tougher. What you need to do when submitting a potential defect to Mongrel is to follow this process as best you can:

  1. Make sure the defect isn't already reported on the mailing list or the bug tracker.

  2. Reduce the bug to the simplest code and procedure that demonstrates it. Many times just doing this will help you find any problems and fix them yourself.

  3. If after reducing the bug you find it's actually a Mongrel defect, post a report to the http://www.rubyforge.org/projects/mongrel/ bug Tracker.

  4. Be sure to include the versions of all the software you use, your reduced code, and the test procedure.

  5. Wait for it to be fixed. Usually defects are fixed in one version and quite quickly.

  6. If you can submit a patch that fixes it, then that's great. Make sure the patch is against the release you're using so we can validate it.

Zed Sez

I'm not a porn star. I know, I look like one, but I'm not. Yet for some reason people feel they can shove giant patches at me without warning me and expect me to take it. I call this "code fisting." It's where you make a giant patch but don't write any tests, don't ask me if it's worth doing, and expect it to be applied now! Don't be a code-fister. I work on this stuff for fun, and really like helping people, but if you're going to give me a patch, be gentle. Tell me about it, communicate, send me the changes in small chunks, and make sure it's tested. I call this method of communicating change requests "code lube."

The worst thing you can do, though, is quote sections of the HTTP 1.1 RFC at me like you're a fire-and-brimstone minister yelling scripture out of the Bible at your flock. I don't think any Web server actually conforms to the HTTP 1.1 RFC because it's vague and complex. Many requirements in the RFC only apply in certain contexts and sometimes they contradict other sections. Rather than being a rules lawyer and throwing a paragraph of the RFC out of context at the bug tracker, go off and implement the requested feature yourself. Code is the currency of respect in the open-source world; write some if you want respect.


Section 8. Performance

Performance measurement and tuning could eat up an entire book by itself. Modern performance measurement involves statistical analysis, experimental design, and some simulation. It's heavy stuff but usually you only need a few quick techniques to get started.

Zed Sez

The recommendations in this section are not to be taken as rules you apply blindly. Computer geeks are notorious for hearing a rule and then processing it as "the truth" forever until the heat-death of the universe. I remember working with many Java programmers who would still convert all Strings to StringBuffers even years after Sun fixed the JVM so strings weren't a performance problem. Convincing them this wasn't necessary was impossible. They read it once and it was truth. Nobody could ever change that.

Always question and measure every new deployment. Don't assume that the tricks you used yesterday or even for the last version give you the same performance. Without measurements backing up your assumptions, you might as well divine your server configuration from a coil of sheep entrails (http://en.wikipedia.org/wiki/Extispicy).


Section 8.1. A Simple Process for Tuning Your First Deployment

8.1. A Simple Process for Tuning Your First Deployment

You've built your application, created all your unit tests, and set up a staging server. After months of development, you're now ready to put the thing out in the wild and watch it sink or swim.

To give your application a fighting chance, you'll need to get it properly installed and working really well, then you'll want to tune it. "Tuning" historically meant "apply a list of tips and it'll be fast." We're changing tuning to mean "measuring and applying changes to make statistically significant improvements in measured performance." Let's break this definition down into its pieces to make it clearer:

  1. "Measuring" means using a tool to gather numeric metrics that are the simplest you can possibly collect.

  2. "Applying changes" means make a change after you measure so that you know if the change made a difference.

  3. "Statistically significant"there's a lot more to this, but basically, if you collect your metrics correctly, then you can use simple statistics calculations to figure out if this next measurement actually is better than the one before the change.

  4. "Measured performance" means an improvement in the numbers. There is also a whole other method where you can use visual tricks and psychology to just make people think it's faster.

8.1.1. Setting Your Goals

Performance is to programmers what a room full of gold-coated slot machines is to gambling addicts. They will sit in front of a program tuning it to the complete detriment of the project. What's worse is this happens to everyone. The second you start talking about performance, every programmer becomes interested. It's a sickness.

You must have a goal to escape this trap. Without a target performance goal that is easily measurable, you can spend weeks working on a problem that is actually not possible to solve. Imagine if you spent a week working on tuning a deployment and only three people actually use it. Or worse, you tune it and only after spending a week on the problem you realize that it'll never be fast enough.

There are a few key components to your goal, though:

  1. It must be measurable.

  2. It must be exactly defined.

  3. It must be based on an initial exploration of what's possible.

"Measurable" means that you can equate the goal to some metric that you can collect. Requests per second (req/sec) is usually the best one. No matter what people tell you, it is not possible to measure "users" for performance. You can have 10 users that are massive porn hounds and download at 1,000 req/sec for weeks solid and you can have 10 million users that visit once a week each with 1 req/sec.

"Exactly defined" means that you have to use numbers. It has to be a goal that's defined in terms of req/sec numbers, bytes/sec transfer, etc. Not using numbers means you have no idea whether your measurements are getting you to the goal.

Finally, you can't just pull a goal out of nowhere. Management will typically do this, but you'll have to be ready to do your own analysis and go back to them and tell them it's not possible if their goals are unrealistic. Having solid metrics and numbers that are easily verified will help with this.

A very simple example goal is to simply state the number of requests per second the site should service in the best and worst cases. For example, "The site should consistently serve 120 req/sec for best case (fastest) actions /foo and /bar, and serve 55 req/sec for worst case actions /lamz and /flarz." With this kind of measurable goal, you can get started and then refine the goal as you go, adding more conditions or even reducing the expectations. It also means you can go back to management and tell them early on whether this is possible.

8.1.2. Gathering Your Tools

The preferred tool for many people is either httperf or Apache Bench (ab). The statistics and results of httperf are better defined and accurate, but ab uses much less resources during the test. We've found that ab is fine for quick pen tests (and it even aborts connections without you having to tell it), but that httperf is better for serious production measurement.

Tools to avoid are JMeter, Siege, or any tool that claims to test "users," produces graphs (these are generally useless), or doesn't include common statistical measurements like standard deviation or mean. In fact, if you run the tool once and don't see the words "mean" and "standard deviation" then reject the tool immediately. Without these basic statistical elements you can't even use the tool to see if one measurement is statistically better than a previous one.

8.1.3. Collect Baseline Data

Your next task is to simply collect a baseline of what you can expect for the very best possible performance. This is most easily done by taking your existing deployed applicationdeployed exactly how it will exist in productionand testing a simple "tester" action. I typically set up the following:

  1. A very small Mongrel handler that just returns the word "test."

  2. A similar Rails handler that also returns "test."

  3. A Rails handler that does one query to one small table and returns one result.

Hit each of these with httperf a few times and record that as the best possible performance you can get. The Mongrel handler is the top for possible performance; the Rails handlers are the tops for a plain action and an action with some database behind it.

It's also good to make sure this is what you expect, and then investigate any possible configuration problems that make these slow.

8.1.4. The Tuning Process

After you have your baseline your process is very simple for getting your application faster:

  1. Start with a small example action and measure it, recording the measurement.

  2. Change your configuration (number of Mongrel processes, network configuration, OS configuration, etc.).

  3. Measure this new configuration and compare with the previous measurement. If it isn't faster then remove the change and try something else.

  4. Once you've tuned the configuration to the fastest you possibly can, you can start using a tool like ruby-prof[16] to analyze your code and improve it in the same way.

    [16] Install it with gem install ruby-prof.

It's also important to re-run this process after each deployment to make sure that nothing broke during development or the last configuration change.

Zed Sez

What's all this insistence on "statistically significant differences"? Humans are really bad at figuring out whether two things actually differ, especially when presented with large amounts of information. Statisticians have figured out clever ways to summarize the distributions of large sets of numbers and then use these "summaries" to determine if there actually was a difference worth mentioning. In science this is important because you can't claim to have found a new cure for cancer unless you have some significant results showing the drug actually makes a difference.

Most performance-tuning books never mention this part of statistics. A very few even mention standard deviation let alone mean, but I haven't found any that actually use common statistics significance tests to determine if one measurement is different from another. The main reason (besides stupidity and ignorance) is that talking about significance testing alone can eat up entire books, so we can't even cover it in this one.

If you use a tool like httperf and aren't inclined to learn the math needed to do a real significance test, then just look for changes that move the mean measurement consistently in the right direction. If a change is only a few percentages different, then it is probably not worth the effort and most likely isn't really creating a difference.


Section 9. Security

Back in the evil dark days of the WWW it was believed that a Web server should bend over backwards to accommodate any client written by anyone at any time no matter what the quality. For this reason most Web servers will allow nearly any malicious payload you send at them to pass right on through to your applications and network. Typically the only way to protect yourself is to create a blacklist[17]of known attacks and block them with special security tools.

[17] In the security world it's an established practice to use a "whitelist" rather than "blacklist" when trying to create a secured access policy. Instead of configuring your security to say "I accept everything, except…" you configure it with "I reject everthing, except…".

Mongrel takes a more proactive and strict approach to Web server security. As an example, Mongrel's HTTP processing is handled by a parser that has strict grammar specifications and exact size limits on all elements. Turns out if you simply reject anything that doesn't follow the HTTP 1.1 grammar, then you reject most security attacks without effort.[18]

[18] Zed has a theory that there's a correlation between ambiguity in the grammar specification for HTTP 1.1 and potential for security defect in the Web server or client protocol handler.

In addition to using a parser, Mongrel has extensive testing (at least 90% coverage), a full security policy, and a frequent audit process that involves some advanced auditing and validation techniques such as "fuzzing." By following a consistent security policy and using the most advanced testing tools available, the Mongrel team is able to keep the number of potential security defects down, and be proactive about it rather than reactive. While it's impossible to say Mongrel is completely secure, you can at least verify what policies are in place and make your own judgments.


Section 9.1. Mongrel's Security Design

9.1. Mongrel's Security Design

9.1.1. Strict HTTP 1.1 Parsing

Today's Web servers use hand-written parsers that are incredibly lax about processing their input despite HTTP 1.1 having a fully specified allowed grammar. These Web servers are in effect using a blacklist to determine what's a valid request, "We accept everything, oh except that, oh and that, oh that too."

Mongrel uses a strict HTTP 1.1 parser generated by the fantastic Ragel generator. This parser is not so strict that slightly wrong clients are rejected, so don't worry about losing customers. It is strict, however, in the sizes of each element allowed, the grammar of the important parts, and its grammar is directly comparable with the HTTP 1.1 specification.[19]

[19] You can actually compare them side-by-side to make sure Mongrel is following the specification. That's not possible with a hand-written parser.

Mongrel's use of a parser changes the Web server input security policy from a blacklist to a whitelist. Mongrel is telling the world, "I reject everything except this exact grammar specification." Not only is Mongrel's parser able to reject large numbers of malicious requests without any special "security lists," it is also very fast thanks to Ragel.[20]

[20] Using Ragel also helped cut down development time, made it easier to validate against the RFC, made it possible for others to reuse the parser in their projects, and was just a really great idea.

Zed Sez

All right, pretty people, listen up. You went to school one day (or maybe you didn't, lucky you) and some professor told you that a parser written by hand is faster than what a parser generator (like Ragel) creates. You assumed he was right and never ever, ever questioned him again and will probably die still thinking, "Gee, parsers written by hand are super fast." Parser generators have been under twenty or thirty years of continuous development and research, and can now beat the pants off nearly anything some dip at an Apache group can write by hand. Why? Not because they are faster but because they are correct. Mongrel has shown that the speed is only marginally different, that the parser is not the bottleneck (it's IO, dumbass), and that having the ability to explicitly kill badly behaving clients is the best way to protect against malicious attacks. Most important, though, every hand-written parser (and HTTP clients or servers) starts off really simple, then blossoms into a giant morass of horribly twisted crap that houses all the major security holes. So, take your panties out of their tightly coiled bunch and move on.

9.1.2. Request Size Limitations

The HTTP 1.1 standard doesn't set any limits on anything except maybe the size of allowed cookies (and even that is very loose). You could put a full-length DVD in a header and still technically be correct (the best kind of correct). For a little server running in an interpreted language like Ruby, this is a murderous path to destruction. Without limits on requests, a malicious attacker could easily craft requests that exceeded the server's available memory and potentially cause buffer overflows.

In addition to strictly processing all input in exactly sized elements, Mongrel also has a set of hard-coded size limits. These limits are fairly large for practical purposes, but some people really like to stretch the limits of the standard. The limitations as of 0.3.x are:

  • field names 256 bytes

  • field values 80k bytes

  • request uri 2k bytes

  • request path 12k bytes

  • query string 10k bytes

  • total header 112k bytes

Since Mongrel's creation nobody has complained about these limits. It's quite possible someone may hit them in the future, but simply not using these elements to store their data is much easier than trying to get Mongrel's limits changed.

When you do exceed one of these limits, Mongrel reports an error message to the console or mongrel.log telling you what limit was triggered.

9.1.3. Limiting Concurrent Processors

Mongrel places a limit on the number of concurrently connected clients due mostly to a limitation in Ruby's ability to handle files. If a client connects and there are already too many connected, then Mongrel closes that client and starts trying to kill off any currently running threads that may be too old. It also logs messages saying this was necessary so that you can update your deployment to handle the new load factor.

The alternative to doing this is to simply let Mongrel die. By rejecting some clients and telling you that there's an overload problem, you can still service a portion of your user base until you can scale you deployment up.

9.1.4. No HTTP Pipelining or Keep-Alives

In HTTP 1.1 a feature called "pipelining" was introduced and the request/response model was changed to a "persistent connection" model using keep-alives. This was potentially useful back when making connection requests was expensive over phone lines, but in modern times the benefits are actually placing more load on the HTTP server, sucking up precious available concurrent processors so other people can't enjoy them.

Mongrel is also at the mercy of Ruby's file I/O limitations. There are plans to improve how Ruby processes files, but right now it can only keep 1,024 open files at a time on most systems (even fewer on some). Allowing clients to keep a connection constantly means that a malicious attacker can simply make a series of "trickled connects" until your Mongrel processes run out of available files. It literally takes seconds to do this.

Another questionable point about HTTP pipelining and keep-alives is that there seem to be no statistically significant performance benefits with modern networking equipment or localhost configurations (which Mongrel is almost always deployed in). The complexity increase implementing this part of the standard simply doesn't justify the almost non-existent benefit in today's deployment scenarios.

Rather than implement more and more complexity, Mongrel simply uses the special "Connection: close" response header to indicate to the client and any proxy servers that it's done. This reverts Mongrel back to the HTTP 1.0 behavior of one request per connection, and actually does improve its performance under heavy load for the above reasons. It's possible that this might change in the future, but for now it works out great and only HTTP RFC Nazis seem to care.[21]

[21] More than a few giant companies (including one that does-no-evil) and several proxy server vendors all were caught not honoring the "Connection: close" response in their proxy server software. Ironically they came to me claiming Mongrel didn't follow the RFC, when it was actually their proxy servers that didn't.

Zed Sez

Your application is slow as dirt and you are convinced that you need keep-alives and pipelining to make it go faster. You've got to have it or nothing will work, the world will crumble to dust, and we'll all be replaced by intelligent squids after the human race is long gone. It's that serious.

Look, you don't need any of that simply because this isn't 1996. Back in the day keep-alives and pipelining were added so that people on phone lines didn't have to wait, but the assumption that this is necessary in all networking situations has never been tested heavily. These parts of the RFC only add complexity, are ambiguous, and don't really make things fast enough to justify the development and maintenance overhead for a server that's run on localhost or controlled networks.

Most important, though, is that allowing them makes it so that a malicious attacker just has to start up more than 500 keep-alive or pipelined requests and your whole Ruby application eats it. Why? Because Ruby 1.8.x uses the select() function to do its IO and that only supports 1,024 open files on most systems.

Another problem, though, is that people use keep-alives and pipelining as a Band-Aid for horribly designed systems. I ran into one fellow who desperately needed keep-alives so he could send three characters per request to a client at 2,000 requests/second. Yes, just three characters. Here's a clue: batched processing. In almost every instance that people have claimed to need these features, a simple design change to batched processing or an improved network design removed the problem and simplified everything.

9.1.5. No SSL

Mongrel does not support SSL. Typically you will be running Mongrel behind a static Web server (see Section 4), in which case you'll have SSL support from there. Make sure you are sending headers to Mongrel properly so that it will respond with HTTPS instead of HTTP.

Zed Sez

You don't need SSL either. Well, you need it at some point in your deployment configuration, but why would you want a slow language like Ruby to do the SSL when there are already really great, fast Web servers? Ruby is a precious, slow commodity that you should use only when you absolutely must.

9.1.6. No [Name Your Must-Have Feature]

As you probably see, Mongrel say, "No" in many places where most Web servers say, "Yes, OK." Sometimes this is because no one using Mongrel has needed it yet, sometimes it's because there's a better, simpler way to accomplish the same goal. Mongrel is a different kind of Web server, and frequently you can solve your problem with a different solution.

Ready for another catchphrase? Constraints are liberating. When you are given a billion different options, you can become paralyzed by choice. When you are forced to work within reasonable constraints, you can focus on the problem of getting your task done rather than deciding how to configure your Web server. Mongrel's feature set and limitations work perfectly for about 95% of the current users. Those people who have additional requirements can easily extend Mongrel using the plugins, commands, and handlers. The rest should find a tool better-suited to their job or change their requirements.


Resources

Appendix. Resources

Mongrel Web Site http://mongrel.rubyforge.org

Mongrel Mailing List http://rubyforge.org/mailman/listinfo/mongrel-users

Mongrel FAQ http://mongrel.rubyforge.org/faq.html

Mongrel on Windows http://mongrel.rubyforge.org/docs/win32.html

Mongrel Cluster http://mongrel.rubyforge.org/docs/mongrel_cluster.html

Mongrel Configuration http://mongrel.rubyforge.org/docs/index.html

Mongrel RDoc Documentation http://mongrel.rubyforge.org/rdoc/

Ruby on Rails Mailing List http://lists.rubyonrails.org/mailman/listinfo

Ruby Mailing List http://www.ruby-lang.org/en/20020104.html

GemPlugin RDoc Documentation http://mongrel.rubyforge.org/gem_plugin_rdoc/index.html

RCov Web Site http://eigenclass.org/hiki.rb?

httperf Web site http://www.hpl.hp.com/research/linux/httperf/

Installing Ruby http://www.ruby-lang.org/en/downloads/

Rails Concurrent Requests http://permalink.gmane.org/gmane.comp.lang.ruby.mongrel.general/245

Ruby License http://mongrel.rubyforge.org/license.html

Ragel http://www.cs.queensu.ca/~thurston/ragel/

RFuzz http://rfuzz.rubyforge.org/

Camping Framework http://camping.rubyforge.org

Iowa Framework http://enigo.com/projects/iowa/index.html

Ruby on Rails Framework http://www.rubyonrails.org

Og+Nitro Framework http://www.nitroproject.org/

OSX Leopard Features Rails and Mongrel http://www.apple.com/server/macosx/leopard/more.html

OSX Leopard to Ship with Rails and Mongrel http://weblog.rubyonrails.com/2006/8/7/ruby-on-rails-will-ship-with-os-x-10-5-leopard

Memcached http://www.danga.com/memcached/

DRb RDoc http://www.ruby-doc.org/stdlib/libdoc/drb/rdoc/index.html

DRb Intro by drbrain http://segment7.net/projects/ruby/drb/

Mongrel + Pen by Ezra http://brainspl.at/articles/2006/04/26/dead-simple-deployment

BackgroundDRb http://backgroundrb.rubyforge.org/


Acknowledgments

Appendix. Acknowledgments

From Zed: I'd like to thank all the people who worked on the Mongrel project either by contributing code, ideas, help on the mailing lists, writing documentation, or just laughing at me when I was trying to be funny. Rather than forget someone, I'll just use the power of the Internet and tell people reading this short cut to look on the Mongrel Web site (http://mongrel.rubyforge.org) for a list of people who've helped.

From Matt: For help with this short cut, I'd like to thank Wilson Bilkovich, Trotter Cashion, Jeff Ng, Lee Nussbaum, Josh Viney, Ezra Zygmuntowicz, and all the good folks who helped review and improve it. Zed and I would also like to thank our editor Debra Williams Cauley, who recognized the potential for Mongrel (and Ruby) early on.


About the Authors

Appendix. About the Authors

Matt Pelletier is a partner at EastMedia, a software, mobile, and business development firm based in New York City. In conjunction with technology partner VeriSign, EastMedia provided commercial sponsorship of Mongrel for the Apache Heraldry project, where it serves as a key component of the Identity 2.0 Software stack. Matt is cofounder of NYC.rb, the New York City Ruby group.

Zed Shaw is the original and primary author of Mongrel (as well as a number of other Ruby projects) and plays an active role in the growing Mongrel community, answering questions, responding to feature requests and bug reports, and helping troubleshoot and work with the growing needs and uses.