httpx

This software has been built using and depends on ServerKit.

Overview

Httpx is a new aproach to name-based virtual hosting on UNIX I invented to solve major shortcomings in existing virtual hosting solutions.

Httpx takes over the front-line position of binding and listening on the public address & port of your virtual webhosting server. In this role, httpx snoops incoming requests looking for the now ubiquitous HTTP/1.1 Host request header line. Once the Host line is found by httpx for a given request, httpx performs a simple SELECT on a MySQL database with the found Host value. For valid hosts (those matching rows in the database), a Path column is returned and used as a path into the local filesystem. The purpose of this path is to provide an address for reaching the appropriate web server processes responsible for the domain specified in the Host request header. The file residing at this path is a named UNIX domain socket, with a real httpd process instance waiting at the other side on behalf of the respective domain ready to serve.

Note that httpx is not a proxy, httpx employs the inter-process descriptor passing capabilities of UNIX to hand off the socket to the appropriate process by using a combination of somewhat obscure and scarcely used features. Namely, the MSG_PEEK flag for recv() is utilized for accessing the requests socket buffer contents without actually removing anything from the socket buffer. The SO_RCVLOWAT socket option is utilized to inform the socket layer to block as we accumulate more data in the sockets receive buffer when we have yet to find the Host request header line in the previous recv() calls with the MSG_PEEK flag. Together, when implemented properly, these features allow us to efficiently snoop on the request without leaving any evidence for the destination httpd process to trip on.

By using these features httpx introduces minimal overhead, and allows us to continue using all the popular web server software - even more freely than before, since you don't need to be concerned about things like mod_perl or mod_php security issues or instabilities effecting the shared process space of your entire vhosts pool. Every domain can have it's own minimal specific and simply configured instance of Apache, while sharing a single IP:port. Every domain can even run a completely different web server software if desired.

All that is required is a minor patch to the web server software of interest to enable accepting sockets over a UNIX domain socket stream, which is a pretty trivial change that deviates only slightly from the already present AF_INET bind, listen, and accept arrangement. Once the software has received the passed descriptor, it's the same old tcp socket as it worked with before.

The Shttp ServerKit module as of version 0.0.10 has been modified to support the required UNIX domain socket descriptor passing if you would like to experiment with httpx immediately. Or you can try your hand at patching one of the web servers I've created a patch for.

Compatibility issues with present Linux kernels

Current behavior of MSG_PEEK and SO_RCVLOWAT features in Linux do not cooperate with httpx, and whether they should be changed or not is debatable and has been discussed on LKML, as seen here.

Update:
Looks like these incompatibilities will be fixed in a future Linux version:
httpx may be insane but it presents a practical and compelling argument


Implementation diagram

PDF version

Usage

It is advised that you read the ServerKit documentation before trying to use this module, as most of the basic usage is simply using ServerKit, nothing specific to this module.

You will probably want to use the included ServerKit personality as a starting point, found in the source archive as a directory named personality. Within the personality you will find a c11n file, a modules subdirectory, and a svsdir subdirectory.

The c11n file is a ServerKit configuration file, you will need to at least modify the database settings so ServerKit can successfully communicate with your database system.

You are responsible for creating a database and table compatible with httpx, please see the included INSTALL file for information on the expected schema.


Configuration

The following configuration options are supported:
NameTypeDescription
min_sessions integer Minimum number of sessions to allocate.
max_sessions integer Maximum number of sessions to allocate.
min_threads integer Minimum number of worker threads to create.
max_threads integer Maximum number of worker threads to create.
database string Identifier of database connection pool instance to use, see ServerKit configuration for assistance with database connection pool configuration.
backlog integer Size of kernel-level connection backlog, passed to listen() as-is
listen_address string Address to listen on
listen_port integer Port to bind to
default_path string UNIX domain socket to pass sockets having requests with no Host: header

You can query the module for its supported configuration options by simply running it like a normal executable program. This is the preferred method of keeping informed on what configuration values are supported and what the defaults are.

Here is some sample output of running the module:
swivel@volatile:~/src/httpx-0.0.1$ ./httpx.so 
ServerKit bundled module inspector

- Summary -
Name: httpx
Description: HTTP/1.1 host-specific httpd switch
Module version: 0.0.1
ServerKit build environment version: 2.1.0
Authors: Vito Caputo <vcap***NOSPAM***ru.com>

- Supported configuration options & defaults -
listen_port = 80
listen_address = "0.0.0.0"
backlog = 16
max_sessions = 65536
min_sessions = 64
min_threads = 4
max_threads = 100
# database = ""


Source archives

Release date Tar.gz MD5 checksum
11-26-2008 httpx-0.0.6.tar.gz 7075e6e626fc707e8c121a57774ea044
10-27-2008 httpx-0.0.5.tar.gz 986ff7726d22651d5546aedfe9506147
10-13-2008 httpx-0.0.4.tar.gz d0aaac0be5e93c16fceda654863fef23
09-30-2008 httpx-0.0.3.tar.gz 5f3e1394fb5cd32fb4a6bc6ec65191a1
09-23-2008 httpx-0.0.2.tar.gz a93162c3216aa5114cca5acd1dc608cc
09-21-2008 httpx-0.0.1.tar.gz b06ca8d48d9dcd1e8df41d9a75813f74

Patches for integration with popular web servers

Apache

Apache 1.3.34
A patch which adds the ability to specify an absolute path as a Listen directive argument instead of an IP address and/or port. i.e. Listen "/home/foo.com/.httpx", the specified path will be used for the UNIX domain socket through which socket descriptors will be passed.
* This patch was created against the Debian Apache 1.3.34-4.1+etch1 source distribution.
* This is a 2nd version of the patch which adds setting the umask before calling bind() to guarantee the created unix domain socket is world writable.
* It can be annoying to try reuse the same apache install for suexec and non-suexec instances. It's easier to create a copy of your Apache binary for non-suexec instances and replace the path to the suexec executable with something nonexistant. One can even do this with a hexeditor. E.g. replace "/usr/lib/apache/suexec" with "/usr/lib/apache/noexec" or something else not found.
* Another likely appreciated modification is the last line of ap_can_exec() in main/util.c of Apache, lose the test of other execute. There's no good reason for Apache to look for the other execute bit on scripts, not when Apache runs as the same user as the script owner behind httpx from the get go. If you don't modify this you have to make your scripts executable by other, Apache actually actively enforces this broken policy.

Lighttpd

Lighttpd 1.4.20
A patch which adds the ability to receive file descriptors over the already present UNIX domain socket listener support. To enable this behavior when using the UNIX domain socket listener (when specifying a filesystem path for server.bind) you must specify server.use-httpx = "enabled" as well. Then simply start lighttpd as a regular user with your specialized configuration file, inform httpx of where to route the requests and it's done, simpler than integrating Apache.

License

Httpx is licensed under the terms of the GNU General Public License version 2
 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License version 2 as
 published by the Free Software Foundation.

 This program is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 GNU General Public License for more details.

 You should have received a copy of the GNU General Public License along
 with this program; if not, write to the Free Software Foundation, Inc.,
 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

Contribute

I accept patches or feature requests, simply email me at the email address contained within the source tree. I am also always looking for testers so if you're willing to test new features, let me know.

If you would like to contribute to this project monetarily, feel free to donate via the paypal button below.

© 2008 Vito Caputo