Load Control in the PLT Scheme Web Server

I've recently added a new dispatcher to the PLT Web Server: dispatchers/limit.

It allows you to limit the number of concurrent requests handled by a particular sub-dispatcher. For example, you could limit the number of active servlet requests, but have limitless file-system requests. The documentation gives a good example of using it.

One of the interesting features is the ability to control what happens to requests that occur after the limit has been exceeded. You can choose to (a) make them wait until there is space, (b) cause them to instantly fail, or (c) kill old request handlers and prioritize the new ones.

---

The code that implements this dispatcher is a good example of using the synchronization primitives of PLT Scheme (inherited from Concurrent ML.) Take a look at the code. You'll notice a few things:

• The manager of the resources is abstracted into a separate thread, created at the time the dispatcher is constructed.
• The users of limited dispatcher communicate with the manager through private synchronized channels that are invisible to the rest of the Web Server.
• When a limit request is made, it is sent to the manager with a new first-class reply channel, so that the thread can receive a go-ahead to continue. Notice that this is not the case when the limited request completes: it doesn't need to wait for confirmation.
• Because the channels of communication to the limit manager are completely controlled in this short function, the users can be relied upon to implement the load-shedding behavior. (See how if the reply channel communicates false, then the thread errors, freeing up resources.)
• The manager is implemented as tail-recursive loop, storing the changing state (the current limit and the active requests) in lexical variables.
• The sync function is repeatedly called to decide what activities the manager does at each step. sync will choose one of the available events each time it is called.
• Based on whether the limit has reached it maximums, the manager chooses between allowing new requests in, or implementing the over-limit behavior. It can cause the new threads to block simply by ignoring the incoming request channel (implemented by returning the never-evt, rather than a handler based on it.
• In either case, the manager will wait for requests to finalize and free-up their resources.
• Because request handlers might die unexpectedly if their connection to the Web client is broken, the manager needs to monitor active threads and free their resources if they die. Therefore, it constructs an event for each partner that handles that partners death. This is why we use apply to call sync, rather than calling it directly.

This dispatcher would be much more painful to implement if we did not have the ability to use first-class channels, closures, variable-arity functions, or CML-style synchronization operators.