Weblog

05/15/06: Rails / lighttpd gotcha: AJAX stalling? Check your Content-Length header

So neither Ruby on Rails nor lighttpd set the Content-Length header in HTTP responses on their own. That means that the browser doesn't know when it's gotten the whole response, so it has to keep the connection open in case more bytes come down. In normal web usage this doesn't cause a problem (*) since browsers are smart enough to display partial results. But for AJAX responses it means that, sometimes (**) the onComplete handler (***) doesn't get called for over a minute. The onInteractive handler gets called quickly, but that doesn't help, since usually you do stuff during onComplete when you're guaranteed to have the full response.

Here's the fix in Rails. Put the following in your application.rb:

 after_filter :set_content_length

 def set_content_length
   @response.headers['Content-Length'] = @response.body.length
 end
Anyone who wants to fix the above to check for an existing content-length, be my guest :-)

(**) The stalling happens sporadically, not sure with what pattern. In one app we saw it maybe one in 5 requests; some of those were actually HTTP requests so the

(*) Occasionally the page will be fully visible but the cursor will still be a pointer-plus-spinning-hourglass. And this may cause performance problems in high-volume sites.

(***) onInteractive and onComplete are prototype.js event callbacks; the corresponding standard XMLHttpRequest "ready states" are INTERACTIVE and COMPLETE; "ready state" is a parameter to the onReadyStateChange callback.

(****) Apache is nice enough to tack on a proper Content-Length for you, so if you're running behind Apache you're safe from this problem.

05/15/06: Trapped in the Closet

You've probably done this a thousand times:
private Map foos = new HashMap();

public void getFoo(String key) {
  synchronized (foos) {
    Foo foo = foos.get(key);
    if (foo == null) {
      foo = new Foo(key);
      foos.put(key, foo);
    }
    return foo;
}
If two threads are both trying to get the same item before it exists, the first one will create it, and the second one will block until it's ready. (I've heard of the dreaded double-checked locking conundrum, but I believe that if I close my eyes, plug my ears and say "nah nah nah" then it will never happen.)

This is a common enough creation pattern. So common, in fact, that it feels like there should be a type of Map that does it for you. Well, now there is :-) Christian and I call ours a Closet -- I suppose it's sort of a magic closet that creates things for you if they're not hanging in there already.

private Closet foos = new Closet() {
  protected Foo create(String key) {
    return new Foo(key);
  }
};

public Foo getFoo(String key) {
  return foos.get(key);
}

Much nicer, But wait, we're not done! There's a problem with this design in a world where create() takes a long time. (We came across it while trying to cache Lucene IndexReaders.) If two concurrent threads try to get *different* items, and the first one hasn't been created yet, then the second thread will block even though its item is ready and waiting.

We arrived at a fairly elegant solution to this problem, but the more interesting part was writing the tests first. Writing a unit test for a concurrency problem is always annoying. It's tempting to spawn a bunch of threads and then use sleep() either from your tests or from overridden methods, but that will make your tests slow, and even then they don't always work consistently.

It's better to model your tests either to *simulate* multiple threads, or to actually spawn threads but keep a tight reign on them. The subordinate test threads do a few things, then wait for the main test thread to tell them to proceed. That keeps the logic in your main test clean and simple, like a script for a play with several characters -- Joe does this, Bob says that, Joe walks there, Jane enters stage left.

Now all we need to do is add expiration logic and we've turned our Closet into a cache.

(Full code is unavailable for the moment, sadly... stay tuned.)

04/28/06: JSLint

I've been doing a lot of JavaScript (and a lot of cursing) lately. Edward pointed me to JSLint, which checks the syntax and style of your JS code. I think it's too strict (but it's supposed to be) but I do recommend reading the documentation -- the explanations of /why/ JSLint complains are clear and insightful.

Read More!

alex  Code 

04/09/06: Language Elegance

Keith Braithwaite waxes objective in peripatetic axiom: Keywords, Magic and (E)DSLs -- he gives a pretty good high-level introduction to Smalltalk and what language simplicity really means. He makes the point that Java isn't really object-oriented -- or rather, as I like to put it,
Java is object-oriented, in that it's kind of oriented towards objects, from afar.

There's something beautiful and elegant about a system with as few rules as necessary to get the job done. Java and, sadly, Ruby, overcomplicate life with bifurcations and exceptional cases and defaults and other magic. Only Smalltalk and Scheme reach the level of elegance I'm talking about -- a very small number of rules from which you can derive the universe of software.

(Keith's example is that if, a part of the grammar of any C-style language, is actually implemented as a method of the class Boolean. It takes two blocks as parameters; the concrete Boolean subclass True executes the first and False executes the second.)

And no, I'm not advocating a return to assembly language, although the simplicity of stack- and register-based programming approaches what I'm talking about. In Java there's at least two ways to do so many things. There are primitives *and* objects. There are statics *and* instance members. There are Arrays and ArrayList. There are final classes (like String) and regular ones. And each of these pairs has different semantics, and you keep getting tripped up over things you can and can't do.

Ruby's a little better, in that everything's an object -- oh wait, unless it's a mixin (aka module). Which means that if someone decided to implement some stuff as a mixin, and you want to override it, you can't use "super", you have to declare an alias and call it. And everything following a method name is a parameter, unless it kind of looks like a hash, in which case it's all one parameter, unless it looks like a block, in which case it's the "default block".

My point is, if you remove rules rather than adding them, there's a chance you'll turn Chess into Checkers. But there's also a change you'll reach Go.

04/01/06: Save Considered Harmful

Humane Gmail Autosave solved one of the many problems with the so-called text editor in which I spend most of my time these days. Or rather, in which I often type some stuff, then get distracted into a different task or tab and let it sit for hours. Thankfully gmail does autosave by default now, making this extension unnecessary, though it'd be nice to crank the frequency up to once per second or once per keystroke or something.

Why, in this day and age, is there even a Save command in any application? Its very presence implies -- indeed, guarantees -- that the default state of the world is unsafe. This breaks the rule our ancestors learned over billions of years of interaction with the objective world: when you do something, it stays done, until undone.

I have agreed for decades with the following sentiment:

"...[The] user should never have to explicitly save or store work. The system should treat all produced or acquired data as sacred and make sure that it does not get lost, without user intervention. The user may, of course, deliberately delete anything. The ability to delete (or make any changes) means that universal, unlimited-level undo and redo should be inherent to all systems." - Jef Raskin, The Humane Interface
To Raskin's paragraph I would add that for documents, instead of "Save" should be "Mark" and "Revert To Mark". We could also use a "Publish" to allow you to strip out the undo history so your readers don't get to see all your second-guessing.

A dialog box in which you're editing existing data should update immediately, and have a "Revert" 'button which puts things back to how they were when you opened the dialog. A dialog box in which you're constructing brand new stuff is temporary, so in that case there could be a "Save" button (or, better, "Create"). But it'd be great if instead, it made a new, persistent "proposal" or "draft" for the item, so it would still be there if the power went out.

04/01/06: JavaScript and Ruby support in IDEA

In addition to Ruby, I've been doing a lot of JavaScript programming lately, and I've just moved from Eclipse to IntelliJ IDEA 5.1. Their new JavaScript plugin works like a dream. It makes me hope for an IDEA Ruby plugin.

Read More!

alex  Code 

02/21/06: JavaScript and CSS Versioning in Rails

To defeat browser caching for static files that actually do change once in a while, a common trick is to append a random number to the URL:
<link href="/css/mysite.css?9238238" rel="stylesheet" type="text/css">
If you use a truly random number (or the current time), it will reload every time. However, this is not exactly what you want; instead, you only want it to reload when the file has changed, and use the cached version when it hasn't. A good way to do this is to use the timestamp of the time the file was last modified.

Here's an easy way to do this in Ruby on Rails.

Read More!

alex  Code 

02/18/06: Real Lessons for Rails Deployment

"Bottom Line: If you can, use Lighty. The fact that it's now used by script/server when installed means that lots of other people are looking at it all of the time and that any problems with it will be addressed rapidly. If you need Apache's mod_rewrite or any other of its fabulous arsenal of modules, or simply have to use Apache because that's the corporate line, use Apache httpd 2.0.x if at all possible. And, if you're going to use Lighty, be sure not to use config/lighttpd.conf to stash your production configuration. I used to stash my production lighttpd configuration there, but then the built in script/server commmand introduced in Rails 0.14.something-or-other didn't work so nicely. So now I let Rails create the config/lighttpd.conf file and I stash my production configuration somewhere else."
Real Lessons for Rails Deployment

alex  Links