HTTP FAIL WHALE FAIL AT HTTP

[crucially]

Spectacular failure in using HTTP from twitter -- apparently I last-modified my page now() (and not two months ago), and the ETAG changes on every request.

It changes because the etag is the md5sum of the rendered content, which contains an form_auth_token. So neither If-Match or If-Modified-Since works.

Way to waste bandwidth.

hope:~ sky$ curl -v "http://twitter.com/crucially" > /dev/null
* About to connect() to twitter.com port 80 (#0)
* Trying 128.121.146.100... connected
* Connected to twitter.com (128.121.146.100) port 80 (#0)
> GET /crucially HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin8.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: twitter.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 28 Aug 2008 23:07:50 GMT
< Server: hi
< Last-Modified: Thu, 28 Aug 2008 23:07:50 GMT
< Status: 200 OK
< X-Runtime: 0.12711
< ETag: "a5ca7514effd7287c0aa9a60d443a21e"

hope:~ sky$ curl -v "http://twitter.com/crucially" > /dev/null
* About to connect() to twitter.com port 80 (#0)
* Trying 128.121.146.100... connected
* Connected to twitter.com (128.121.146.100) port 80 (#0)
> GET /crucially HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin8.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: twitter.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 28 Aug 2008 23:09:17 GMT
< Server: hi
< Last-Modified: Thu, 28 Aug 2008 23:09:17 GMT
< Status: 200 OK
< X-Runtime: 0.19511
< ETag: "33afb185e11a9689ea9974140e67b309"

For the feeds, at least the ETAG stays the same

hope:~ sky$ curl -H "If-Match: c64cd62cd2c8f859feb60384c5c43784" -v http://twitter.com/statuses/user_timeline/2334251.atom > /dev/null
* About to connect() to twitter.com port 80 (#0)
* Trying 128.121.146.100... connected
* Connected to twitter.com (128.121.146.100) port 80 (#0)
> GET /statuses/user_timeline/2334251.atom HTTP/1.1
> User-Agent: curl/7.16.3 (powerpc-apple-darwin8.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
> Host: twitter.com
> Accept: */*
> If-Match: c64cd62cd2c8f859feb60384c5c43784
>
< HTTP/1.1 200 OK
< Date: Thu, 28 Aug 2008 23:21:49 GMT
< Server: hi
< Last-Modified: Thu, 28 Aug 2008 23:21:49 GMT
< Status: 200 OK
< X-Runtime: 0.00568
< ETag: "c64cd62cd2c8f859feb60384c5c43784"

Yet, doing a if(if-match == etag) { return not_modified } is apparently too much work.

And here I worry about getting Trailers right!

ps: i might have gotten the If-Match incorrect, if so, apologies

pss: I tried 'If-Match: "c64cd62cd2c8f859feb60384c5c43784"'

*UPDATE*

Don points out i probably want If-None-Match -- this doesn't work either though.