Re: if-modified-since question (protocol problem?)

On 10/05/2006 11:30, VK wrote:

hug wrote:

As far as I can tell, the fact that a 204 response header appears to
work (haven't played with your sample) is coincidence and using it
would be a "hack".

204 No Content header is an official documented HTTP/1 header, so I'm
not sure how could it be a "hack".

The context in which you suggested it

Simply check the data source server-side and take a server-side
decision if it's the same or not. ... If the data is the same,
simply throw back
Status: 204 No Content\n\n
-- 1147172754.774022.49240@xxxxxxxxxxxxxxxxxxxxxxxxxxxx

is incorrect, as evidenced by what follows.

Its application domain are the situations where you want to send a
request, but in response you are interested only if your request
processed properly or not (thus you are not interested in the content
unless it contains something new to show).
^^^^^^^^^^^
To base the decision on whether there is something 'new to show' would require knowledge about something 'old' - a cached resource. If this information is communicated using If-Modified-Since, then there should only be two types of response: 304, to indicate that the existing variant can be served from the cache; or, whatever would result from a normal GET request. As the OP has made no indication of responding with anything other than 200 upon success, 204 is not reasonable. To use it as a substitute for 304 would be, at best, a hack (if it worked as intended).

[snip]

The script behind is forcely a bit more complicated. A good part of it
is taken by the sub generating proper date in RFC1123 format. A great
mistery to me of all known server-side scripting languages (Perl, PHP,
JSP, ASP etc):- this absolutely necessary format is not available by
default language tools.

PHP has already been covered. In JSP, one should (JSP is on my list of 'Things to Learn') be able to use the java.text.SimpleDateFormat class with a similar pattern and an English locale:

"EEE, dd MM yyyy HH:mm:ss 'GMT'"

Presumably, ASP will have a similar feature somewhere.

[snip]

print "Date: $date\n";

I don't know Perl, but shouldn't all instances of \n in your header lines be replaced with \r\n?

print "Expires: $date\n";
print "Content-Type: text/html; iso-8859-1\n\n";

If you don't send a Last-Modified header,

[snip]

elsif ( defined($ENV{HTTP_IF_MODIFIED_SINCE}) ){

....why would you expect If-Modified-Since in a subsequent request?

[snip]

1) In theory any cached page may be in two states: fresh (not expired
upon headers) and stale (expired upon headers).

Not 'in theory', in practice. Those are the only two states for an existing cached resource.

If you call a page which is cached but stale, UA supposes to validate
this page before displaying:

Supposed to, yes. However, a client may be configured to return a stale response (unless forced not to by a Cache-Control directive).

thus contact the origin server and check if the page was updated.

The origin server need not be involved in revalidation if an intermediate cache can satisfy the request.

[snip]

3) By refreshing the page several times and trying to navigate back
later, you see right away big differences in caching mrchanics in
different UA's

I see no differences in how the browsers behave in this instance, so it would be helpful for you to describe what you think you're seeing.

- No Last-Modified header is sent in the response, so no
If-Modified-Since is used in a subsequent request. That
particular branch of your code should never be executed.
- The Expires header that is sent is equal to the value of the
Date header, therefore the resource is immediately considered
stale. The resource will be cached, but because it is stale,
subsequent requests will be directed towards the origin
server unless expressly overridden by the user.

[snip]

[Caching] defines a great part of usability thus the caching
neuristics is one of the most complicated and often copyrighted part
of the UA's.

If cache-related headers are sent properly, no heuristics will need to be employed.

Part of the reason why people get confused about caching behaviour is they do something like sending a validator, but no freshness information. Consequently, the browser must guess what do to. Some browsers, like Opera, give the user the opportunity to set an explicit waiting period. Others, like Internet Explorer, will guess a time. If the result is inappropriate, things will go wrong; set the time explicitly, and it doesn't.

[snip]

On a real run it will be MySGL, PostgreSQL or another database
request. But the nature of the situation doesn't change: the HTTP
mediator (this script) has no means to know if data has changed or not. Respectively it cannot set any reasonable Last-Modified header. You (your database driver) has to check the data against the request and inform the mediator.

That's not necessarily difficult though, is it? A time stamp can be updated whenever a change is made to a particular document. For instance, a CMS or BB might keep track of - perhaps for the purpose of display - when a user edited or added content. That same date, combined with the last modified time of the template used to generate the document (whether it's an XSLT style ***, a script, or whatever), can be used to generate a Last-Modified header.

5) Overall for dynamic sources simply serve all responses with
cache-preventing headers [...].

If a resource is truly dynamic (that is, it will change on every request), then sure. There'll be no point in caching it. If not, then at least some effort should be made to cache it, even if it means that revalidation should always occur. Why send data if there's no need to bother?

Mostly (but no guarantees of any kind) UA will re-query for the same
page from the server, where you can take some decisions.

What sort of decisions do you think one /could/ make? If the data hasn't been cached at all, there's no option but to send an entity.

6) An icing on the cake: Opera doesn't check the cache expiration at
all when going back and forwards in the history - it does it only when
you click a link.

Good. It's not supposed to.

History mechanisms and caches are different. In particular
history mechanisms SHOULD NOT try to show a semantically
transparent view of the current state of a resource. Rather, a
history mechanism is meant to show exactly what the user saw at
the time when the resource was retrieved.
-- 13.13 History Lists, RFC 2616

7) The huge popularity of IXMLHTTPRequest/XMLHttpRequest (aka AJAX)
may get a bit clearer now.

'AJAX' is popular because it's seen to be a new technology. That's the same reason why some people think it's a good idea to churn out XHTML; they believe it to be the latest and greatest (despite being six years old as of January, earlier this year).

Mike

--
Michael Winter
Prefix subject with [News] before replying by e-mail.
.