Re: Issue with TIdHttp.Head() routine

Giganews Newsgroups
Subject: Re: Issue with TIdHttp.Head() routine
Posted by:  Remy Lebeau (TeamB) (
Date: Fri, 30 Jun 2006

"Scott Bussinger" <scot…> wrote in message

> A change was made at some point to the TIdHTTPProtocol.ProcessResponse()
> function. The code in question looks like:

The changes were made by me, and I put comments in the code to explain why
the changes where made.  As specified in the HTTP RFCs, the HEAD command is
strictly forbidden from returning any kind of body content at all.  But some
non-compliant servers do it anyway, so the changes were made to discard that
content so that the socket does not have unread data remaining in it.  This
is especially important when keep-alives are used, since the unread data
would carry over to the next command and corrupt its response (which is
exactly the situation that prompted the changes in the first place).

> In case of TIdHttp.Head(), the DiscardContent(5000) call is always made

Correct.  Because there is no way to know ahead of time whether the server
is going to send forbidden data or not, so TIdHTTP waits a few seconds to
make sure.

> This routine will under normal circumstances now generate a timeout
> exception and this exception is allowed to bubble up to the original
> calling routine.

No, it isn't.  ReadResult() uses the timeout value with
CheckForDataOnSource(), which does not throw a timeout error.  And all of
this is only under the situation where the server does not specify whether
there is any body data included to begin with, so TIdHTTP has to figure it
out manually.

> So if you have any code that calls Head() you will now get an
> EIdReadTimeout exception that you never got before.

There should be no such exception occuring.  If there is, then the code need
to be digged into again.  But offhand, glancing at the code, I see no way
for that to happen.    Unless the server is actually sending forbidden data,
which would cause ReadResult() to call ReadStream().  That would be the
only place that the exception could be occuring, such as if ReadStream() is
waiting for a server-side disconnect that never occurs because of
keep-alives.  But if the server is sending body data with a keep-alive
enabled, then the 'Content-Length' header must be included, but that is also
forbidden in a HEAD response anyway so a disconnect is required.

> Just about as bad, if I put a try...except...end around the call to
> Head() to mask the issue, then every call to Head() takes a
> minimum of 5 seconds while it waits for the timeout to occur

Sorry, but there is currently no way around that.  TIdHTTP has to do at
least minimal checking to make sure the server is not sending data that it
should not be sending in the first place.

> It's not clear to me why that DiscardContent stuff was added in the first

The comments in the code explained that.




In response to

Issue with TIdHttp.Head() routine posted by Scott Bussinger on Fri, 30 Jun 2006