Re: where to submit fixes?

Giganews Newsgroups
Subject: Re: where to submit fixes?
Posted by:  Remy Lebeau \(Indy Team\) (re…
Date: Thu, 6 Jan 2011

"jvlad" <dm…> wrote in message

> you may want to try download a 4GB file using HTTP
> component with compression turned on. Under a 32bit
> OS it won't go due to a deficiency in compressor's
> implementation.  It will try to allocate whole the file in
> memory then uncompress it.

And the problem is...?  If the client app is downloading to a file, then the
file gets pre-sized to 4GB, which optimizes writes to it.  If the client app
is downloading to memory, then it will take up 4GB whether the memory is
pre-allocated or not, and the decompression is going to fail either way
since a 32-bit process cannot allocate 4GB of contigious memory.  At least
the pre-allocation allows the failure to be detected before the data is
downloaded, rather than during the download.

> Please let me know where to find instructions on how to
> submit patches.

You can post the patch here and we will review it, but offhand I do not see
how it will make much difference.

On the other hand, what would be nice is for TIdHTTP be able to decompress
data from the IOHandler to the destination TStream directly, much like
TIdFTP can.  The main reason it does not right now is because HTTP provides
a few different ways of packaging the data it sends, whereas FTP does not.
So TIdHTTP reads the inbound data into an intermediate TMemoryStream first,
based on how the HTTP server sends it, and then it decompresses that into
the target TStream.  If the HTTP read handling were integrated into the
decompression logic, such as by adding new DecompressHTTPFromIO() type of
methods into TIdZLibCompressorBase and TIdCompressorZLib, then the
intermediate TMemoryStream could be removed.  That would at least free up
the memory used by the compressed data and make it available for the
decompressed data.  But you will still have problems when the decompression
reaches beyond the >2GB limit imposed by the OS.

Remy Lebeau (TeamB)


In response to

where to submit fixes? posted by jvlad on Fri, 7 Jan 2011