I've been spending some time today making a few mods in the Web Connection ISAPI interface. Specifically a discussion that I had on the Message Board with Randy Pearson a few days ago made me think about adding better support for large file output from a Web Connection applications.
The problem is that Web Connection running under COM is limited by the string capacities that Visual FoxPro can handle which is limited to 16 megs. So if you try to generate output bigger than 16megs and send it back in the output, Visual FoxPro will error out. Actually, the truth is that it's possible that this will occur sooner because Web Connection does string concats to build the COM output and depending on when the chunk gets written it might occur before the string actually hits 16 megs. Along the same lines Web Connection in that scenario would be using HUGE amounts of memory because as mentioned the string gets shuffled back and forth a bit during concats. So there'd be at least two copies in memory (one from the caller who created the string, and one from the storage on the class). There may be more depending on how the user created the string.The ISAPI extension too might have multiple copies of the string as the string gets shuffled from a COM string into a binary Ansi string to push down the wire.
So... I implemented a new method on the Response object called TransmitFile. TransmitFile basically sends allows sending output directly from a file to the ISAPI extension and back to the client without ever having to load up a string in the Visual FoxPro end, or even passing this huge string through the ISAPI extension. Rather Transmit file uses a pointer to a file on disk, and reads and transmits the file directly. Not only is this much more efficient as it relieves resource usage in both VFP and the ISAPI interface but it also allows creation of truly dynamic content into files for regular Web Connection requests.
Here are a couple of examples that demonstrates how this works. If you simply want to send a file to the client to display you can use the following:
FUNCTION TransmitFile
Response.TransmitFile("d:\sailbig.jpg","image/jpeg")
RETURN
Of course you can also dynamically generate this file or otherwise use a dynamic path. Note that Web Connection use RevertToSelf() to set its security environment when it sends this file which means it reverts back to the underlying system account (ie. SYSTEM in IIS 5 and earlier - or whatever account your Web Connection Application pool is running under - usually system, or possibly a specific account).
Transmit file simply creates an HTTP header and sends the file and it will display in the browser. If you want a Download dialog, you can use Response.DownloadFile() which adds a Content-Disposition header to force a Save As dialog to popup.
The above is sending a pre-existing file directly from disk, but you can also use this functionality to dynamically generate huge output. For example, let's say you wanted to created large HTML output that exceeds 16megs like this:
FUNCTION HugeOutput
LOCAL lcOutputFile, loResponse
lcOutputFile = Server.oConfig.oWwDemo.cHtmlPagePath + ;
"temp\" + SYS(2015) + ".htm"
loResponse = CREATEOBJECT("wwResponseFile",lcOutputFile)
FOR x=1 TO 1000000
loResponse.Write("012345678901234567890" + "<br>")
ENDFOR
loResponse = .null.
Response.TransmitFile(lcOutputFile,"text/html")
*** Some housekeeping
DeleteFiles(JustPath( lcOutputFile ) + "\_*.*",900) && timeout in 15 minutes
ENDFUNC
Here the trick is to NOT use the default Response object for output generation because in COM this object at some point will always point to string which it has to return to the ISAPI extension.
Instead you can use a new wwResponseFile object which writes output to a file. Not only is this required in order for TransmitFile to work, but it's also vastly faster than doing string concatenation on a string this size (assuming it would even fit). Strings over a few megs in VFP become very slow and string concatenation becomes painfully slow. Using wwResponseFile is good practice if you're generating large output that exceeds a few megs.
The output gets written to a file which is then send down with TransmitFile. Note that I use the wwUtils DeleteFiles() function to clean up the temp directory after 15 minutes - DeleteFiles deletes all files older than 15 minutes in the path specified.
How does it work
TransmitFile actually doesn't send the file to the ISAPI extension but rather it sends a file pointer to it, which the ISAPI extension picks up and writes. If you look at the output generated from a request that calls TransmitFile you'd see:
HTTP/1.1 200 OK
Content-type: text/html
Content-Length: 25000000
WC_TRANSMITFILE: d:\westwind\wconnect\temp\_1GI1E5RHH.htm
As you can see Web Connection is generating a proper HTML header including the content size for the output. The ISAPI extension then picks up this file referenced and sends the file back.
This functionality is implemented for both COM and File based operation and as mentioned above the Response.DownloadFile() functionality has also been updated to use TransmitFile since it's obviously much more reliable.
Security
Security of the TransmitFile operation runs under the machine account that is running the ISAPI extension. In other words, not the Impersonated account (IUSR_) but the underlying machine account, usually SYSTEM. In IIS 6 this is configurable via the Application Pool Impersonation setting - in IIS 5 and earlier it will be System. If SYSTEM is the operative account this means files have to come from the local machine since SYSTEM doesn't have rights to read files from Network locations.