After the ongoing discussion on the Universal Thread with Claude "I'll use an MTDLL to fix my kitchen sink" Fuchs I decided to again do some performance checks on West Wind Web Connection. Claude continues to point out that ASP + COM is faster, more scalable, easier and free. I can't argue the last point but on the other counts Claude is going to have to back up his arguments at some point.

For those not familiar with West Wind Web Connection - it's a Web interface for Visual FoxPro that basically provides a framework to build Web applications using pure Visual FoxPro without requiring ASP or ASP.NET. With it you can build and run your entire Web application from within the Visual FoxPro environment from coding through debugging without the headaches of dealing with COM objects during the development process.

We’ve had a number of discussions going back and forth over this. I’ve never been a fan of ASP + COM or ASP.NET + COM for that matter and I've been the counter point to Claude's “VFP COM components are the end all of Web development“. I love working and running pure ASP.NET, but COM in either ASP environments has always been a headache to work with and to manage at deployment and runtime. As Claude points out though it can be done if you’re willing to live with the implications.

Let's compare (from my obviously subjective perspective mind you <g>), shall we?

Development

My primary beef with COM inside of IIS is that the ASP debugging cycle with COM objects is a pain in the ass. First off every time you want to run your app you have to compile your server. Before you do you have to make sure that IIS (or the hosting COM+ application if you’re going that route) is shut down if the component was previously loaded. If you do this manually you have to run a program (IISReset) or script (to shut down either IIS or the COM+ hosting container). Alternately you can hook up a Project Hook to fire this operation automatically but even if automatic this process takes time and requires a recompile no matter what. For deployment it gets even worse as you potentially have to re-register the COM object (if the type library interface has changed).

In addition, by default there’s zero native debugging support for COM components – COM components are compiled code and cannot be debugged at runtime. There are workarounds for this by basically instantiating your components into the Visual FoxPro development environment and debugging them  (see here and here) there which is a hack at best. It works but it’s very clumsy, unnatural and prone to problems (such as object being corrupt, or timeing out on you – especially true for the intrinsic ASP Context objects).

West Wind Web Connection (and other tools like Active FoxPro Pages) on the other hand uses a separate mechanism to allow the application to run at design time without having to run as a COM server. This allows the ability to build and debug your application in the Visual FoxPro environment. You DO <YourWebApp>.prg, and the application server runs inside of Visual FoxPro. An error occurs and if no error handling is enabled VFP stops and the debugger pops up on the offending line of code. In other words the process is identical to the way you build desktop applications. West Wind Web Connection uses a file based mechanism for this debugging mode. For deployment the app can then be compiled into a COM server which gets called much more efficiently from the Web Server through COM and the highly efficient and VFP optimized thread pool that is the core of the West Wind Web Connection ISAPI connector (wc.dll).

To me at least right here is the most significant reason of why I never enjoyed working with ASP + COM. I don’t know about you but I don’t write code perfectly the first time and I go back and forth a lot and having to constantly compile and wait for the COM+ package or IIS to shut down and restart. This is an extra step or delay that gets in the way of work.

Add to that are the deployment issues of having to do this same maneuver on a live server which invariably requires access to the server. There’s no way I know of to remotely install the COM component unless you have full admin privileges and a full remote connection to the server. Often you will not only to update the component but also reregister it if the COM interface changes which requires you ‘to be there’.

Performance

Performance is always important for any Web application. West Wind Web Connection and ASP+COM do fine for performance in most situations. In fact, since both solution use Visual FoxPro the biggest bottleneck for performance isn’t really the call architecture (ie. ISAPI to ASP into COM or ISAPI -> West Wind Web Connection -> COM). In pure performance terms of call overhead in the past ASP + COM had a slight edge over West Wind Web Connection. This is due to the fact that ASP is running InProc to IIS while West Wind Web Connection uses Out Of Process COM servers to communicate with IIS. At first blush this seems a big hit, but as it turns out a properly managed Out of Process component can wring out better performance than an InProcess VFP component.

ASP + COM loads COM objects on EVERY HIT and every access of a VFP method or property occurs over COM boundaries. This means that every hit automatically incurs the overhead of loading a VFP object. ASP optimizes this process a bit by ‘caching’ any COM object it loads by doing an extra AddRef on it. This keeps the VFP runtime loaded in IIS at all times, so object loads are fast. However, there still is overhead in loading the object every time both in terms of the COM overhead as well as VFP having to construct the object. If you’re using the recommended Session object there’s a fair amount of overhead because it basically needs to set up a data environment and reset a number of system settings (SET EXLUSIVE, SET DELETED etc.). In addition any access to the VFP object from an ASP page or VFP accessing any of the ASP Context object properties/methods goes over COM boundaries which by itself has considerable overhead (compared to native VFP property or method access).

West Wind Web Connection on the other hand runs as a small set of objects that are pooled by the ISAPI extension. WC runs a pool of COM servers and keeps them alive. When a new request comes in it doesn’t create a new instance but rather hands off an already running instance from the pool. There’s no overhead in constructing this object. Additionally this object can maintain its running state between hits meaning you can do things like leave data files open and leave expensive-to-load objects attached to the server object. This is very important when dealing with VFP data – and you’ll see why this particular issue affects ASP even in the simple perf tests below. ASP must reopen data files on every hit – you can’t cache anything in ASP COM objects because objects go away after each hit.

As it turns out in terms of pure call overhead ASP+COM and West Wind Web Connection (in COM mode) have roughly the same performance parameters. But as requests get more complex West Wind Web Connection holds an edge, especially if the ASP page makes multiple calls into the COM object. West Wind Web Connection always uses a single COM call per request, while ASP + COM may use one (if you have an ASP framework that basically does all the work – CGI style just like West Wind Web Connection) or many if you’re using VFP business objects inside of an ASP page.

So, I sat down today and set up some simple perf tests that demonstrate actual performance to clear up this confusion. I'm going to show code for each of these so you can set these up yourself.

My hardware for testing:

• 2.4Ghz Dell Inspiron 8500
• 1gb of memory
• Windows 2003 Server

Obviously not the most highend hardware, but we're not so much concerned with raw throughput as with comparative numbers.

I ran several tests here:

• A simple HelloWorld request
• A simple data IncCounter request
• A more complex E-Commerce application

I ran these both in West Wind Web Connection and ASP + COM and ASP + COM+ + COM environments. I didn’t bother with ASP.NET – since it will be slower than ASP+COM (there are perf numbers that compare ASP and ASP. Net here)

First off, let me say that when I ran the West Wind Web Connection tests I was a little surprised by the numbers I saw! I haven't done these tests in a while and in the past ASP was ahead in the very simple examples because ASP. Net’s call InProc call overhead generally is very fast. This is no longer the case however, as West Wind Web Connection has caught up with ASP on that count. However, the numbers changed so drastically that at first I didn't quite believe what I was seeing, but I was able to verify the numbers though with the IncCounter sample which essentially logs every hit. In short, West Wind Web Connection was able to achieve in excess of 300 requests/second for the simple request, as did ASP with plain COM.

Let’s start by showing the results and then I’ll explain what I did in these samples. Here are the performance numbers given in requests per second running Microsoft Web Application Stress Tool. Each of the applications were set up with 25 threads and a stress multiplier of 2 with no delays - this basically provides non-stop pounding without pauses of the app of 50 true simultaneous users. Note this is not the same as 50 normal users on your site since no real users wouldn’t be clicking manically like this. This test is designed to purposefully max out the applications to feed them as many requests as they can handle

I ran each test 3 times and then took the average value. Most of the test were nearly identical in all three runs though.

Note: The code for the simple samples is posted at the end of this entry.

The numbers are interesting in that since my last tests the numbers have jumped drastically. Incidentally I ran these tests under VFP 9. I then compiled under VFP 8 and found that performance dropped about 10% for the simple tests, but stayed roughly the same for more complex tests. This seems to indicate that VFP 9 is more efficient in making the COM calls.

Note that the real comparison here is between West Wind Web Connection (COM) and ASP/COM. West Wind Web Connection in File mode is really not meant for deployed applications (although that isn’t stopping people from using it that way anyway <shrug>). The real surprise here is how badly COM+ fares in this scenario – COM+ is by far the slowest mechanism and a whopping three times slower than ASP without it. COM+ is great when you need the features it provides (distributed transactions, security, remoting, events, queueing etc. – none of which are likely to be important for the typical VFP Web app) but it sure sucks for performance.

As you can see in the numbers above West Wind Web Connection and ASP + COM roughly have the same overhead in making COM calls. The Hello World sample basically measures only call overhead – that is getting to the VFP code, and returning the result. Since the VFP code doesn’t really do much at all, the full overhead comes from the passage through the pipeline – from IIS -> ISAPI -> ASP/WC -> COM and back. Interestingly enough in the past ASP + COM had a decided edge over West Wind Web Connection in raw call overhead, but there have been some enhancements in the West Wind Web Connection ISAPI interface recently that have optimized the pool management code providing better performance that now just about matches ASP.

You can see some of the effects of ASP and data loading in the IncCounter sample. West Wind Web Connection can keep the (small) table open on each request while ASP has to re-open on every hit which causes the numbers for ASP to drop a bit compared to the HelloWorld COM sample. Because the table stays open in WWWC there’s very little extra overhead in the request and almost no difference in request times between HelloWorld and IncCounter. If you were dealing with many and larger tables this sort of drop off would become much more pronounced as ASP+COM must reopen tables on every hit.

The HelloWorld2 sample was provided to demonstrate the effect of additional COM calls from within the VFP component. It calls the ASP Object Context to retrieve the intrinsic Response object and then writes out to this object. Compared to the plain HelloWorld output this is significantly slower – 25%. If you can avoid retrieving the COM+ Object Context inside of your COM component you can save yourself a good chunk of overhead. IOW, returning values and letting the ASP page handle display of the data or passing Request values as parameters rather than retrieving them in your component might be more efficient.

The West Wind Web Store sample runs the West Wind West Wind Web Store E-Commerce application which was originally designed for Web Connection. However, I have previously set up an ASP and ASP.Net demo for ASP/ASP. Net pages using the wwStore business objects for some conference sessions to demonstrate what’s involved in migrating from ASP to ASP.NET using VFP business objects. This isn’t a 1 to 1 match, but it’s doing roughly the same thing. You’ll notice that in the West Wind Web Store sample the times here don’t vary wildly – this is because most of the processing occurs inside of the business object and we’re not dealing with a typical Hello World sample.

The bottom line here is that inside of a real business application that does real work call overhead is the least of your worries. 90% of the request overhead comes from the code that runs inside of VFP accessing data. Since everything uses VFP for this processing overall there’s very little over all performance difference assuming the same code. However, writing ASP or WWWC code is very different and the approach may vary widely. Making tons of COM calls in ASP can be expensive. But if you (like Claude’s ActiveVFP) merely use ASP as a connector interface to a VFP framework then there’s usually just a single COM call involved.

What's interesting here is the huge performance difference between COM and File based messaging modes in West Wind Web Connection. This problem is slightly exaggerated in this test due to running on a laptop with a slower hard disk. Since this stress test puts the app under serious load File based ends up being slower by almost a third. Let that be a reminder for those of you that are balking at moving your West Wind Web Connection apps to COM and are facing load issues.

Scalability

Ah yes, VFP and scalability. It’s always an issue because VFP is a single threaded environment. Were performance deals mostly with individual requests, scalability deals with the wider range usability for many users. How many users can access the site before it breaks? Scalability is not something that can be easily expressed by numbers because it depends intrinsically on your environment – CPU, disk, what else is running etc. defines this scope.

VFP faces special challenges here because VFP is not truly multithreaded and needs help via simulation of multi-threading. ASP accomplishes this with STA Model (Single Threaded Apartment) COM invocations – that is a thread marshaled interface that invokes a component on a thread and isolates it on that thread to allow multiple components to run separate threads simultaneously.

West Wind Web Connection uses an internal thread pool that instantiates COM components on a pool thread and keeps it alive on this thread. As requests come in hands of these instances for processing of requests. One advantage of this approach is that West Wind Web Connection has control over the thread pool, the lifetime of the VFP instance (so it can be started and stopped and recycled on demand) and over the execution of the COM object so it can immediately detect failures and timeouts and take appropriate action such as restarting crashed or hung server.

Both approaches work and as the performance values above show have roughly the same amount of overhead. ASP’s InProc mechanism is faster than WWWC’s OutOfProc access, but West Wind Web Connection is more efficient because it doesn’t have to start up instances. The end result for base call overhead is similar.

The real issue for scaling on the local machine deals with long and CPU intensive requests. VFP is notoriously bad at processing complex SQL statements or other query and report commands and eating up all available CPU. This causes problems when multiple complex requests are running side by side. They end up competing for the CPU and if enough requests get fired at the same time can easily cause a deadlock situation where the incoming request queue gets flooded and timed out before the request even gets around to be processed.

ASP handles multiple requests by simply letting them all come and be processed up to a maximum amount which is equal to the number of worker threads made available to ASP. This number is 25 by default and can be changed in the registry (ASP. Net has a similar setting for the thread pool and ASP. Net IO threads which can both be configured in machine.config). The potential problem here is that too many simultaneous requests have the potential to lock the server. The more request running simultaneously the more chance for overload situations. This is a problem only if there are long running CPU intensive requests that are part of the application.

West Wind Web Connection works differently as it has a fixed size pool of server instances that are pre-configured and loaded up front, which limits the number of simultaneous requests can run at any time. The recommendation is 2-3 instances per processor in typical applications. This minimizes issues with too many simultaneous requests processing at the same time and vying for limited CPU resources. In addition, West Wind Web Connection also has a URL based switch that allows requests that are known to be long running to be executed outside of the pool of preloaded instances. This allows this long request to leave the pool intact and thus let shorter requests continue to run.

Long running requests are a problem in any Web application and the ideal solution for this regardless of ASP or WWWC is to offload these processes asynchronously to another machine and indirectly wait for completion. For one approach to handle this check out this article.

Finally scalability can be achieved by ‘scaling out’ to another machine or machines using Load Balancing software. This mechanism is available to any Web platform (although ASP can be a problem if you’re using Session state). It basically works through tying multiple machines together into a Virtual IP address – one address that automatically routes to a pool of servers. There are many commercial solutions available and an included solution in the Network Load Balancing Service that is bundled with Windows Server 2003 (or Windows 2000 Advanced Server). For more information check out this article.

Stability

Part of the reason I’ve always been skeptical of ASP + COM is the fact that the InProc environment of COM objects gives the application no control over failures that occur in the COM object. While regular errors can be easily handled with Error methods or COMRETURNERROR, there’s no way to handle a C5 exception. The problem here is that with ASP + COM there’s no mechanism to capture this failure and act on it. Worse, if the failure is severe enough or repeats, it will corrupt the running COM server and cause that COM server to stop responding. It will not reload, but simply stop responding. IIS will continue running. This makes it hard to detect the failure.

This is easy to demonstrate BTW, in a concocted but still valid test:

• Create a COM Server for use in ASP
• Add a method with this code:

PROCEDURE CrashGPF

DECLARE GetSystemTime ;
   IN WIN32API ;
   STRING @
  
lcText=" "
lcText = GetSystemTime(@lcText)

RETURN "Made it through"
ENDFUNC

• Compile the whole thing into an MTDLL
• Execute from an ASP page with:

<%

Result = CrashGpf()

Response.Write("Made it through a GPF")

%>

When you run this you will find that the COM server returns an error the first time. A more cryptic error the second time, and outright refuses to load on the third failure. From that point forward the COM object will no longer respond. The behavior you see may vary and while this is an extreme constructed scenario, this behavior is quite possible.

Unfortunately VFP does crash from time to time. It’s getting rarer and rarer especially with VFP 9 but it still happens and is more likely to happen when your server is under load.

West Wind Web Connection handles failures more directly because its COM pool manager actually is responsible for running the COM component. It can capture the exception if it should occur and destroy the bad server instance (or try to anyway – a C5 usually leaves no reference only a ref count <g>) and start up a new one. Along the same lines it can also catch a hung server – one that has some sort of endless loop or other failure that causes untimely response. West Wind Web Connection can kill the server based on configurable timeout and restart it. In short there’s a manager object presiding over the actual COM invocation calls that monitors the operation of the Web -> VFP interface and can detect many abnormalities.

It can also shut down the servers – remotely if you choose – and automatically hot swap the Executable at runtime without shutting down the Web Service or anything else. A built in mechanism unloads the servers swaps in a new binary, then restarts the servers with a single command – an operation that takes 5-10 seconds at most.

Samples

West Wind Web Connection Setup
For these samples West Wind Web Connection is set up with a plain jane setup. Install defaults for wcdemo application except the following:

·         Compiled to an EXE COM server

·         Server configured to not log and not display request info

These last settings are set merely to provide the apples to apples comparison to ASP which performs neither logging nor request display. Adding this back in will add approximately 20% overhead in these short hit samples (mostly due to the screen UI – note that West Wind Web Connection can also run completely invisibly) .

The two methods are implemented in the standard wwDemo class that comes with West Wind Web Connection (you can get the shareware version from here although you can't compile the shareware version to a COM object). I simply added two methods:

FUNCTION HelloWorld()

Response.ContentType = "text/html"

Response.Write("<html><body>Hello World</body></html>")

RETURN

************************************************************************

* aspTools :: IncCounter

*********************************

***  Function: Increments a counter in the registry.

***      Pass: lcCounter  -  Name of counter to increase

***            lnValue    -  (optional) Set the value of the counter

***    Return: Increased  Counter value  -  -1 on failure

************************************************************************

FUNCTION IncCounter()

LOCAL lnValue

lcCounter = "Test"

lnSetValue = 0

*lnSetValue=IIF(EMPTY(lnSetValue),0,lnSetValue)

IF !USED("WebCounters")

   IF !FILE("WebCounters.dbf")

        SELE 0

      CREATE table ("WEBCOUNTERS") ;

      (    NAME        C (20),;

           COUNTER       I )

       USE

   ENDIF

   USE ( "WEBCOUNTERS") IN 0 ALIAS WebCounters

ENDIF

SELE WebCounters

LOCATE FOR UPPER(name) = UPPER(lcCounter)

IF !FOUND()

   INSERT INTO WEBCounters  VALUES (lcCounter,1)

   lnValue = 1

ELSE

   IF RLOCK()

      IF lnSetValue > 0

         REPLACE Counter with lnSetValue

      ELSE

         IF lnSetValue < 0

            REPLACE Counter with 0

            DELETE

         ELSE

            REPLACE Counter with Counter + 1

         ENDIF

      ENDIF

      lnValue = Counter

      UNLOCK

   ELSE

      lnValue = 0

   ENDIF  

ENDIF

Response.Write(TRANSFORM(lnValue))

*

RETURN lnValue

ENDFUNC

ASP Setup
ASP is also configured very basic. I used the default settings on the server. The samples are based on the ASP.NET and ASP samples I published a while back. You can get the code for the server here. Here's the base interface of this server:

*************************************************************

DEFINE CLASS ASPTools AS Custom OLEPUBLIC

*************************************************************

***    Author: Rick Strahl

***            (c) West Wind Technologies, 1996

***   Contact: www.west-wind.com

***  Function: Some example routines for implementing

***            a page counter and custom logging as

***            well as a very simple example on how to

***            send HTML back to embed inside an ASP page

*************************************************************

cAppStartPath = ""

oScriptingContext = .NULL.

************************************************************************

* aspTools :: Init

*********************************

***  Function: Set the server's environment. IMPORTANT!

************************************************************************

FUNCTION INIT

*** Make all required environment settings here

*** KEEP IT SIMPLE: Remember your object is created

***                 on EVERY ASP page hit!

SET RESOURCE OFF   && Best to compile into a CONFIG.FPW

SET EXCLUSIVE OFF

SET REPROCESS TO 2 SECONDS

SET CPDIALOG OFF

SET DELETED ON

SET EXACT OFF

SET SAFETY OFF

*** IMPORTANT: Figure out your DLL startup path

IF application.Startmode = 3 OR Application.StartMode = 5

      THIS.cAppStartPath = ADDBS(JUSTPATH(Application.ServerName))

ELSE

    THIS.cAppStartPath = SYS(5) + ADDBS(CURDIR())

ENDIF

*** If you access VFP data you probably will have to

*** use this path plus a relative path to get to it!

*** You can SET PATH here, or else always access data

*** with the explicit path

DO PATH WITH THIS.cAppStartpath

*oMTS  = CreateObject("MTxAS.AppServer.1")

*THIS.oScriptingContext = oMTS.GetObjectContext()

ENDFUNC

FUNCTION HelloWorld

   RETURN "Hello World"

ENDFUNC

FUNCTION Hello2World

   oMTS  = CreateObject("MTxAS.AppServer.1")

   THIS.oScriptingContext = oMTS.GetObjectContext()

  loResponse = THIS.oScriptingContext.item("Response")

  loResponse.ContentType = "text/html"

  loResponse.Write("<html><body>Hello World</body></html>")

  loResponse.End()

ENDFUNC

************************************************************************

* aspTools :: IncCounter

*********************************

***  Function: Increments a counter in the registry.

***      Pass: lcCounter  -  Name of counter to increase

***            lnValue    -  (optional) Set the value of the counter

***    Return: Increased  Counter value  -  -1 on failure

************************************************************************

FUNCTION IncCounter(lcCounter as String, lnSetValue as Integer) as Integer

LOCAL lnValue

SET STEP ON

lnSetValue=IIF(EMPTY(lnSetValue),0,lnSetValue)

IF !USED("WebCounters")

   IF !FILE(THIS.cAppStartPath + "WebCounters.dbf")

        SELE 0

            CREATE table (THIS.cAppStartPath + "WEBCOUNTERS") ;

            (    NAME        C (20),;

                 COUNTER       I )

             USE

   ENDIF

   USE (THIS.cAppStartPath + "WEBCOUNTERS") IN 0 ALIAS WebCounters

ENDIF

SELE WebCounters

LOCATE FOR UPPER(name) = UPPER(lcCounter)

IF !FOUND()

   INSERT INTO WEBCounters  VALUES (lcCounter,1)

   lnValue = 1

ELSE

   IF RLOCK()

      IF lnSetValue > 0

         REPLACE Counter with lnSetValue

      ELSE

         IF lnSetValue < 0

            REPLACE Counter with 0

            DELETE

         ELSE

            REPLACE Counter with Counter + 1

         ENDIF

      ENDIF

      lnValue = Counter

      UNLOCK

   ELSE

      lnValue = 0

   ENDIF  

ENDIF

RETURN lnValue

ENDFUNC

ENDDEFINE

This is compiled into an MTDLL and then called from ASP with the following scripts.

HelloWorld.asp

<%

set oVFP = CreateObject("aspDemos.asptools")

Response.Write( oVFP.HelloWorld() )

%>

HelloWorld2.asp

<%

set oVFP = CreateObject("aspDemos.asptools")

oVFP.Hello2World()

%>

IncCounterSpeed.asp

<%

SET oVFP = Server.CreateObject("aspdemos.asptools")

Response.Write( oVFP.IncCounter("OzFox2004",0) )

%>