White Papers                  Home |  White Papers  |  Message Board |  Search |  Products |  Purchase | News |  

The Rising Complexity of Software Development



By Rick Strahl



Last Updated: 08/01/2001



Here we are in the age of component development. Web applications are becoming more and more prominent and distributed application development is a term that's getting thrown around more commonly. Over the last few years, software development technology has grown by leaps and bounds and we've been able to build applications that were difficult if not impossible just a short time ago.


So, why is it that with all these technological advances, it seems like software development is getting harder rather than easier? We have so many more tools and opportunities, yet the process of producing software seems to get ever more complex — even for relatively small applications.


Certainly, the sheer volume of technologies available today is boggling to the mind, and programmers find it’s harder and harder to keep up with technology in general. Most developers who have been around for a few years can probably still remember the days when they worked mostly within a single development tool and environment. If you knew C++ or VB or Visual FoxPro or DonkeyBase, you were able to build full-featured applications that made your clients happy.


Today, the world looks very different, with many intertwined technologies requiring an understanding not only of a programming language, but also of the underlying platform and supporting technologies. Nowhere is this breadth of knowledge requirement more apparent than in Web development.


You have to know your backend application software (today mostly a COM-based environment with components in VB, VFP or C++), a scripting engine that can drive these COM objects (something like ASP, Cold Fusion or Web Connection), HTML design to actually display your content, client side scripting which is in yet another language (JavaScript most likely) and last, but not least, a solid understanding of at least the basics of how a Web application works and passes data over the wire. You also have to know about stateless programming, keeping track of users, and managing security in your application.


And, in addition to the application issues, you also must deal with network issues including security, bandwidth availability and scalability, as well as understanding the operating system and the server software installed on it. Then, finally, you need to know how to install everything in the right order to make it all work. If you're building distributed applications that utilize XML and SOAP, things get even trickier, since you're actually working on two or more applications simultaneously.


Is your head spinning yet? If you're like most developers, you're probably scratching your head and thinking, “Where do I start?” Maybe you have been keeping up by reading the programmer's journals and playing with the latest and greatest, and perhaps you’ve already gotten your feet wet with building Web and distributed applications. Even then, it's very difficult to keep the whole picture in view in order to make smart decisions on what works and what doesn't. And, I'm sure you've run into situations more than once where you've been overwhelmed by the mere multitude of possible problems that could be throwing your application out of whack! Sound familiar?


There are several implications to this situation. First, we have come to a point where very few

people have a good understanding of the technology as a whole. There are lots of people and companies who have specialized in their own fields of expertise and look at the world through that limited vision. What has worked in the past may no longer be the best solution, but many people have given up on keeping up because of the continual changes in technology.


As a side effect, it's getting harder and harder to find qualified developers who can build applications with new and upcoming technologies, and rates are skyrocketing as a result. It's also very difficult for people in charge of hiring to make educated decisions on the skill levels of the developers they recruit. A lot of hiring and consulting bids today start on a pure trial by fire basis.

Will it get better?

The big software companies providing the technologies and development tools claim they're doing their part to make life easier for developers by providing solutions, working on standards and making tools everyone can use. In some ways, this is true — especially where new functionality is involved. But in other ways, such as ease of use and implementation, there's a lot of counterproductive behavior going on in our industry.


There's a lot of talk about standards and interoperability of tools and technologies. There's also plenty of discussion about keeping standards simple, so they can be easily implemented. However, much of the technology being developed these days is not aimed at making life easier for developers, but simply is focused on providing more functionality.


A few examples include the myriad of XML standards. XML was designed to be a simple data interchange format that is flexible, extensible and easy to implement. But, anyone who has followed the latest standards and is trying to keep up knows that most of the new XML initiatives are difficult to grasp at best and even harder to actually use. Standards are drafted with inconsistencies in naming and formatting and often in cryptic relationships that are difficult to interpret and parse. There are books filled with explanations of a “simple” standard that has run amok. The power is there — it's just a bear to figure out how to use it!


This type of scenario, repeated in many places, results in greater complexity for the tool builders and more incompatibility between the various tools. For example, take a look at the WSDL standard (a sort of type library service for Web Services) in relation to the SOAP specification, and see how a simple hierarchical class structure can be disassembled into a mess that’s difficult to parse and understand. It's no wonder that the various SOAP implementations are not interoperating well today!

Perpetual beta

Another all-too-common problem is the state of perpetual beta. Software companies, especially the big tool vendors, throw out technology that is not ready for mission-critical deployment, yet the companies and the trade press that cover them treat the software as if it were final. If you pick up a programming magazine today, you undoubtedly will find a large chunk of it dedicated to Microsoft's forthcoming .Net platform — a product that's been in pre-beta and beta for over a year now and probably isn't going to release until the end of this year. Yet, we're hearing about it as if it is already a viable product, with magazine articles that are incomplete or out of date because it's beta and things are changing constantly. As bad as that sounds, those articles tend to be the best documentation available, since the “product” will probably ship with incomplete documentation and few samples.


It goes as far as having a Microsoft representative at the April 2001 FoxTeach event announcing, “.Net is the fastest, most robust and reliable way to build Internet applications today!” Maybe this was a beta test of the marketing message that Microsoft is pushing? Unfortunately, the Microsoft spokesperson was serious. Here we have unproven and unfinished software that is draining resources from developers and those who train developers — all for a product that is still many months away from release. And, everyone is working blind, because the documentation is not all there.


Because of the news coverage, people in charge are taking note and are now not sure how to move forward with new projects. “Should we stick with tried and true or go with the new hot and totally unproven now?” People had already started building solutions on the first versions of the .NET beta, even though the stability was questionable at best. Now, the many changes in Beta 2 have just broken many of those running applications.


Another beta example from Microsoft is the MSSOAP Toolkit. That product was in beta for over 1 year, and went through one complete overhaul that left the old version worthless. Of course, as soon as the new version was released in beta state, it was announced that the old version would no longer be supported. Again, many people have wasted a fair amount of time working with early versions of these tools, only to end up with lots of code that is completely incompatible with the final version.

The new version of the MSSOAP Toolkit was recently in its Beta 2 stage, and has finally been released. But, Beta 2 was treated like a full product — my, how times have changed. Remember the good old days when magazines would have maybe a simple preview of up and coming software tools, but would never print anything based on beta software? Those days are gone, and perpetual beta now preys on your time.


Software development in this day and age won't be easy. No matter how good the tools, it's important to understand the underlying architecture and the way things work, to be a solid developer. However, I think we ought to make our voices heard with the tool vendors. Let them know that perpetual beta testing by users is not the best way to build quality soft-are — it's counterproductive and takes away from the productivity of the people writing code. It's also important to have stable technology that doesn't change every few months. People have to be able to absorb the new technologies in order to become proficient and actually build viable solutions. Documentation and samples are paramount — another peg that has slipped badly over the years as the new complexities make it ever more important. 

What can we do?

We developers need to let tool companies know that one major aspect of software development should be making the job easier — easier for you, and not just what the vendor considers easier. There is a difference. To vendors, something that demos nicely is easy. To you, something that saves you time all day long is easy. We also need to ask for simplicity and elegance in the design of frameworks and tools, and simplicity and accessibility in standards. It doesn't work to have standards that do everything under the sun, but are so complex that nobody can understand or use them. It's time to make yourself heard, so talk to your software representatives. Post messages on news and discussion groups. It makes a difference, and the powers that be need to hear that all is not well in fairyland.


You can also prepare yourself for “the new stuff” by learning about generic technologies like XML, system architecture, Web development and design, rather than focusing exclusively on the tools that implement them. It's always been my feeling that an understanding of how the technology works behind the scenes is more important than just knowing that it works. That knowledge will let you build your applications with confidence and will help you switch between tools more easily.


If you are a developer who wants to keep up, but also has tight deadlines to meet, my advice is to back off from the bleeding edge and stay one step behind. Don't use beta software, or if you do, use only late betas — don't encourage software vendors to let you do their testing for them! And, most importantly, remember that good application architecture and design skills often are much more important than new features in developer tools. Most of all, stay realistic about what you want to accomplish and what you really need to get the job done, using tools that have been proven instead of the latest vaporware.


For comments, you can post a message at:



Rick Strahl is president of West Wind Technologies on Maui, Hawaii. The company specializes in Web and distributed application development and tools with focus on Windows 2000, ISAPI, .Net and Visual Studio. Rick is author of West Wind Web Connection, a powerful and widely used Web application framework for Visual FoxPro and West Wind HTML Help Builder. He's also a Microsoft Most Valuable Professional, and a frequent contributor to magazines and books. He is co-publisher and co-editor of CoDe magazine, and his book, "Internet Applications with Visual FoxPro 6.0", is published by Hentzenwerke Publishing. For more information please visit: http://www.west-wind.com/.


Amazon Honor System Click Here to Pay Learn More





  White Papers                  Home |  White Papers  |  Message Board |  Search |  Products |  Purchase | News |