The number of messages per second a product can process is dependent on the XML parser employed. The SOAP request must be decoded by the parser and then handed off to the appropriate Java method. An inefficient XML parser will degrade performance significantly as the size of the request increases, because there is more XML to decode.
Also entering the equation is the way in which a platform marshals and unmarshals arguments--in English, the way a platform transforms an XML element into a Java argument and vice versa. The way in which this is handled varies according to implementation. For example, Cape Clear's implementation handles DOC/LIT differently from the way it does RPC/ENC. In the DOC/LIT model, every argument--even primitive data types, such as an integer--is encoded into a corresponding Java class. That class may be simple, consisting of a single piece of data, or complex, a complete object that may comprise individual objects within it. In the RPC/ENC model, the arguments are often passed to the Web service as a primitive, which can increase performance.
Some platforms, such as those from Novell or Systinet, use the DOC/LIT model of processing even for RPC/ENC, which can degrade performance because of the additional instantiation of objects which are, in some regard, spurious. BEA provides an extra level of abstraction that allows elements within a SOAP request to be mapped to Java methods or arguments that do not necessarily share the same name. This is a definite value-add, making BEA's implementation more flexible, but it does add additional time to the process of marshaling and unmarshaling arguments.
Because Web services are in their infancy, it is not expected that the implementation of a Web service will result in it being inundated with thousands of messages per second. If that happens, we suggest taking advantage of clustering and load balancing, either via the products where available or through an external solution. Because of the real-world use of Web services, we were more concerned with the products' ability to maintain consistent performance as the number of concurrent users increased. In other words, we were examining the ability of each product to scale and were less concerned with the raw message throughput capability of the products under test.
For example, we were impressed with both BEA's and Iona's ability to remain consistent under load, even though the number of messages each processed per second appeared to be low as tested under an out-of-the-box configuration. Sun's product showed a high processing capability while maintaining consistency, while the other products decreased in their processing capabilities as the load increased.