In one of his articles on building a CAB-like infrastructure, Jeremy Miller brought out a pretty interesting point: The choice between using Events/Delegates and Direct Method Calls. This can be a controversial topic at hand, but it's an interesting aspect, and can significantly affect the readability of the code.

Events and delegates are very powerful features I use all the time. However, here's a tip: Be careful about building an entire framework or complex structure based purely around delegate calls. I've been spending a lot of time the past few days tracing, debugging and running through the code in the Base Adapter sample code in the BizTalk Server 2004 SDK [1], which is exactly a little framework for building BizTalk Adapters.

Now, the base adapter is OK, in that it does make it a bit faster to develop adapters by taking care of a lot of the boilerplate code necessary. However, a significant part of it is built out of classes that use a lot of delegates (not even proper events, unfortunately) to notify of significant events around each other; like for example that a batch was submitted to BizTalk or that a message needs to be suspended.

The unfortunate side effect of this is that tracing the base adapter code in source code form requires a lot of patience and a lot of Find References/Find Definition jumps all around to figure out exactly what's going on. In fact, I've spent quite a few hours on it and the code can be significantly convoluted; and parts of it are just a lot easier to trace using Step Though/Step Into in the debugger; which is rather frustrating.

So next time you want to build a framework or library and think gluing all together with delegates and events is cool; make sure you've got a real use case for them and that they are not going to end up hindering readability and maintainability.

[1] The version included in the BizTalk 2006 SDK is different and a lot simpler. I still don't quite like it and generally prefer to write directly against the Adapter Framework APIs, but that's just me.

BizTalk, Base Adapter, Delegates

Nicholas Allen has been posting some really great entries on Poison Message handling in WCF and specifically the options the MSMQ binding offers for that. He also mentions that MSMQ v4 (Longhorn server?) will support some new features to make this easier: Retry Queues. Very interesting stuff.

One thing I wanted to comment on to hopefully complement a bit Nicholas' explanations is the relationship between poison messages and in-order processing.

The term "poison messages" might seem to relate to a kind of malicious attack; that is, something intentional. However, that's not necessarily true. A poison message is in essence anything that prevents you from continuing processing by failing repeatedly. This could be something unintentional but well intented, like receiving a malformed message from a trusted business partner that you still need to process.

If you don't need to guarantee in-order processing of the received messages, then dealing with poison messages can be dealt very efficiently in an asynchronous fashion like the one Nicholas describes: You move the problem message somewhere else, and continue processing the messages on the incoming queue; meanwhile you deal with the message, possibly fix it and eventually feed it back into the incoming queue for processing.

However, if you do need to guarantee in-order processing of received messages, this is not an option. If the poison message was a valid business message (like a malformed order from one of your big customers), you can't simply move the message and continue processing. Instead, the situation needs to be resolved in a synchronous fashion: you must inspect and resolve the problem message before you continue processing.

Normally, doing this requires some manual intervention from an operator or analyst, but it also requires your service to proactively deal with this situations. At the least, a responsible application will suspend all processing when it detects a poison message and raise alerts so that the operator knows a problem ocurrs. Notice that it will need to suspend processing, but it doesn't mean it needs to stop receiving messages altogether. Normally, you'll still receive messages and queue them for later processing.

The operator or analyst would then inspect the problem message and determine if it's not something important (or if it indeed was a malicious message) and discard it. If he/she determines it is a valid request message, he/she might need to fix it by hand or with appropriate tools (for example: remove an invalid character that was causing problems or change the message encoding) and then feed it back to the service. In the latter case, you need to provide a way to put the message back into the processing queue in the right position (i.e. at the beginning) and in either case you need to provide a way for the operator to ask the service to restart processing of received messages.

One possible way to support feeding messages to the service for immediate processing and not having to mess around with the incoming queues is to provide private alternative endpoints that bypass the normal receive queues. Fortunately, doing this with WCF is easy because you can expose a single service as multiple endpoints using different bindings, for example, expose an alternative HTTP-based endpoint besides the main MSMQ based one. In BizTalk, for example, one would do it by having an alternative receive location (for this purpose usually a FILE receive location works very well).

One thing to keep in mind is that it's important that these alternate endpoints are private only, are not advertised and are kept secure, as they could be used by rogue agents to disrupt your in-order processing mechanisms.

Windows Communication Foundation, WCF, Architecture

Harry Pierson posted a story about Windows Communication Foundation and his "epiphany" (my words) on the usefulnes of WS-ReliableMessaging in Web Services. I found several comments here that made me think. Sam Gentile recently commented as well on the importance of WS-RM, and I did a few quick posts there.

Let me first start this discussion that I believe that the basic concept behind reliable messaging (and WS-RM) is indeed very important and very needed in the Web Services World to make it easier to implement reliable services and integrate disparate applications.

That said, I've been frank in the past and say that I believe that the WS-RM spec, as it exists right now, is remarkably lacking. Harry mentioned the core point: It does not demand persistence of the message and conversation state from the endpoints involved in the communication. Thus, as WS-RM currently exists, and how platforms such as WCF implement it, it means you don't really have the guarantee of reliablity. At best, all you have is the illussion of reliability. The only thing you can be sure of is that one of your endpoints is gonna fail, sooner or later.

If there is one thing that I have learned from my work on Application Integration (with and without BizTalk), is that ensuring reliable communication across a distributed application infrastructure is a tough nut to crack. There's a lot of very significant issues that can arise here, depending on the type of problem domain you're solving: Message delivery, delayed deliver, synchronous and asynchronous interchanges, ordered deliver (and even tougher, in-order processing), lost and duplicated messages and so on.

A spec like WS-RM doesn't try to solve all this issues, for obvious reasons, and that really is fine. A lot of these issues cannot be taken into account by themselves, but only in the context of the overall architecture and requirements. In particular, how the individual endpoints are designed can make a huge impact in which of the issues mentioned before are significant and how significant.

Some architectural choices will make some of these problems go away without having to write more code for it. For example, a given service might have a set of messages created in such a way that the operations they represent are idempotent. In that case, duplicated messages should not be a problem, and spending a lot of time to avoid them would not be an efficient use of your development budget, as they are a non issue. In other contexts, for example, missing one or two messages during high load might also not be important because perhaps the contents can be reconstructed from subsequent exchanges.

So given this, why do I make so much fuss about WS-RM if it wasn't meant to be the end-all solution anyway? Because of the way it has been positioned by the marketplace: It is usually portrayed as saying that it "finally solves the reliable messaging problem for Web Services". It doesn't. Unfortunately, the WCF literature hasn't helped clear this misconception and actually has made the problem a bit worse.

The other problem I have with WS-RM is that it does a bare minimum of work to help the reliability issue. And for better or worse, it doesn't do what I consider is a significant element of what that bare minimum should be, which is to require implementations to force endpoint applications to survive restarts/crashes. My bigguest concern is people building solutions assuming that because they used WCF and turned the RM bit on then they are done and their services are reliable, because that's not really true. Reliable services take a lot of work, and flipping a bit is just not gonna do it, but unfortunately in the way it has been sold to the masses, it sure sounds like learning about all those pesky reliability issues isn't needed when "WCF just does it for us" [1]...

That's my opinion, by the way, and I might be wrong. You need to do your own decision about whether the required/supported functionality is enough for your needs, but be aware of what it is that WS-RM brings to the table and be sure you're making an informed decision.

Harry made a really interesting comparison in his post: "If HTTP is basically UDP, then WS-* is trying to be TCP". I won't comment on whether the comparison is valid or not, but I do want to say that if we (as in "the community", or the "developer world" or whatever) just spent several years working on reliable messaging for webservices just to get what the underlying transport protocol, invented more than 20 years ago, already did to start with, then boy, we have really lost the train this time.

What I want to say by this is if we're going to work on creating a significant improvement on the reliable messaging field, then we need to move forward to solve higher level problems with higher level abstractions. Abstractions such as Workflow and Orchestration are significant advances in this field, for example, because they make it easier to write the endpoint applications in such a way that state can be persisted and thus it's easier for them to survive restarts and allow easier retry mechanisms.

That's why I sometimes despair when I see a whole bunch of fuss made about using Workflow Foundation for such tasks such as driving navigation between three pages in a web application: Great way to make something that was already very easy into a significantly more complex solution that brings dubious benefits (in my humble opinion) to the table. There are far more interesting uses for something like WF but hey, that's just me

One last comment Harry made that got my attention was right at the end, when he says "Eventually, I would love to see something that has the programming semantics of SSB and the interoperability of WCF". It sounds to me (though I might be misinterpreting him) that what he wants sure sounds like an interoperable, open queued messaging platform.

Personally, I'm very interested in such a platform as I believe it would be a great asset to implement reliable and interoperable message driven systems. That's the reason I'm watching very closely the development of the Advanced Message Queuing Protocol (AMQP) especification. And, there's no reason why AMQP could not be a great transport protocol for reliable Web Services!

[1] I do want to be very clear that I'm not bashing the WCF team here. Technically speaking, they're within what the WS-RM spec requires of them (and I fault the spec directly for that), and they probably have the best intentions in the world about trying to make it easier for us developers to use these technologies. They also had to make some very tough decisions about what they would support in V1.

Architecture. Web Services, WCF, WF, Messaging

A while ago there was a big fuss about correctly writing and using exceptions. Here's one of my pet peeves that I forgot to mention at that time was this: Make your exception classes serializable!

Honestly, I'm sick and tired of having to debug weird SerializationExceptions because an instance of a poorly written exception class crossed an AppDomain boundary. I could understand it if it was an exception class that really was impossible to serialize, but the reality is this is caused mainly by lazyness on the developer's part.

Tip #1: Exceptions are not serializable by default. If you create a custom exception class, it is your responsability to ensure it is serializable.

Tip #2: Marking an exception class with [Serializable] is not enough. System.Exception implements ISerializable, so it forces you to do so as well.

Here's what you should keep in mind when writing an exception class:

1. Mark the exception type with the [Serializable] attribute.
2. Add an empty protected serialization constructor that simply delegates to the base class:

protected MyException(SerializationInfo info, StreamingContext ctxt) : base(info, ctxt)

For most exception classes, this will be enough, since most don't actually add new properties and simply rely on the message.

However, if your custom exception class has custom properties, then it MUST override ISerializable.GetObjectData() (and don't forget to call the base one), and you MUST unpack those properties in your serialization constructor.

It's not just about remoting!

What many people seem to forget is that serialization happens in a lot of places without them knowing it. The underlying remoting and serialization mechanisms in .NET (and I don't mean remote object invokation) are the basis for all cross AppDomain calls, even when they are inside the same process boundary.

More and more we see applications and environments where multiple AppDomains are the norm, rather than the exception: VSTO, MMC 3.0, BizTalk, ASP.NET and averything in between (including anything with COM interop in it). And this will grow in the future because using multiple AppDomains is the preffered way of hosting plugins safely and even applications will support extensibility in this fashion in the future.

In each of those cases there's the possibility that your exception might cross an AppDomain boundary, so please make sure it does so gracefully. If you're a library writer (open source or commercial), be aware that the chance that a user might use your library in one of those contexts is always there, so make sure it is a good citizen.

.NET, C#, Serialization, Architecture

Here are some articles I found interesting that I've read recently:

• Workflow in Application Integration: Kevin Francis talks a bit about application integration architecture in the enterprise on the Architecture Journal 9. Despite the title, it doesn't really talk much about Workflow, but rather about the whole architectural perspective on integration.
• Commonwealth Bank of Australia CommSee Solution: An interesting look at a case study, in this case the CommSee application at the CBA. Part 1 focuses on the overall backend architecture for the application; Part 2 focuses on the front end: a Smart client Windows Forms application and Part 3 focuses on the infrastructure and development/test process.
  A few other architecture case studies can be found here.
• Building Interoperable Insurance Systems with .NET 3.0 Technologies: An interesting case study on building an insurance system based on insurance standards (ACORD), WCF and Web Services, Windows Workflow Foundation, a smart client and BizTalk Server 2006. It also appears to use "SQL Server 2006" . Some really interesting bits here...

Architecture, WCF, BizTalk Server, WF