After writing my last post, I realized that an even better way to do this is to actually formalize the binding as a custom binding:

public class OneWayBinding : CustomBinding
{
   public OneWayBinding(Binding baseBinding) 
      : base(baseBinding)
   {
   }

   public override BindingElementCollection CreateBindingElements()
   {
      BindingElementCollection bec = base.CreateBindingElements();
      bec.Insert(0, new OneWayBindingElement());
      return bec;
   }
}

Then you can simply say:

CustomBinding binding = new OneWayBinding(new NetTcpBinding());

Windows Communication Foundation

Udi Dahan ranted a bit about the WCF transport channel model and it's extensive use of generics. More to the point, Udi was running into some trouble trying to simply send a message, one way, over an arbitrary transport channel that could either be one way or not. The problem he runs into is dealing in a generic fashion with the different Channel Shapes that WCF exposes.

I commented on his blog that maybe it would be possible for him to use a custom binding and using the OneWayBindingElement binding element to "change" the shape of the underlying, two-way channel into a single-way channel he could use by just coding against the IOutputChannel interface. Udi responsed:

"I’m trying to write transports for NServiceBus which will make use of the WCF bindings. So, since BasicHttpBinding only supports IRequestChannel and not IOutputChannel I have to deal with this. It’s just nuts that there isn’t some more useful shared interface between them." 

From his description, I gather that Udi is (with good intentions of keeping things simple) equating channels with standard bindings, but this isn't quite so. The BasicHttpBinding is just a binding; not a channel itself. However, this standard binding will certainly force a matched two-way channel shape (IRequestChannel) based on the underlying transport channel used (HttpTransportChannel/HttpsTransportChannel).

There's nothing inherently wrong about creating a new binding that is based on one of the standard channels but has extra binding elements layered on top. Here's a sample:

EndpointAddress epa =
   new EndpointAddress("http://localhost:8254/myservice/myserv.svc");

WSHttpBinding wsHttp = new WSHttpBinding(SecurityMode.None, false);

CustomBinding binding = CreateOneWayBinding(wsHttp);
// could also be
// CreateOneWayBinding(new BasicHttpBinding());

IChannelFactory<IOutputChannel> factory =
   binding.BuildChannelFactory<IOutputChannel>();
factory.Open();
IOutputChannel channel = factory.CreateChannel(epa);

Message myMessage = Message.CreateMessage(
   binding.MessageVersion, "Process", "some sample contents"
   );
channel.Open();
channel.Send(myMessage);
channel.Close();
factory.Close();

The CreateOneWayBinding() method used in the sample above is fairly simple:

static CustomBinding CreateOneWayBinding(Binding baseBinding)
{
   List<BindingElement> elems = new List<BindingElement>();
   elems.Add(new OneWayBindingElement());
   elems.AddRange(baseBinding.CreateBindingElements());
   return new CustomBinding(elems);
}

It basically creates a new CustomBinding based on the binding elements of the original binding (in this case one of the standard ones) and then layers the OneWayBindingElement, channel shape changing element on top.

Note: For some reason, this would not work if message security was enabled with the WSHttpBinding, but I didn't spent too much time looking at what the issue was, but otherwise works fine with the WSHttpBinding, BasicHttpBinding and NetTcpBinding, which is enough to make it useful on its own.

Windows Communication Foundation

Hammilton Verissimo (Hammett) was showing off his desktop this morning and asked about what other people's desktop look like. Here's mine:

I usually keep around opened Firefox, Vim, Outlook (minimized) and either Foobar 2000 or iTunes (not running right now), and the Virtual Machine I'm working on at the time (usually my trusty one with VS2005, .NET 3.0 and BizTalk 2006).

And since I'm a card-carrying member of the infamous Single-Monitor Club, there's nothing else to see .

Desktop

Jesus Rodriguez started an interesting discussion here about the differences between WCF Behaviors and BizTalk Server Pipelines, focused on what the right use cases for each one are when developing solutions on BizTalk Server 2006 R2 and the WCF adapters included in it. I think this is a really interesting discussion, and one I've given a bit of thought over the past few weeks. As Jesus correctly points out, this discussion is also more important because of the changes that might ripple through the product as WCF and WF permeate the underlying BizTalk architecture as they get integrated into the core product.

Instead of explicitly talking about Behaviors Vs. Pipelines; I'd like to approach the topic from a slightly different point of view: the messaging stacks.

In BizTalk 2006 R2, integration between WCF and BizTalk is done by talking advantage of one of the key extensibility points in BizTalk: The adapter framework. In other words, both WCF send and receive facilities are exposed to BizTalk (and from BizTalk) through a set of adapters that hook the WCF model to the BizTalk Messaging Engine (and ultimately the BizTalk Message Box).

This is obviously a simplification of sorts, but the main idea is that, for this release, the BizTalk architecture itself didn't suffer significant changes in order to enable WCF connectivity. At the same time, WCF didn't need itself any changes either; though certainly a number of extensions needed to be added in order to create fully functional BizTalk our of the core WCF binding/channels. In many ways, this is a testament of the power and flexibility of the extensibility models for both platforms. On the other hand, it means that both messaging stacks are almost fully present, with all the differences and similarities they have.

What will future versions of WCF/BizTalk look like? I honestly don't know, but one can speculate a bit. An interesting thing that might give us some clues about what is to come is to examine some interesting differences between the two models. While there are many similarities, there are also some key aspects where WCF approaches things differently from BizTalk. Here are a few:

Messages

A lot of people mistakenly think that BizTalk Server is all about XML messages and that anything flowing through BizTalk must be XML. While it is true that XML messages are first class citizen in BizTalk and that many aspects are modeled around XML-like concepts, this isn't really true.

In particular, at the messaging level, when you're working with Adapters and Pipelines, messages are simply streams of bytes. What those bytes contain is really irrelevant to BizTalk; all it cares is that it can read and manipulate those streams. In fact; it's pretty easy at this layer to deal with purely obscure or binary data; though obviously some components will make assumptions about what the data should look like (example: the Xml Disassembler expects to be able to interpret the message stream as XML text).

In WCF, on the other hand, the message concept is more closely tied to XML; however, not to the textual XML representation, but rather to the XML InfoSet. This means that it is still perfectly possible to deal with other kinds of data as long as there's a component in the WCF stack that can make it look like an XML Infoset. This can be done either by actually translating from another representation into XML (say a database structure to XML) or by "faking it" (such as a binary stream with fake open/close XML tags).

In practical terms it means that, for the most part, you can accomplish the same thing in both models; though it does mean some things get done at different layers in the stack. For example, in BizTalk, interpreting message content is usually done fairly up in the stack (i.e. the pipeline, unless you have an application adapter with specific needs), meaning things like message format are only looked into until fairly late in the game. In WCF, however, these tasks are usually done very early own, usually at the request of the Transport Channel.

This does not mean that the transport channel author has to necessarily need to implement things like message parsing, but it may mean that it is his/her responsibility to ensure it gets done. For example, a transport channel will call the configured MessageEncoder to encode/decode messages into/from the network.

Message Structure

Another interesting aspects is message structure. In BizTalk Server, messages are not really a single entity. Instead, each message is composed of one or more "message parts", each one containing it's own data stream independent of the rest, with one of them being marked as the "body" of the message. Also, each message carries around a property bag [1] associated to it (the Message Context), which can be used to carry "out-of-band" data.  and that is a critical part of BizTalk's Pub/Sub mechanism.

WCF messages aren't quite structured the same way. In fact, WCF messages don't quite have the concept of "multi-part messages" as clear as the one in BizTalk Server (though support for protocols like MTOM certainly make it possible) and so only really have a single, body data stream (or really, Xml feed) associated with them. However, unlike BizTalk, they also have two different ways to carry out of band data: Message Headers (obvious, if you consider that the WCF model is closely tied to SOAP), and Message Properties.

Assembling and Disassembling

Another interesting difference between BizTalk and WCF, and that Jesus already pointed to, is the fact that, in WCF, interchanges are really one-to-one throughout the WCF stack. That is, one message comes in, one message comes out.

This is rather different from BizTalk, in which the assembling and disassembling stages of send and receive pipelines respectively were created explicitly with the idea that messages might be joined or split during processing. This is why the stages are referred to using the terms assemble/disassemble and not simply as "parsing" or "translating".

On the receive side, this enables some really powerful mechanics for processing large interchanges, by breaking (debatching) messages as they come in into smaller parts that can be processed individually. Similarly, on the send side, multiple messages can be joined into a single, larger message. Unfortunately, the latter functionality is never used by the messaging engine (though it is accessible when calling pipelines directly from orchestrations).

So this is one point where there's a significant difference between the two messaging models. Certainly, nothing would prevent a WCF transport channel from implementing it's own disassembling/assembling functionality, though it might be a complex job, and likely not very reusable. It might be possible to implement it at higher levels of the channel stack as well, but I really haven't looked too closely into this, as, in general, this isn't as useful a feature in the context WCF services are normally used (but it is a very important one in the context BizTalk is used).

Channel Shapes

BizTalk Server only really has two different Message Exchange Patterns: Either you support exchanging messages in a single direction (one-way) or in both directions (two-way).

WCF, however, supports 3 different message exchange patterns (you could say the third one can also be simulated in BizTalk). The key differentiating here, however, is that WCF creates, on top of these three basic MEPs, a large number of different Channel Shapes, which complement the basic interactions by separating stateless and state-aware variants. Thus, the appearance of Session-aware channels.

It's obvious that WCF's model is here quite a bit more powerful than what BizTalk offers right out of the box, though it can be argued as well that this makes the model a lot more complex (and more confusing) for channel authors. BizTalk essentially assumes that all interactions are stateless (i.e. the concept of session doesn't exist); which is partially mitigated by the powerful orchestration + correlation sets combo.

I think it will be extremely interesting to see how the concept of sessions is brought into the main BizTalk architecture in a future version as WCF gets integrated more deeply with the BizTalk messaging engine. It certainly adds a few new challenges to the decoupled, highly asynchronous interaction model between BizTalk components.

Conclusion

I don't really know what the future will bring for these two platforms, but certainly the combination of WCF + WF + BizTalk is going to bring significant changes to the platform. I have a few gut-feelings about what it may look like, but they don't even qualify as educated guesses . Still, it's going to be pretty interesting to see how it evolves.

[1] There's also a property bag associated with each message part; but it is used less often and isn't as important as the message context.

BizTalk, Windows Communication Foundation

Yossi Dahan commented here on being surprised by the implications of the elementFormDefault option in XSD schemas, and particularly in relation to BizTalk.

I don't find the behavior surprising myself anymore, as I already was familiar with the implications of elementFormDefault. In reality, this is something that I don't particularly like about XSD, as I find that leaving it to "unqualified" can lead, as Yossi found out, to some weird looking schemas (at least for me). Mind you, regardless of this option, qualified-ness of an element can also be enabled/disabled for each element individually.

There are, however, 2 things that make this option more obscure than it should be:

1. The default value defined in XSD for elementFormDefault is "unqualified".
2. When you install BizTalk, now Visual Studio will have 2 different templates for XSD schemas. The original one in Visual Studio has elementFormDefault="qualified", while the one installed by BizTalk doesn't specify elementFormDefault (thus leaving it in the default value of "unqualified"). Which one you get depends on the context in which you use the Add New Item menu in VS.

In general, I recommend always watching out for this issue and always explicitly setting elementFormDefault to the appropriate value. In my mind, that appropriate value should always be "qualified" for new schemas .

Also, just for the sake of completeness, it is worth saying that, under the XSD specification, all root elements defined in a schema are always qualified (i.e. should be associated to the XML targetNamespace of the schema). The elementFormDefault only controls whether child elements are qualified or unqualified. This is why the elementFormDefault option doesn't really "drop namespaces" from the XML document instances; it merely restricts it to the root elements.

XSD, BIzTalk

I recently ran into a problem with Castle's [ValidateRegEx(...)] validation attribute and Monorail, where if you used a regular expression that contained '\' characters to escape symbols, they would be incorrectly rendered in the corresponding client-side Javascript expression.

I posted about the issue in the Castle forums, alongside one possible way to fix it, in case someone runs into the same problem.

Monorail

I've been building a small application the last few days using Castle, including both Monorail and ActiveRecord, which I had not used before. It's been a very pleasant experience overall, as I dig a bit deeper into some of the features in Castle I had not used before. At the same time, it's been a bit frustrating as some of the documentation isn't quite up to date, or, more commonly, doesn't quite list all the necessary steps.

One of the aspects that gave me a bit of trouble getting to run was using Castle.Components.Validator to add validation to the application. Hammett has a nice introduction to the validation framework on this screencast, but even following this and the documentation I couldn't quite get it to work (I think part of the issue was that I use brail, while he uses nvelocity).

In this example, I'm using ActiveRecord for persistence so that will affect some of the code, so be aware of that. Here's exactly what worked for me.

Step 1: Reference Castle.Components.Validator.dll on your project where your model classes are defined. Where necessary, reference the Castle.Components.Validator namespace. Make sure you pick a recent copy of the Castle libraries from the build server .

Step 2: Add validation attributes to the properties of your model classes. Make sure you explicitly specify a proper error message on your attributes, because the default messages might not be as informative. Here's an example:

Step 3: Inherit your monorail controller from SmartDispatcherController so that the validation facilities are available. In my case, I'll use the ARSmartDispatcherController class so that I can use the nice ActiveRecord integration facilities as well:

Step 4: Use the [DataBiind] or [ARDataBind] attributes on your handler methods, and make sure you enable the Validate property:

Step 5: Inside your handler code, check if the bound object contains error before persisting and if so redirect as appropriate if not. You can use the HasValidationError() method to check if the object triggered validation errors, and then call GetValidationSummary() to get the list of error messages.

I'm currently wrapping GetErrorSummary() in my base controller class so that I can return a single string with all validation errors to avoid having to loop around in the view (I'm lazy):

So far you should already have server-side validation working. Now let's add client side validation:

Step 6: Include the necessary for client side validation to work; you'll want to do this in your layout view:

The last one, in particular, is pretty important; without it you'll later get bizarre JS errors like "Validation is not defined". This actually took me a while to get because it's the only one that Hammett doesn't explicitly include in his demo.

Step 7: Enable client-side validation in your form. For this, you need to add a couple of custom attributes to your form tag: immediate and useLabels and set them to true. Using FormHelper, here's what it would look like:

For the curious, this actually generates the following <form/> tag:

The closing tag will also contain the validation script using whatever validation provider your selected (prototypes, by default), unless you disable validation using FormHelper.DisableValidation().

Step 8: Declare your field labels and input controls. Using FormHelper, again, this is easy:

Step 9: Put the error messages from the server-side validation somewhere on your form:

And now, client side validation should be working Notice that the client-side validation messages seem to be tagged with the "validation-advice" CSS class, so that makes them easy to style.

Monorail, Castle, ActiveRecord

I'm please to announce the release of Version 1.1 of my PipelineTesting library for BizTalk Server 2006. PipelineTesting can be used with your favorite unit testing framework to create automated, repeatable tests for the following BizTalk artifacts:

• Custom Pipeline Components
• Custom Pipelines
• XML and Flat File Schemas

Version 1.1 adds the following features:

• Improved XML documentation (it actually builds the XML documents without warnings!)
• Bug fixes
• A new, streamlined and more intuitive API, which also includes wrappers for easy configuration of standard XML and Flat File assembler and disassembler components. The original API is still available, for backwards compatibility.

As usual, the library can be downloaded here. The package includes all source code for the library itself and its unit tests, and a pre-compiled binary and XML doc comments file.

Using the library

To use the library, you need to reference the following assemblies:

• Winterdom.BizTalk.PipelineTesting.dll
• Microsoft.BizTalk.Pipeline.dll
• Microsoft.BizTalk.Pipeline.Components.dll

You will need to include the following namespaces:

Introduction to the API

I won't cover all the API, however, let me introduce the basic mechanics of how the library can be used.

The general way in which the library is used is:

1. Create a send or receive pipeline. Pipelines are represented in the API as instances of either the ReceivePipelineWrapper or SendPipelineWrapper classes.
   PipelineTesting provides ways to create an instance of a pre-compiled BizTalk pipeline (including the built-in pipelines in BizTalk), or creating an empty pipeline.
2. Configure the pipeline, by adding any components you require (if needed) in the proper stages of the pipeline, as well as providing it with any document specifications (schemas) that will be required by assembler or disassembler components in the pipelines.
   This allows the library to run completely independent of your BizTalk configuration, so you don't even need to deploy your stuff to the server before taking advantage of the library; providing a very quick feedback cycle!
3. Create input messages to provide to the library. You can create messages with any shape or characteristics that BizTalk allows, such as multi-part messages, adding context properties and so on.
4. Execute the pipeline: Just pass in the input messages and validate the resulting messages!

Example: A simple XML Send Pipeline

The following example shows how to create a standard XMLTransmit pipeline, provide a schema to use and execute the pipeline with an input message:

Example: Batching messages in a Send Pipeline

Here's a more interesting test for a send pipeline. This one will create an empty Send pipeline, add a standard XML Assembler component to it, configured with an Envelope and a Document schema and then feed multiple input messages to be assembled into a single one, encoded with S/MIME:

Notice that, in this example, we didn't explicitly add the schemas to the pipeline itself; that's because when they are configured on the XML assembler, the pipeline picks it up automatically.

Example: Parse a flat file

In this example, I'll create a new receive pipeline and add a Flat File disassembler to it to parse a flat file from a BizTalk schema:

Conclusion

This brief introduction should give you an idea of the capabilities of the PipelineTesting library. As you can see, the new API is fairly intuitive and simple to use and everyone should be able to use it with minimal effort.

If you have any questions, comments or run into a bug, do let me know; I'll do my best to answer and fix any issues!

BizTalk, Testing

Virtual PC 2007 is a great improvement over VPC 2004, since it fixes a number of bugs that caused problems, and overall, it has been working very nicely for me since RTM. However, two issues drive me nuts:

1. Whenever the Clipboard sharing between the host OS and the guest OS breaks down (meaning you cannot copy on one and paste on the other). When this happens, it usually means that certain clipboard operations will fail even locally within the guest OS (for example, trying to copy or paste an image within the guest OS will fail).
2. Whenever the Ctrl key gets "stuck". Whenever this happens, if you switch to the Host OS and then give focus back to the virtual machine, it's as if the Ctrl key was pressed, causing all sorts of problems. Hitting a few keys and the Ctrl key a few times usually will un-stick it, but it's a pain in the neck. I've been on the verge of breaking my keyboard a few times today already because of this issue!

Virtual PC

Yesterday morning I was going through some client/server code I had written, mostly some custom serialization/deserialization code. The code itself was working fine; I had decent set of unit tests for it. Instead, I was focusing on doing some integration testing as well as taking some measurements of how performance was when large object sets where serialized/deserialized.

The code itself is fairly simple, and although it uses XML as the serialization format, it still is fairly efficient (XmlReader/Writer based code); however I needed to see how other factors would impact it. On the server side, I had built a TextWriter implementation on top of my networking stack, which made kept the code real simple.

In my first test, I tried having it work with a small set of objects. The result? Doing real work 110ms; serializing the results and sending them over the wire? 1100ms. Not very good.

Looking at it a bit more, I realized that the problem was that the underlying networking stack didn't do any buffering at all; if I send 3 bytes, it would send a TCP packet with those 3 bytes right away; since the XmlTextWriter class does a lot of WriteChar() calls on the underlying TextWriter, performance was abysmal.

So I went ahead and implemented a simple buffering scheme inside my TextWriter object, and that worked fine. Second results? Doing real work: 85ms, serializing the results: 120ms.

I then started looking at how it would perform with large object sets (as in like a few thousands of them). My first result? Doing real work: 6640ms, serializing the results? 130ms. That can't be right! Tried it a few more times; with always the same results.

Indeed it wasn't right. Turned out that while I was sending small result sets; all was well because the serialized representation of the objects was smaller than the small buffer size I was testing initially with (about 4KB), but as soon as that went over that number, I started sending larger packages down the network fairly fast and the underlying network library would throw an exception after the second or third package was sent. So indeed, failing was turning out to be faster than succeeding!

I didn't notice the exception that was occurring at first because the serialization code was wrapped with a try/catch block that, if it trapped an exception, would serialize the error information and send it along the wire as well; which was pretty useless as that was precisely what was failing .

.NET, Coding

I've been using a bit today some of the tools in Visual Studio 2005 Team Edition for Database Professionals today, and I gotta say, some of the features of the product are quite useful. Now I know why Jeff was always so enthusiastic about it . The schema comparison tool is quite useful, for example.

That said, I admit I was a bit baffled at first about how to get started with it; a few things aren't quite as intuitive as one could hope for.

Visual Studio

...And I wonder why Visual Studio takes so much time to startup :)

Visual Studio

Ayende picked up on Jeremy Miller's latest post and commented on the whole idea of "trusting the developers in your team". I think it's safe to say that Ayende is right on the money: If you can't trust them, and you don't want to coach them and help them become better developers, what's the purpose? Why have you hired them in the first place?

However, it's not just a trust issue. I think I've said it in the past; but this issue it's also one of business and understanding of the world of software development. I've voiced my concerns to this line of thinking to people before around here, and one counterclaim I've seen brought up time and time and again by project managers and even owners of IT shops here is that they need the untrained, less skilled, less trusted developer.

It's not that they don't trust them; it's that they hire them to be untrusted and kept with low-skills because some tasks don't require "special" skills. Invariably, I hear stuff like "why would you hire a top-notch developer to create the web UI of the application?" Well.... in my experience, that's usually one of the places that a) is the most visible part of the application, b) is one that can take a lot of time during development, c) is easy to get wrong and others. Besides, this is definitely along the lines of the Mort issues discussed a while ago.

Software Development

So much for trying to solve this by attempting to change my visual studio configuration. As soon as I tried to start the Visual Studio 2005 installation to add the missing components I was greeted by the infamous error:

d:\vs\vs_setup.msi could not be opened.

Uninstalling VS2005 to reinstall wouldn't work, either; it does use the same setup after all. So now I have the pleasure of having to start setting up my development virtual machine from scratch courtesy of a screwed VS installation. <sarcasm>Thanks goodness this is just a virtual machine and not my main one</sarcasm>.

Visual Studio

From the "stupid installer pisses me off" department:

Today I decided to try to install the Visual Studio 2005 Team Suite Database Professional Add-On (got tired just typing that) on my Visual Studio Team Suite installation. I installed it without issues, and then proceeded to install SR1.

Imagine my surprise when it asked if I wanted to upgrade the Database Professional TRIAL edition! It sounded fishy because I had not installed any trials, so went to the download page again to check if it was a trial and it didn't say so. However, far down the page it says:

This add-on provides Visual Studio 2005 Team Suite users with the additional functionality provided by Visual Studio 2005 Team Edition for Database Professionals. Visual Studio 2005 Team Suite is required to use this add-on. If you do not currently own Visual Studio 2005 Team Suite Edition but would like to evaluate Visual Studio 2005 Team Edition for Database Professionals in a trial mode, you may do so by first installing the Visual Studio 2005 Team Suite Trial Edition and then installing this add-on which will function until the Team Suite trial period expires.

Do I have Team Suite installed? Hell yeah! So why does it says it installed in trial mode, then? The answer? Team Suite is, apparently, a figment of my imagination or something. After doing a lot of memory, here's what I think happened:

When I recreated my development virtual machine, I proceeded to install Visual Studio 2005 Team Suite as always. However, during installation, I decided to uncheck the testing tools, based on the fact that I never use them (prefer NUnit myself) and that they would only slow me down.

Well, apparently, just because I decided to not install one feature, that was enough for the Database Professional installer to decide that I do not have "Team Suite" installed. I'd argue that I installed from the Team Suite media with a Team Suite license, but apparently that's not enough for the stupid Installer. Now I have to go and uninstall it, install more crap on Visual Studio, reinstall SP1 and reinstall this. It better be worth it... 

Visual Studio

The new version of my PipelineTesting library is coming along nicely. I'm mostly done with the new API, though there are still a few changes that I'm pondering about. However, right now this is already implemented:

• Creating standard pipelines
• Creating pipelines from compiled types
• Building pipelines from scratch
• Adding document specifications and other things to pipeline contexts for execution
• Creating XML and Flat File Assembler and Disassembler components (only a couple of properties are not exposed at this point)

I may still decide to rename a few methods and I'm considering providing an alternative to the current MessageHelper class and its horrible interface. Meanwhile, here's the current state of the library :)

BizTalk, Testing