Charles Young has written this article "Receive Pipeline Woes V2"  few months back. It's one of the very good articles out there, which explains some of the insights of BizTalk Server internals. But under the heading "Seekable Streams" it appears that the description is not correct. Recently, I came across a strange problem with custom pipeline component and Biztalk MAPs.

My custom pipeline component does very simple task something like reading the incoming message, logging it in the database and putting the message back into the stream using MemoryStream. I can't use the DATA property of IBaseMessage, because I'm using HTTP adapter which gives a Network stream that's not Seekable. Accessing "Position" property or changing the "Seek" position will throw an exception ("method not implemented"). So, I need to populate a local MemoryStream by buffering the input stream in order to do any manipulation on the complete message I received via HTTP adapter inside my pipeline component.

Charles Young in his article (under Seekable Streams heading) says "The first of the two problems is that BizTalk 2004 fails to execute inbound maps over seekable streams.   It is as simple, and as brutal, as that.   If, after a pipeline has completed its work, it returns a message whose body part contains a seekable stream, and if BizTalk matches an inbound map to the MessageType property of the message, the map will always fail with a “The root element is missing” error.   If the stream is non-seekable, and positioned at the beginning of the stream, the map will succeed (unless, of course, there is some other problem)."

I've just put together a sample here, which shows the problem is due to the way you handle stream and the way you flush the stream inside your pipeline component and its nothing to do with whether the stream is seekable or not.

The sample application (Download it here) contains the following artifacts

1.LogMessageWithStreamFlush (Pipeline Component)

2. LogMessageWithoutStreamFlush (Pipeline Component)

3. Receive.LogMessageWithStreamFlush (Biztalk Receive Pipeline)

4. Receive.LogMessageWithoutStreamFlush (Biztalk Receive Pipeline)

5. Tester.XSD, Developer.XSD and Tester2Developer Biztalk Map.

In the sample application Biztalk receive pipelines are configured with appropriate pipeline component (i.e .Receive.LogMessageWithStreamFlush BizTalk pipeline component will have LogMessageWithStreamFlush pipeline component in the Decode stage). The custom receive pipelines receives the incoming message, reads it completely into a local buffer, and then puts the message back into the stream via MemoryStream. A map will be applied on the receive port to transform the message.

Download the Sample. 

To setup the project

1. Copy the zip file to C:\BTSSamples and open the solution SeekableStreamsAndMaps.sln

2. Compile and Deploy the project from Visual Studio.

3. Copy the pipeline component dll "DDL.Samples.SeekableStreamAndMaps.PipelineComponents.dll" to "Pipeline Components" folder under Biztalk 2004/2006 installation directory (normally under c:\program files\microsoft biztalk server 2004(2006).

4. Open Biztalk Adminstrator console. Right-click on the application "SeekableStreamsAndMaps" and import the binding file "binding.xml" from the sample download.

5. Restart biztalk host instance.

Configuration:

Two receive ports (FILE Adapter) are created one with "Receive.LogMessageWithStreamFlush" pipeline and another with "Receive.LogMessageWithoutStreamFlush" pipeline component. Both the Receive ports has got the map "Tester2Developer" which transforms the incoming "Tester" message into "Developer" message.

A send port (FILE Adapter) is created with a "Filter" expression subscribed to messages received via any of the Receive ports we configured in the previous steps.

Execution:

1. Copy and paste the sample message "Tester.xml" (from FileDrop\MSG_SAMPLES) into the folder MSG_IN_WITH_STREAM_FLUSH, you'll see the expected transformed output "Developer" in the output folder "MSG_OUT"

2. Copy and paste the same sample message into the folder MSG_IN_WITHOUT_STREAM_FLUSH, then you'll see the exception as shown below in the event viewer

The Messaging Engine failed while executing the inbound map for the message coming from source URL:"C:\BTSSamples\SeekableStreamsAndMaps\FileDrop\ MSG_IN_WITHOUT_STREAM_FLUSH\*.xml" with the Message Type "http://www.digitaldeposit.net/samples#Tester". Details:"Root element is missing."

The exception is same as what's Charles Young mentioned in his article.

So, What's the difference between two pipeline components:

Below is the code from Execute method of the pipeline component LogMessageWithStreamFlush, which works fine without any issue.

string message = GetMessage(inmsg);

//Log the message to DB or Filesystem or anywhere you like
System.Diagnostics.Debug.WriteLine(message);

//Promote MessageType
string btsNamespace =  "http://schemas.microsoft.com/BizTalk/2003/system-properties"
inmsg.Context.Promote( "MessageType",  btsNamespace, "http://www.digitaldeposit.net/samples#Tester");  //Hardcoded for the demo

//Write the message back into inmsg stream
MemoryStream memStream = new MemoryStream();
StreamWriter sw = new StreamWriter(memStream);
sw.Write(message);
sw.Flush();
memStream.Flush();
memStream.Position = 0;

inmsg.BodyPart.Data = memStream;
inmsg.BodyPart.Data.Position = 0;

return inmsg;

In the second pipeline component(the one which throws the exception LogMessageWithoutStreamFlush ), I've just commented the two statements which Flushes the stream sw.Flush() and memStream.Flush(). You can see from the above code, I've used MemoryStream to put the message back into the IBaseMessage body part. MemoryStream is fully seekable, you can put the position of the Stream anywhere within the boundary since I got memory as the backend data store. So, it proves the reason why MAP is failiing with "Root element is missing" is purely because we didn't flush the stream, which resulted in empty content of the stream when the MAP starts to pull it.

NOTE: This method is quite expensive and should not be used for high volume systems. As Charles Young explained in his article you should wrap the original stream in a new stream object that processes the data on the fly during read operation, making it stateless. But in my case we were expecting one message every 1 hour once, so its not worth spending so much time on writing a proper stream handing pipeline component.