Microsoft BizTalk Server Performance Optimization Guide

Microsoft published  this guide to provide in depth information for optimizing the performance of a BizTalk Server solution. Full end-to-end performance testing is frequently overlooked during enterprise application deployment. Knowing that Microsoft has built a scalable messaging infrastructure, many organizations that use BizTalk Server spend little or no time conducting performance testing of their own applications. BizTalk Server applications consist of many parts, which may include custom-built components as well as those provided by Microsoft. It is impossible for Microsoft to performance test every possible combination of these components. Therefore, fully and properly conducting a performance test of your application is a critical step of any deployment. The purpose of this guide is to consolidate and provide prescriptive guidance on the best practices and techniques that should be followed to optimize BizTalk Server performance.
Check this guide and follow the right steps to optimize your solution:

Microsoft talks BizTalk 2009 and beyond

Microsoft is preparing a major release of BizTalk Server that will take advantage of functionality added to the company’s application platform stack since BizTalk Server 2003 R2 shipped.BizTalk Server 2009 will become generally available in the first half of 2009, said Burley Kawasaki, director of product management in the Connected Systems Division at Microsoft, and a Community Technology Preview will be release before the end of this year.Application platform support will be extended to the .NET Framework 3.5 SP1, SQL Server 2008, Visual Studio 2008 SP1 and Windows Server 2008. BizTalk Server will take advantage of failover clustering features from Windows Server and will support Hyper-V, Microsoft’s hypervisor.Integration with Visual Studio Team Foundation Server adds ALM functions such as automated builds, bug tracking, source controls and multi-party development. The underlying BizTalk project system for Visual Studio will also be improved, the company says.The new features require Visual Studio 2008, said Kawasaki. “A lot of integration[s] we are doing builds on top of 2008 wave,” he added. What’s more, BizTalk will have out-of-the-box Business Activity Monitoring capabilities with SQL Server 2008 Analysis Services, as well as supporting Unified Dimensional Model cubes and real-time aggregations that are used in Microsoft’s PerformancePoint Server 2007 business intelligence product.“I believe the planned improvements that will become available in [BizTalk Server 2009] signal Microsoft's continuing intent to make this product a leading solution in the integration space, while ensuring that it's development is synchronized with other major initiatives inside of Microsoft. That's a good strategy,” said Ken Vollmer, a principal analyst for Forrester Research.Outside of the Microsoft stack and beyond the firewall, the 2009 release is slated to have a UDDI 3.0 registry that will provide for extended discovery services, registry affiliation, a subscription API and support for digital certificates for authentication.For business-to-business integrations, Microsoft is broadening its support for AS2 and EDI data sharing protocols. New capabilities will include support for multiple message attachments, configurable auto message resend and file name preservation.BizTalk Server 2009 updates all business rules and message schemas for compliance with the SWIFTReady Financial EAI Gold certification, increasing functionality as a result.The company also vows to offer broader guidance for applying Enterprise Service Bus usage patterns and other tasks.Inside the firewall, BizTalk will include implementations of more recent versions of CICS, CICS HTPP Transport, DB2, DB2/400, DB2 Universal Database, IMS and WebSphere MQ, according to Kawasaki.In addition, the company is developing a Line of Business Adapter for Oracle E-Business Suites and SQL Server, as well as improving its existing set of adapters, said Kawasaki. Adapters are available for MySAP ERP, Oracle Database and Siebel CRM business applications.The company is also updating BizTalk’s RFID standards support to adhere to industry standards, including LLRP, TDS, TDT and WS Discovery. Another standard, EPCIS, will not be fully implemented.Customers should expect a follow-up release within two years, Kawasaki noted. Microsoft will continue to update BizTalk to remain aligned with its application platform, he said. In the release beyond BizTalk 2009, Microsoft intends to collaborate with its research division to develop productivity enhancements like complex mapping, said Kawasaki. Other features will includes complex trading partner management, low-latency messaging enhancements and additional ESB guidance.Another goal is to extend the reach of BizTalk’s enterprise asset tracking to portable devices and laptops by leveraging its existing RFID platform.Customers will also be able to utilize its “Oslo” wave of modeling tools to create composite applications. Services exposed through Windows Communication Foundation by previous versions of BizTalk may also be used with Oslo modeling technologies, said Kawasaki.

BizTalk Server 2006 or WF? Choosing the Right Workflow Tool for Your Project

I have seen the questions about the differences between BizTalk and WF. I think that you should have a good knowledge in both of technologies in order to make an accurate comparison, thus I had searched within the internet to find an official comparison, I had found it at http://msdn.microsoft.com/en-us/library/cc303238.aspx .

Presenting @ DotNetWork.org: Introduction to EAI , B2B and BPM



Date
Saturday, August 30th 2008,12:00 - 16:00
Location
Canadian International College, @ "El-Tagamo3 El-5ames"

The Biztalk Adapter Pack Poster

The BizTalk Server Team Blog announce the availability of the new BizTalk Adapter Pack Poster.
This poster covers interoperability with Line-of-Business Applications using the BizTalk Adapter Pack and the Windows Communication Foundation (WCF) LOB Adapter SDK.
This poster depicts the functionality, components, architecture, and usage/hosting scenarios of BizTalk Adapter Pack 2.0 and of the WCF LOB Adapter SDK. When printed in full scale, this poster size is 38”x 26”.


Download it from HERE

Write your first Singleton orchestration

As you know In BizTalk Orchestrations, a new instance of an Orchestration is created every time a message arrives at the Receive Port. So what will be the number of orchestrations if we developed an orchestration responsible to login the user ? WOW. for each user want to login, there will be an instance of orchestration .

So How can I solve this?

Do you know that there is a singleton pattern which is a design pattern that is used to restrict instantiation of a class to one object?

Depending in this term we will build a singleton orchestration which will always be only one instance of the Orchestration

HOW !!!!!!!!





Could you please explain more?

The first message that arrives at the Receive port creates a new instance of the Orchestration (initialize the correlation). All other messages would use the same Orchestration for processing (following the correlation), all the messages which arrive at the orchestration must have common property. In our example it will the receive port name

The recipe is very simple, do the following and you will get a singleton orchestration :




1)Create a correlation type with the property BTS.ReceivePortName.

2)Create a correlation set with the correlation type created in the previous step

3)In the first Receive Shape (Activate = true), set the property "Initializing Correlation Sets" as the correlation set ,and in the Second receive , set the property "Following Correlation Sets" as the correlation type .

4)Do you remember while(true) loop , we will create a similar loop similar to this loop in our orchestration

5)The listen and delay shapes is just for Massage , in order not to let the orchestration being within the memory all the time .

You can download the whole project from here

BizTalk Advanced Orchestration Concepts and Best Practices Slides and Demos

http://www.biztalkgurus.com/media/p/15575.aspx

Oracle Adapter : There is an error in XML document (1, 40)

I had developed an application which requests for data from an Oracle Database according to the search criteria , everything works fine as I check the Orchestration debugger the search result return from the database successfully , but the error occurred in the web service that generated by the BizTalk.

Detailed Error:
A response message sent to adapter "SOAP" on receive port "WebPort_OracleAdapter_Proxy/OracleAdapter_OracleOrchesteration_rcvRequest" with URI "/OracleAdapter_Proxy/OracleAdapter_OracleOrchesteration_rcvRequest.asmx" is suspended. Error details: There is an error in XML document (1, 40). MessageId: {F231205A-68A4-45D7-9D3A-33CAF2D5889F} InstanceID: {B77F31E2-D2D8-4D12-9768-A5B97938E20B} For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp Operating System: windows 2003 Microsoft BizTalk Server 2006
Oracle Client version : 10.2.0

In order to fix this kind of error , check the following :


This problem is coming up because the URI that is generated by the adapter is rejected by the SOAP adapter. like if the URI for the targetNamespace is something below, it is not valid URI for SOAP adapter:"http://schemas.microsoft.com/[OracleDb:/MSADAPTER/SCOTT/Tables/MYTEST][OracleDb://MSADAPTER/SCOTT/Tables/MYTEST]" In this case the work around is change the URI within the Orchestration so that it is a valid URI for the webservice. Special characters "[" must be replaced with '%' followed by their ISO hex representation. In addition, I am not sure about the "://" string appearing two times in the same string. Here is a valid URI format as per the XML rfchttp://www.rfc-editor.org/rfc/rfc3986.txt. Or else if we do not want to change the schema to be sent to the Oracle adapter there is another workaround where in we need to use Transform in the orchestration.

1. For this create a new schema similar to the one generated by Oracle and change it to a different target name space without any special characters, something like this "http://schemas.microsoft.com/OracleDb/MSADAPTER/SCOTT/Tables/MYTEST"

2. Then create a map whose source schema points to the one given by Oracle and destination schema corresponds to the one you created in step 1. Moreover, correspond the elements of source to destination.

3. Add a Transform (making use of the above Map) in the orchestration before sending it to the SOAP.


SQL Server Reporting Services - Custom subtotals in a matrix

Last day I had faced a strange issue related to the subtotal of matrix as following:




By the way, I am using the subtotal of the column group and I am using a static row for calculating the male and the female for each row group, the weird thing that the subtotal did not pay attention to the static row. As you know, you cannot change much on the behavior of your subtotals in your matrix. When you create a subtotal it calculates a subtotal and that’s about it....!!!!!!

So the only way to force the subtotal column to calculate the value of our static total is using the custom expression.

so in the data field do the following : in our example :

1) Right click the static total cell
2) Click the expression
3) Put the following expression:

=Iif(InScope("matrix1_ColumnGroup1"),
Iif(InScope("matrix1_RowGroup1"),
"In Cell",
"In Subtotal of RowGroup1"),
Iif(InScope("matrix1_RowGroup1"),
"In Subtotal of ColumnGroup1",
"In Subtotal of entire matrix"))


Replace "In Cell", "In Subtotal of RowGroup1", "In Subtotal of ColumnGroup1" and/or "In Subtotal of entire matrix" with the expressions or fields that you want.or let it as it and see the result and then you can decide what you can do ..!!!!!

in our example :

=Iif(InScope("matrix1_ColumnGroup1"),
Iif(InScope("matrix1_RowGroup1"),
Fields!male.Value + Fields!Female.Value, sum(Fields!male.Value)+sum(Fields!female.Value)),
Iif(InScope("matrix1_RowGroup1"),
sum(Fields!male.Value)+sum(Fields!female.Value),
"In Subtotal of entire matrix"))


finally Thank you Jorg Klein .who gives me the solution of Custom expressions for subtotals in a matrix

BizUnit Designer had been Launched

BizUnit Designer is a GUI that allows rapid creation of BizUnit test cases which can be used for unit testing or system testing distributed applications. The easy to use drag and drop user interface has minimal learning curve which will get you up and running with test cases in a matter of minutes.

For more information about this product : http://www.codeplex.com/bud
For downloading this product : http://www.codeplex.com/bud/Release/ProjectReleases.aspx?ReleaseId=11936

BizTalk Server 2006 Documenter

Creates compiled help files for a given BTS 2006 installation. This tool can be run on an ad-hoc basis using the UI or from the command line as a post build/deploy task to create a compiled help file describing a BTS 2006 installation. It will compile: BTS Host configuration, Send/Receive port configuration, Orchestration diagrams, Schema and Map content, Pipeline process flow, Adapter configuration, Rule engine vocabularies and policies, More… and publish them as compiled help files. Optionally you can embed custom HTML content and custom descriptions for all BTS artifacts to produce a more customized look and feel to the CHM output

For more information about this tool : http://www.codeplex.com/BizTalkDocumenter
You can downoad this tool from the following link :
http://www.codeplex.com/BizTalkDocumenter/Release/ProjectReleases.aspx?ReleaseId=5426

BizTalk Server Pipeline Component Wizard

When you create a new pipeline component, there are a number of options available to you: you could write it completely from scratch, or you could simply use Martijn Hoogendoorn’s Pipeline Component Wizard, which can save you heaps of time.The Pipeline Component Wizard can be downloaded from the following link:
http://www.codeplex.com/btsplcw/Release/ProjectReleases.aspx?ReleaseId=2248

BizTalk Server 2006 Orchestration Profiler

This tool can be run to gain a consolidated view of orchestration tracking data for a specified period of time to help developers get an idea of how their orchestrations are being processed and how much coverage they are getting in their testing. In addition to simple coverage information the data presented helps to identify latency and code path exceptions by highlighting long running and error prone orchestration shapes which is key to effective performance testing......

Have a look here for a complete description.
You can download this amazing tool from here

8 Tips And Tricks For Better BizTalk Programming

The purpose of this article is to share those useful tips with you , you can find it here : http://msdn.microsoft.com/msdnmag/issues/07/05/BizTalk/default.aspx

BizTalk Server 2006 Best Practices Analyzer

There is a helpful tool which examines the BizTalk Server 2006 deployment and generates a list of issues pertaining to best practices standards for BizTalk Server deployments this tool called "BizTalk Server 2006 Best Practices Analyzer" you can download from here: http://www.microsoft.com/downloads/details.aspx?FamilyID=dda047e3-408e-48ba-83f9-f397226cd6d4&displaylang=en

Spotlight on BizTalk 2006 Custom Functoid

A very important topic I will talk about which is how to develop a custom functoid. Especially Developing a Custom Referenced Functoid.

A Custom Referenced Functoid!!! You mean there are other types of custom functoid.
Yes, there are three types. Let us define them

A Custom Referenced Functoid: Custom referenced functoids do not copy implementation code inline into the map. Instead, a reference to the assembly, class, and method is placed in the extension object file associated with the generated style sheet and called at run time.


A Custom Inline Functoid: Custom inline functoids provide functionality by copying implementation code directly into a map and not by referencing an assembly, class, and method name like a custom referenced functoid.

A Custom Cumulative Functoid: Use a custom cumulative functoid to perform accumulation operations for values that occur multiple times within an instance message.
Now after you knowing what the types of Custom Functoids are, let us develop our custom referenced functoid.


Note #1: all custom functoids must derive from the BaseFunctoid class. You must first override the constructor and make a set of calls that tell BizTalk Mapper about your custom functoid. Then you need to write the functoid logic
Fine, our custom functoid name Percentage, which will calculate the percentage where it takes two input parameters.

1) Fine, create a new project its type is class library and name it CustomFunctoid


2) Add a reference of Microsoft.BizTalk.BaseFunctoids you can find it here :
X:\Program Files\Microsoft BizTalk Server 2006\Developer Tools\Microsoft.BizTalk.BaseFunctoids.dll (where X the drive that holds the installation of BizTalk server 2006)


3) let you class implement from BaseFunctoid
public class Percentage : BaseFunctoid


4) let the constructor override the base
public Percentage(): base()


5) write the following line of code in the construction :

this.ID = 3003;
SetupResourceAssembl("CustomFunctoid.CustomFunctoidResource",
System.Reflection.Assembly.GetExecutingAssembly());
SetName("PERCENTAGE_NAME");
SetTooltip("PERCENTAGE_TOOLTIP");
SetDescription("PERCENTAGE_DESCRIPTION");
SetBitmap("PERCENTAGE_BITMAP");
this.SetMinParams(2);
this.SetMaxParams(2);
SetExternalFunctionName(GetType().Assembly.FullName, "CustomFunctoid.Percentage", "CalculatePercentage");
this.Category = FunctoidCategory.Math;
AddInputConnectionType(ConnectionType.AllExceptRecord);
AddInputConnectionType(ConnectionType.AllExceptRecord);
this.OutputConnectionType = ConnectionType.AllExceptRecord;

Where:


· ID: Assign a unique ID to the functoid


· SetupResourceAssembly: Point to the resource assembly
NOTE #2: Include a resource file with your project. If building with Visual Studio, the resource assembly must be ProjectName.ResourceName. Example: Our resource file named CustomFunctoidResource which includes the value of name, tooltip , description, and the Icon of our custom Functoid , and our project named CustomFunctoid


· SetName , SetTooltip , SetDescription, SetBitmap: set the name , tooltip , description and the icon of our custom functoid

· SetMinParams, SetMaxParams: Specify the number of parameters accepted , where SetMinParams is used to set the number of required parameters and the SetMaxParams is used to method to set the number of optional parameters

· SetExternalFunctionName: Tell BizTalk Server which methods to invoke for your functoid

· Category: Assign the functoid to one or more categories, for example Math, String, MassCopy…etc.
· AddInputConnectionType : Declare what can connect to your functoid

· OutputConnectionType : Declare what your functoid can connect to


6)Now create the function that will calculate the percentage which is :


public float CalculatePercentage(float x, float y)
{
float c = float.MinValue;
c = (x / y) * 100;
return c;
}

7) Create a Strong key , and Assign it to your class library


8) Build the Class Library and copy the DLL to : X:\Program Files\Microsoft BizTalk Server 2006\Developer Tools\Mapper Extensions (where X is the drive that holds the installation of BizTalk Server 2006)

9) Add this dll it to the GAC ( open the Visual Studio 2005 Command Prompt and type GACUTIL –i X:\CustomFunctoid.dll)


10) Add it to the ToolBox in Visual Studio 2005 (right click on the ToolBox, choose Add/Remove items, click the Functoids tab and browse to the Customfunctoid.dll file and make sure it’s checked.


you can download the whole project from here

Sql Adapter MSDTC Problem

I am getting this error:
New transaction cannot enlist in the specified transaction coordinator
When I am trying to generate a schema from a database existed in a
machine the OS of it is Windows 2003 SP1, and the O.S. of the
machine that am trying to generating the schemas from is Windows XP SP2 , so
I had tried the following solutions and nothing happen

  1. I had manipulate the communication settings on both of the BizTalk machine and the SQL machine using the instructions this article :
    i.
    http://msdn2.microsoft.com/en-us/library/ms679479.aspx
  2. In addition, I had added MSDTC into the exception list in the Windows Firewall settings. using the following steps:
    1. In Control Panel, open Windows Firewall.
    2. Click the Exceptions tab, and then click Add Program.
    3. Click Browse, and then add c:\windows\system32\msdtc.exe.
    4. In Programs and Services, select the Msdtc.exe check box, and then click OK.

  3. I had turned off RPC security , using the following steps :
    1. Start Registry Editor (Regedt32.exe).
    2. Locate the following key in the registry: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC
    3. On the Edit menu, click Add Value, and then add the following registry value:

Finally, AfterI had researched for a long time within the blogs , I found that you should Enable NetBIOS on your network adapter taking the following steps:

  1. Start => Control panel => Network Connections
  2. Open Properties for the you network connection
  3. TCP/IP Properties
  4. Advanced and select WINS Tab

  5. Ensure that the "Enable NetBIOS over TCP/IP" is selected



WCF-BasicHttp adapter Vs. SOAP adapter

In the last article , you can find it here. I had given a very simple view about WCF and the benefits that will give it to us , in this article I will talk about WCF adapter of BizTalk 2006 R2; imagine the benefits of WCF with BizTalk . :))

There is a video by Steven W. Thomas called "BizTalk 2006 R2 WCF Adapter First Look Video"; it was a very good and very simple video.
You can watch this video from here

But I would like to mention important things, What Is the WCF-BasicHttp Adapter? And Why the WCF-BasicHttp adapter is preferred than the SOAP adapter?

WCF-basicHttp means you can do cross-computer communication with legacy ASMX-based Web services and clients that conform to the WS-I Basic Profile 1.1, using either the HTTP or HTTPS transport with text encoding. However, you will not be able to take advantage of features that are supported by WS-* protocols.

What is the problem of that, I can do it using the SOAP adapter!!!!

You are totally right but check the following:

  • WCF adapters documents per second rate is general be better than SOAP adapter.
  • SOAP adapter has the disadvantage of serializing and deserializing messages between .NET types and raw XML data which in some cases could result in unwanted data modification. WCF adapters take the approach of preserving the data stream.
  • WCF adapters publish what is essentially the same schema as will be used in the BizTalk runtime. In contrast, SOAP adapter returns metadata generated by reflecting on the .NET classes generated to represent the schemas. Since .NET does not preserve all the nuances of XSD schemas, the exposed WSDL may not accurately reflect the schemas used by the BizTalk runtime.
  • It is easier to migrate from WCF-BasicHttp to WCF-WSHttp if needed.


Amazing right, enjoy… :))

So....is That the WCF..!!!!

In the last few days, I was trying to simplify the definition of WCF And what are the benefits that the WCF will give it to us?

What Is WCF?

The WCF is the shortened Windows Communication Foundation which is a new communication subsystem to enable applications, in one machine or across multiple machines connected by a network, to communicate. WCF applications can be developed in any language which can target the .NET runtime.

This new technology (WCF) simplifies development of connected applications through a new service-oriented programming model, HOW? It enables the development of more secure, reliable, transacted services that interoperate with non-Microsoft platforms and integrate with existing investments where that is the key of SOA model

Where In Service Orientation you think different and describe things via services and divide your system into smaller parts which run as services. These services can communicate with others via messages. In each service you can apply Object Orientation to accomplish goals of that service.

Object Orientation!!! We are talking about SOA!!!

By the way Service Orientation is a complement to Object Orientation. It means you will not kick Object Orientation out to use Service Orientation. Service Orientation uses Object Orientation in its core, but there are some distributed scenarios that can be viewed by Object Orientation so you use Service Orientation to describe these scenarios.

Cool, we can do it using the ASP.net web services.

That's right but checks the following benefits and then you can decide

  • Because WCF can communicate using Web services, interoperability with other platforms that also support SOAP, such as the leading J2EE-based application servers, is straightforward.
  • You can also configure and extend WCF to communicate with Web services using messages not based on SOAP, for example, simple XML formats like RSS.
  • Performance is of paramount concern for most businesses. WCF is developed with the goal of being one of the fastest distributed application platform developed by Microsoft.
  • To allow optimal performance when both parties in a communication are built on WCF, the wire encoding used in this case is an optimized binary version of an XML Information Set. Messages still conform to the data structure of a SOAP message, but their encoding uses a binary representation of that data structure rather than the standard angle-brackets-and-text format of the XML 1.0 text encoding. Using this option makes sense for communicating with the call center client application, because it is also built on WCF, and performance is an important concern.
  • Managing object lifetimes, defining distributed transactions, and other aspects of Enterprise Services are now provided by WCF. They are available to any WCF-based application, which means that the rental car reservation application can use them with any of the other applications it communicates with.
  • Because it supports a large set of the WS-* specifications, WCF helps provide reliability, security, and transactions when communicating with any platform that also supports these specifications.
  • The WCF option for queued messaging, built on Message Queuing, allows applications to use persistent queuing without using another set of application programming interfaces.


Finally we can say that WCF is flexible. For example, while WCF uses SOAP as an underlying structure, it is not bound to using SOAP for wire communication. In fact, WCF can be configured to process "plain" XML data that is not wrapped in a SOAP envelope. WCF can also be extended to support specific XML formats, such as ATOM (a popular RSS standard), and even non-XML formats, such as JavaScript Object Notation (JSON). This flexibility ensures that code written today will be valid in the future, even if protocols change or are replaced. Therefore, WCF was designed for the present and the future.