Deep Dive : Middleware


Middleware sits "in the middle" between application software that may be working on different operating systems. It is similar to the middle layer of three-tier single system architecture, except that it is stretched across multiple systems or applications. Examples include EAI software, telecommunications software, transaction monitors, and messaging-and-queuing software.

The distinction between operating system and middleware functionality is, to some extent, arbitrary. While core kernel functionality can only be provided by the operating system itself, some functionality previously provided by separately sold middleware is now integrated in operating systems. A typical example is the TCP/IP stack for telecommunications, nowadays included in virtually every operating system.

In simulation technology, middleware is generally used in the context of the high level architecture (HLA) that applies to many distributed simulations. It is a layer of software that lies between the application code and the run-time infrastructure. Middleware generally consists of a library of functions, and enables a number of applications – simulations or federates in HLA terminology – to page these functions from the common library rather than re-create them for each application.

In the computer industry, middleware is a general term for any programming that serves to "glue together" or mediate between two separate and often already existing programs. A common application of middleware is to allow programs written for access to a particular database to access other databases.

Typically, middleware programs provide messaging services so that different applications can communicate. The systematic tying together of disparate applications, often through the use of middleware, is known as enterprise application integration (EAI).

Okay, I think that the previous section define the term middleware from academic view, maybe we can be a bit more specific about what middleware is by describing what it does. First, though, we need a basic model to show where middleware fits among the other pieces of the client/server puzzle.

We'll divide our discussion into three areas:

  • Basic client/server component model
  • Basic middleware services
  • Types of middleware

Basic Client/Server Component Model

We've created a simple network application model to help explain the concept of middleware, and we'll use it throughout this guide. Instead of the OSI network model, we have basic chunks on servers and clients, including operating system, application program, and the connectivity pieces like network protocols and middleware software.



Here's our model which is defined in the context of database networking middleware:


 

Figure 1 : Basic Client/Server Component Model


The middleware is right there in the middle (called database connectivity middleware in our diagram). Notice that middleware components are instantiated on both client and server platforms. Since some definitions of middleware include code that isolates operating systems from hardware platform differences (like the hardware abstraction layer or HAL in Windows NT), let's keep our discussion focused on middleware for distributed systems. In some cases, there will be more middleware services provided by an intermediate device like a database gateway. The next diagram provides a more specific example using Oracle and PowerBuilder on common platforms:




Figure 2 : Client/Server Components-An Example
What does the middleware do? Simply put, it provides a set of services to applications like a client/server database as depicted in our diagram. What are those services? Read on.



Basic Middleware Services

The basic set of middleware services are offered by most products today. We'll refer to Microsoft's SQL Server RDBMS (Relational Data Base Management System or Relational) and its DB-Library data access middleware as an example. Services offered include:

Client/Server Connectivity -Middleware provides the mechanism by which network applications communicate across the network. This includes in the case of database networking for example the service of putting packages of query results data into network transport packets. Microsoft SQL Server, for example, uses Sybase's Tabular Data Stream (TDS) protocol to handle formatting of data for transport across the network. This session layer interaction may also have its own timers and even error control to handle automatic retransmission of lost packets. One feature common is the ability for the client to interrupt the current operation on the server to cancel a large query response download. (We'll discuss the various mechanisms provided in depth when we discuss the typology of Middleware later.)

Platform Transparency -Client and server don't have to have intimate knowledge of each other in order for work to get done. Differences between platform specific encodings like big-endian and little-endian or EBCDIC and ASCII are typically hidden by middleware. Middleware often runs on a variety of platforms, letting the organization utilize all its existing desktop and server hardware as applications require. Still, some middleware products find it hard to look beyond Windows clients and UNIX or Windows NT servers. Make sure the middleware you're buying handles all the platforms you really have deployed. Microsoft SQL Server DB-Library middleware provides access only to Windows NT servers (since that's the only supported SQL Server host platform), but does so from DOS, Windows (3.1, 95, NT), Mac and UNIX clients (though not all this client software ships with SQL Server itself).



Network Transparency and Isolation -Middleware often makes networking choices transparent to application programmers. Actually, though, every middleware product we've ever heard of runs on TCP/IP, with all the other protocols coming in a distant second. If you want to be more prepared to run tomorrow's middleware, get on the TCP/IP bandwagon. Then again, don't let application programmers become too divorced from networking decisions; it can lead to poorly designed applications.



SQL Server supports multiple protocols between clients and servers, though some are specific to a given platform. From Macs, the choices are TCP/IP and AppleTalk. From PCs, there are more choices: TCP/IP, NetWare IPX/SPX, and NetBIOS/NetBEUI (Named Pipes). In some cases, the client and server don't even have to run the same network protocol between them. An intermediate device which might best be called a database relay can get the two end nodes talking to each other. But, they'll need the same middleware product everywhere to make this happen.



Application and Tool Support (APIs) -Before middleware can be used, it must present its own API to client applications that might use it. For shrink-wrapped tools like a database query tool, the API support can be critical. While ODBC has provided some level of transparency across multiple proprietary database APIs, many RDBMS vendors still encourage using their own proprietary APIs. Be sure you know what APIs your middleware offers as well as what APIs your tools can use. Hopefully there's a match! SQL Server offers both ODBC standard and DB-Library proprietary APIs on the client. For more generic middleware, the API on the server must be available as well; for RDBMS middleware, the server side is typically hard-coded to support an RDBMS.



Language Support -Middleware often provides transparency across different SQL database dialects. Even when coding in embedded SQL in a 3GL, the middleware mig ht allow the use of a single SQL dialect across a variety of RDBMS back ends. Outside of the database specific middleware products, generic middleware products often allow different programming languages to be used to create the distinct pieces of an application (pieces that reside on different machines). Since SQL Server's DB-Library only supports SQL Server RDBMSs, the SQL dialect supported is Transact SQL, a superset of ANSI 89 SQL created by Sybase and Microsoft.



RDBMS Support -When we focus on database networking middleware (also called data access middleware), middleware may also provide a level of transparency across different data storage formats. It will make different RDBMSs look like the same RDBMS. ODBC is one way of hiding RDBMS differences, but middleware products often provide multiple RDBMS support from both proprietary and standard APIs. SQL Server's DB-Library middleware does support ODBC interfaces, but still natively gets users to SQL Server RDBMS only.



Much database networking middleware is closely tied to the RDBMS of that same vendor. To get to even more data sources, database gateway products are needed. Third party products like TechGnosis SequeLink or IBI EDA/SQL offer more variety in RDBMSs. Even Microsoft has recently allied with IBI to have their multi-RDBMS connectivity solution connected to the DB-Library network so that DB-Library clients can get to RDBMSs other than SQL Server.



For non-data access products, RDBMS functionality can still be provided using extensions to make accessing that kind of data directly easy over the middleware solution deployed. NetWeave is one messaging vendor that also has database access software options.



Clearly we've run through quite a number of useful kitchen appliances already, way beyond just the sink. Let's examine the more advanced services middleware might be providing.



Types of middleware

For many programmers, middleware is like a black box. Middleware helps get the job done more quickly and keeps the programmer from having to worry about even more things. Most application programmers prefer to keep that black box closed. They'll learn how to make calls to the box and get results back, but they won't try to understand what goes on in the box.

Middleware is sometimes called plumbing because it connects two sides of an application and passes data between them. Common middleware categories include:

  1. Message Oriented Middleware. This is a large category and includes asynchronous store and forward application messaging capabilities as well as integration brokers that perform message transformation and routing or even business process coordination.
  2. Object Middleware. This category consists largely of Object Request Brokers that were mentioned on one of the earlier definitions.
  3. RPC Middleware. This type of middleware provides for calling procedures on remote systems, hence the name Remote Procedure Call. Unlike message oriented middleware, RPC middleware represents synchronous interactions between systems and is commonly used within an application.
  4. Database Middleware. Database middleware allows direct access to data structures and provides interaction directly with databases. There are database gateways and a variety of connectivity options. Extract, Transform, and Load (ETL) packages are included in this category.
  5. Transaction Middleware. This category as used in the Middleware Resource Center includes traditional transaction processing monitors (TPM) and web application servers. One could make the case for splitting the category.
  6. Portals. We include enterprise portal servers as middleware largely because they facilitate what we refer to as "front end" integration. They allow interaction between the user's desktop and back end systems and services.

WSCF.blue v1.0

WSCF.blue is a Visual Studio .NET 2008 add-in that provides the following features:

· A WSDL Wizard that allows the developer to step through the creation of a WSDL from one or more XSDs.

· A Data Contract Generator (similar to XSD.exe, XSDObjectGen.exe and SvcUtil.exe) that generates the .NET equivalent classes of the XSD types.

· A Service/Endpoint Stub (support for self-hosted and web-hosted) Generator and

· A Client Proxy Generator.

· A Generate Data Contract Code feature that supports the selection of multiple XSD/WSDL source files.

· A Paste XML as Schema option that generates a schema for a block of XML in the clipboard.

· Support for C# and VB.NET code generation.

· You can choose if operation methods on your service class will throw a NotImplementedException, call an implementation method in a partial class, or will be defined as abstract methods.

· Force the SOAP actions (Action and ReplyAction) applied to each operation contract follow the standard WCF format

· Errors found while processing the WSDL are reported in a WSCF.blue pane in the Visual Studio Output window.

Please verify that the receive location exists....

Please verify that the receive location exists, and that the isolated adapter runs under an account that has access to the BizTalk databases.

This error occurs because the web service is running on the application pool which security account user is not a member of the BizTalk Isolated Host Users group. in order to solve this error, you need to add the user that runs the web-service into the BizTalk Isolated Host Users group. Or change the application pool to the pool which user is already a member of the BizTalk Isolated Host Users Group. :)


COM+ was unable to talk to the Microsoft Distributed Transaction Coordinator (Exception from HRESULT: 0x8004E00F)

I've received this error while deploying a MSI application; although, I've found a solution of this error, which is:

"when you run a 32-bit program on a 64-bit Windows Server 2003-based computer"

however, I've deployed this application before. :| :|.
and Suddenly, I've check the services, and I found the Distributed Transaction Coordinator service is stopped !!!

after starting it, the error is gone. :)

Happy BizTalk to you – the Final

Welcome to machine again.

In this section we will discuss some of EAI patterns, and how these could be implemented in BizTalk.

  • Aggregator:

    Aggregator is the pattern of receiving information from multiple sources and consolidating it into a single message


  • Splitter:

    How can we process a message if it contains multiple elements, each of which may have to be processed in a different way?

    Use a Splitter to break out the composite message into a series of individual messages, each containing data related to one item

    In BizTalk, there is a concept known as Envelope splitting. This is a useful feature in BizTalk for splitting messages.

    The Splitter pattern takes a single message and splits it into multiple messages


  • Parallel Convoy:

    A parallel convoy enables multiple single messages to join together to achieve a required result. The set of related messages can arrive in any order, but BizTalk Server must receive all of them before starting the process.


  • Sequential Convoy:

    The Sequential Convoy pattern enables multiple single items to ‎join together to achieve something that an individual item cannot ‎accomplish by itself. A sequential convoy is a set of related items ‎that have a predefined order. Although the items do not have to ‎be exactly the same, BizTalk Server must receive the items in a ‎sequential order.‎


In order to understand better what business requirements require convoy messaging, consider the three business scenarios outlined in the following section.


Scenario 1: Hospital Admits a New Patient

Business Overview: A hospital admits a new patient. The hospital requires several pieces of information from the new patient including insurance information, past medical history, and contact information. Several different people collect this information including an insurance specialist, a nurse, and a receptionist. Several different systems process this information. The order in which collection and submission of this information occurs is not guaranteed. This is because some information collectors may be busy with other patients, the medical records department may be behind in their schedule, or the insurance system may be not functioning correctly. Assembling this information for the patient in an organized manner must occur throughout the time the patient spends at the hospital. This guarantees that the patient receives accurate care and correct billing.


Technical Overview: This process involves receiving several different pieces of information about a new patient in any order. All the pieces of information are required before the business process continues and admits the patient. This reduces the number of non-payments by guaranteeing the admission of only those patients with all the requirements. This also creates a race condition in which any one of the various messages can be the first to arrive and start the business process for the new patient. Because this is a new patient, no patient number is available for correlation or reference. Accomplishing correlation in this case occurs by using date of birth, social security number, or a combination of other demographics. Any one of the three messages may be required to start a new orchestration; this creates a race condition inside the message box.


Scenario 2: Standardized Test Taking and Grading

Business Overview: A student is getting ready to take a standardized test. As part of the testing process, the student registers for the test. After registered the student checks in at the testing center on the correct date and takes the test. Upon check out, checking the student's identity occurs again and the student leaves the testing center. Finally, the student's test arrives at the grading center. Only after the completions of all these steps in order, can the student's test be graded and a score recorded.

Technical Overview: Completion of this business process requires receipt of several different types of messages. In this case, there is a predefined order to follow in which to process these messages. After receipt of the four single messages, sending a single notice occurs to allow the student's test to be graded.

Convoy ‎ means multiple single items must be related together in order to achieve something that the individual item cannot accomplish by itself

Convoy processing can be additionally broken out into two separate categories as shown in the following list:

  • Sequential convoy - Sequential convoys are a set of related items that have a predefined order. The items do not have to be exactly the same. Grouping together requires that all they have to have is a purpose to group.
  • A real world example of a sequential convoy is a group of friends headed to a party traveling in separate vehicles. The overall goal is for all of them to reach to the party but in this case, only the first person has the directions. It is up to the lead vehicle to make sure they drive to the correct location and that all the other vehicles following are not lost. In order to make sure they all stay together, the leader instructs each following vehicle to put a blue flag on the vehicle's antenna. There are many other vehicles on the road going home, to work, or to another party but only the vehicles going to this party in this group have the blue flag.
  • Parallel convoy - Items that are in any order, but must all be present before something occurs, Parallel convoys are a set of related items that can be in any order. BizTalk Server 2004 must receive all the items before achieving the final goal. Shown here is this basic pattern. Another way of thinking of parallel convoys is like a door with many locks. The locks can be unlocked in any order. Only when the last key is turned will the door open. The key ring may have many different keys on it, but only one set matches this combination of locks.


  • Message Broker:

    Message Broker is the pattern of determining the destination of a message and still maintaining control over the message flow the goal of our generic message broker will be to dispatch any type of message to its destination decided at runtime. So, by avoiding type dependency and early routing binding (basically, hard-coding) we would get single very flexible orchestration which can easily handle requirement changes - one of the top goals of good application design.



    So, the question is how to implement these patterns in BizTalk?

    The answer is very simple, check this link J
    http://www.codeplex.com/PatternWizard

    And enjoy BizTalking




Happy BizTalk to you – part 3

Always design a thing by considering it in its next larger context — a chair in a room, a room in a house, a house in an environment, an environment in a city plan.

Introduction:

In the last post http://mohammad-yousri.blogspot.com/2009/06/happy-biztalk-to-you-part-2.html

I asked a very innocent question: Which integration approaches best address which of these criteria?

And I talked about the different approaches; do you remember the last approach? It was the Messaging, thus, we need to explain the different types of message channels. Are you ready? Go.


Let's check the following conversation between three messages; they met each other in the channel.

Message #1: Hello

Message #2: Hey, where are you going?

Message #1: I am going to System A, you?

Message #2: Don't know, I am waiting the admin to decide.

Message #3 3 3 3 3 3 3 3 3 3 3 3's…. :

Here We Go Again, Here We Go Again. the original is still there. the original is still there

Message #2: Wow, where are these 3's going?

Message #1: They are copy of 3's, and they are going to the subscribers.


  • Point-to-Point Connection

    A point-to-point connection ensures that only one receiver receives a particular message. For this to work, the sending system must know the location of the receiving node. The sending system often must translate the message into a format that the receiving system understands.

    When you use point-to-point connections, each system determines the address of all the other nodes that it needs to communicate with. When target addresses or protocol details change, all the systems that communicate with the target server must be updated. As the size of your integration network grows and as the frequency of change increases, the operational cost associated with this approach becomes significant.

    The strength of the Point-to-Point Connection pattern is how simple it is to implement. The weakness of the Point-to-Point Connection pattern is the duplication of transformation and routing code between systems, and the high configuration cost of endpoint address changes. To minimize these weaknesses, you can add another layer of indirection between endpoints that contains a broker. (Message #1)


  • Message Broker

    A message broker is a physical component that handles the communication between applications. Instead of communicating with each other, applications communicate only with the message broker. An application sends a message to the message broker, providing the logical name of the receivers. The message broker looks up applications registered under the logical name and then passes the message to them. Communication between applications involves only the sender, the message broker, and the designated receivers. The message broker does not send the message to any other applications. Use a central Message Broker that can receive messages from multiple destinations, determine the correct destination and route the message to the correct channel. (Message #2)

  • Publish/Subscribe

    At a high level, the Publish/Subscribe [Buschmann96] pattern helps keep cooperating systems synchronized by one-way propagation of messages because one publisher sends a message to any number of intended subscribers.

    A Publish-Subscribe Channel works like this: It has one input channel that splits into multiple output channels, one for each subscriber. When an event is published into the channel, the Publish-Subscribe Channel delivers a copy of the message to each of the output channels. (Message #3)



Happy BizTalk to you – part 2‎

Today I am going to explain the application integration options, by the way there's more than one approach for integrating applications, each approach ‎addresses some of the integration criteria better than others. The various ‎approaches can be summed up in four main integration styles: ‎

Wait !!!!!!!!! What are the integration criteria?

What makes for good application integration? If integration needs ‎were always the same, there would only be one integration style. Yet ‎like any complex technological effort, application integration involves a ‎range of considerations and consequences that should be taken into ‎account for any integration opportunity. So, find below the integration criteria:

  • Application coupling — Even integrated applications should minimize their dependencies on each other so that each can evolve without causing problems for the others. Tightly coupled applications make numerous assumptions about how the other applications work; when the applications change and break those assumptions, the integration breaks. The interface for integrating applications should be specific enough to implement useful functionality, but general enough to allow that implementation to change as needed.


  • Integration simplicity — when integrating an application into an enterprise, developers should strive to minimize changing the application and minimize the amount of integration code needed.

    Yet changes and new code will usually be necessary to provide good integration functionality, and the approaches with the least impact on the application may not provide the best integration into the enterprise.


  • Integration technology — Different integration techniques require varying amounts of specialized software and hardware. These special tools can be expensive, can lead to vendor lock-in, and increase the burden on developers to understand how to use the tools to integrate applications.


  • Data format — integrated applications must agree on the format of the data they exchange, or must have an intermediate traslator to unify applications that insist on different data formats. A related issue is data format evolution and extensibility—how the format can change over time and how that will affect the applications.


  • Data timeliness — Integration should minimize the length of time between when one application decides to share some data and other applications have that data. Data should be exchanged frequently in small chunks, rather than waiting to exchange a large set of unrelated items.

    Applications should be informed as soon as shared data is ready for consumption. Latency in data sharing has to be factored into the integration design; the longer sharing can take, the more opportunity for shared data to become stale, and the more complex integration becomes.


  • Data or functionality — integrated applications may not want to simply share data, they may wish to share functionality such that each application can invoke the functionality in the others.

    Invoking functionality remotely can be difficult to achieve, and even though it may seem the same as invoking local functionality, it works quite differently, with significant consequences for how well the integration works.


As you can see, there are several different criteria that must be considered when choosing and designing an integration approach. The question then becomes:

Which integration approaches best address which of these criteria? ‎

  • File Transfer — Have each application produce files of shared data for others to consume, and consume files that others have produced. In the File Transfer pattern, one application produces a file and transfers it so that other applications can consume it. Because files are a universal unit of storage for all enterprise operating systems, this method is often simple to implement.

    The applications must agree on file naming conventions and the directories they appear in. The writer of a file must implement a strategy to keep the filenames unique. The applications must agree on which one will delete old files, and that application will have to know when a file is old and no longer needed. The applications will need to implement a locking mechanism or follow a timing convention to ensure that one application is not trying to read the file while another is still writing it. If all of the applications do not have access to the same disk, then some application must take responsibility for transferring the file from one disk to another.


    While this approach seems simplistic, it does have some advantages. Using files affords good physical decoupling between the different processes. If the target system is not immediately available to receive the file, it can be stored until the system becomes ready. Also, files are essentially platform and language independent as long as they use a common character set.

    However, batch file transfers also present a number of challenges. Data changes in one system may not be available in the other system until the next day. This may be confusing to users, and cause data integrity problems if the (outdated) data is updated in the other system. Additionally, batch exchanges tend to extract all available information from one system and replicate it to the other system. If only few changes were made, this approach will cause large amounts of unnecessary data transmission.

  • Shared Database — Have the applications store the data they wish ‎to share in a common database.‎

    In an attempt to eliminate data synchronization issues and the ‎replication of huge amounts of data, some enterprises went to ‎create a shared database resource that would be shared by all ‎systems. Having the data in a single place would eliminate the ‎need for duplication of data. Synchronization issues and ‎contention can be addressed by the locking and protection ‎mechanisms provided by most database systems. The key ‎drawback of this solution lies in the difficulty of defining a data ‎model that fits all applications. Many applications are built or ‎purchased with proprietary data models and many attempts to ‎morph all these models into a single, universal data model turned ‎out to be a never ending task. Also, accessing a single database ‎may cause a serious performance bottleneck. In addition, this ‎solution still simply exchanges data items, rather than managing ‎business functions across systems.‎

  • Remote Procedure Invocation — Have each application expose ‎some of its procedures so that they can be invoked remotely, and ‎have applications invoke those to run behavior and exchange data.‎ Remote Procedure Call (RPC) mechanisms isolate the application ‎from the raw data exchange mechanisms through an additional ‎layer. This layer manages the marshalling of complex data types ‎into byte streams as required by the transport layer. As a result, ‎RPC mechanisms allow an application to transparently invoke a ‎function implemented in another application. The marshalling ‎mechanisms rely on stubs generated from the interface ‎specification in a language-neutral Interface Definition Language ‎‎ (IDL).‎

    While RPC-based systems made dealing with the integration much ‎simpler, these systems still share the drawbacks of unreliable, ‎synchronous communication with the raw data exchange ‎methods. RPC-based systems also imply brittle point-to-point ‎connections between senders and, receivers that become difficult ‎to maintain as the number of participants increases. Furthermore, ‎most vendor-provided RPC implementations are not interoperable ‎with other products.‎

  • Messaging — Have each application connect to a common ‎messaging system and exchange data and invoke behavior using ‎messages.‎

    Messaging systems try to overcome the drawbacks of the previous ‎solutions by providing reliable, asynchronous data transfer. An ‎application can publish data to the integration layer and can be ‎assured that the data will be delivered to the recipient(s). The ‎originating system is not required to wait for an acknowledgement ‎and can continue the primary process flow. Messaging systems ‎also incorporate addressing schemes that avoid the difficult to ‎maintain point-to-point connections that are typical in RPC ‎systems. At this time, it appears that messaging systems provide ‎the best basis for an integration solution. However, there are still ‎some challenges to be overcome. Messaging systems are still ‎comparatively new and the body of knowledge around architecting ‎these systems is still evolving. Asynchronous systems require a ‎different approach to systems architecture and design than ‎synchronous systems do. Furthermore, the asynchronous nature ‎of the interaction can make testing and debugging more difficult as ‎well.‎

Ummmm….Messaging, seems that we are talking about BizTalk.
J

Conclusion:

Today, we talk about the integration criteria which make your integration application good, according to these criteria; you have to choose the best approach that matches the criteria is.

In the next article, I will explain the different between messaging channels, so, be there.



Happy BizTalk to you – Part 1

Welcome To machine!

Today, I would like to introduce the series of Happy BizTalk to you; the idea behind these articles is to familiarize you with a set of design patterns in BizTalk orchestrations to simplify the programming process automatically. The plan is to get you some overview automatically, no additional effort required. When you're familiar with the design patterns of BizTalk, you can face a programming issue and — Bam! — A solution will come to you more quickly. Instead of banging your head against the wall in desperation, you'll say, "What I need here is the Splitter pattern Or the Aggregator pattern". Let's go

What are the Patterns?

People think in patterns. It is the way we naturally communicate ideas related ‎to complex subject areas such as music, science, medicine, chess, and ‎software design. Patterns are not new. We all use them intuitively as part of ‎the learning process without really thinking about it. And because our minds ‎naturally use patterns to perform complex tasks, you can find patterns nearly ‎everywhere.‎

For example : The Patterns in Sports

Consider what happens during a soccer game or an American football game.‎ if you look closer at patterns, you will find relationships between them. In ‎sports, for example, teams have certain plays for offense and certain plays for ‎defense; the patterns that describe two players' actions must fit into a larger ‎pattern that the team is following. In this sense, patterns can be described in ‎terms of hierarchies.‎

What are you talking about exactly?

You know that BizTalk implements the meaning of EAI (Enterprise Application Integration), but we need to define the following :

  • What is Enterprise Application?

    The Enterprise Application is software which provides business logic support functionality for a ‎organization, typically in commercial organizations, which aims to improve ‎the organization's productivity and efficiency.‎Enterprise applications are those which are accessed at the same time by ‎many users and are hosted on single servers. They mostly have multiple ‎user roles, security access and integration with other applications. They ‎help companies to maintain and access enterprise data like finance, ‎employee, sales, production, etc.‎

  • What is the Integration of Enterprise Application?

    Integration, Integration, integration….ummmm! , actually, Integration is an act of combining two or more things in such a way that ‎they work together. In this respect, integration is connected to enterprise ‎applications and information technology. Enterprise applications store a ‎huge amount of persistent data which are accessed simultaneously by ‎multiple users. ‎

    The definition of "integration" is very broad. To us it means connecting ‎computer systems, companies or people. However, in EAI there are six types of integration ‎projects:‎

    • ‎Information Portals
    • ‎Data Replication
    • ‎Shared Business Functions
    • ‎Service-Oriented Architectures
    • ‎Distributed Business Processes
    • ‎Business-to-Business Integration

Take a break!

  1. Information Portals : ‎

    For example, to verify the status of an order, a customer service representative may have to access the order management system on the mainframe plus log on to the system that manages orders placed over the Web. Information portals aggregate information from multiple sources into a single display to avoid having the user access multiple systems for information. Simple information portals divide the screen into multiple zones, each of which displays information from a different system.

  2. ‎Data Replication : ‎

    Many business systems require access to the same data. For example, a ‎customer's address may be used in the customer care system (when the ‎customer calls to change it), the accounting system (to compute sales tax), the ‎shipping system (to label the shipment) and the billing system (to send an ‎invoice). Many of these systems are going to have their own data stores to ‎store customer related information. When a customer calls to change his or ‎her address all these systems need to change their copy of the customer's ‎address. This can be accomplished by implementing an integration strategy ‎based on data replication. There are many different ways to implement data ‎replication. For example, some database vendors build replication functions ‎into the database, we can export data into files and re-import them into the ‎other system, or we can use message-oriented middleware to transport data ‎records inside messages.‎

  3. ‎Shared Business Functions: ‎

    In the same way that many business applications store redundant data, they also tend to implement redundant functionality. Multiple systems may need to check whether a social-security number is valid, whether the address matches the specified postal code or whether a particular item is in stock. It makes business sense to expose these functions as a shared business function that is implemented once and available as a service to other systems.

    A shared business function can address some of the same needs as data replication. For example, we could implement a business function called 'Get Customer Address' that could allow other systems to request the customer's address when it is needed rather than always storing a redundant copy. The decision between these two approaches is driven by a number of criteria, such as the amount of control we have over the systems (calling a shared function is usually more intrusive than loading data into the database) or the rate of change (an address may be needed frequently but change very infrequently).

  4. ‎Service-Oriented Architecture : ‎

Shared business functions are often referred to as services. A service is a well-‎defined function that is universally available and responds to requests from ‎‎"service consumers". Once an enterprise assembles a collection of useful ‎services, managing the services becomes an important function. First of all, ‎applications need some form of service directory, a centralized list of all ‎available services. Second, each service needs to describe its interface in such ‎a way that an application can "negotiate" a communications contract with the ‎service. These two functions, service discovery and negotiation, are the key ‎elements that make up a service oriented architecture.‎

Service-oriented architectures (SOAs) blur the line between integration and ‎distributed applications. A new application can be developed using existing, ‎remote services that may be provided by other applications. Therefore, calling ‎a service may be considered integration between the two applications. On the ‎other hand a service-oriented architecture usually provides tools that make ‎calling an external service almost as simple as calling a local method ‎‎(performance considerations aside). Because all services are available in a ‎consistent manner, SOAs are sometimes referred to as "service bus ‎architectures".‎

  1. ‎Business-to-Business Integration

    So far we have mainly considered the interaction between applications and business functions inside an enterprise. In many cases, business functions may be available from outside suppliers or business partners. For example, the shipping company may provide a service for customers to compute shipping cost or track shipments. Or a business may use an outside provider to compute sales tax rates. Likewise, integration frequently occurs between business partners. A customer may contact a retailer to inquire on the price and the availability of an item. In response, the retailer may ask the supplier for the status of an expected shipment that contains the out-of-stock item.

    Many of the above considerations apply equally to business-to-business integration. However, communicating across the Internet or some other network usually raises new issues related to transport protocols and security. Also, since many business partners may collaborate in an electronic "conversation" standardized data formats are critically important.

    What is of the biggest buzz words in enterprise architecture and integration? ‎ Loose Coupling

    Someone may ask; how could the Loose Coupling help us?

    Okay, let's assume we are building an on-line banking system that allows customers to ‎deposit money into their account from another bank. To perform this function, the ‎front-end Web application has to be integrated with the back-end financial system ‎that manages fund transfers. The easiest way to connect the two systems is ‎through the TCP/IP protocol. Every self-respecting operating system or ‎programming library created in the last 15 years is certain to include a TCP/IP stack. ‎TCP/IP is the ubiquitous communications protocol that transports data between ‎the millions of computers connected to the Internet and local networks. Why not ‎use the most ubiquitous of all network protocols to communicate between two ‎applications? Let's assume that the remote function that deposits money into a ‎person's account takes only the person's name and the Dollar amount as ‎arguments. The following few lines of code then suffice to call such a function over ‎TCP/IP.

    byte[] amount = BitConverter.GetBytes(258);

    But we didn't keep in mind in this example the following:

  • Platform Technology ‎ the internal representations of numbers and objects
  • Location all components have to be available at the same time
  • Data Format the list of parameters and their types must match
Because of the platform technology are not the same, the problem of Big-Endian and Little- Endian may be occurred
Have you figured the problem?
What about the location, Data format?
So, the TCP is not the right option in our example, in order to make the solution more loosely coupled we can try to remove ‎these dependencies one by one. We should use a standard data format ‎that is self-describing and platform independent, such as XML. Instead of ‎sending information directly to a specific machine we should send it to an ‎addressable "channel". A channel is a logical address that both sender and ‎receiver can agree on the same channel without being aware of each ‎other's identity. Using channels resolves the location- dependency, but still ‎requires all components to be available at the same time if the channel is ‎implemented using a connection-oriented protocol... In order to remove ‎this temporal dependency we can enhance the channel to queue up sent ‎requests until the network and the receiving system are ready. To support ‎queuing of requests inside the channel, we need wrap data into self-‎contained messages so that the channel knows how much data to buffer ‎and deliver at any one time. Lastly, the two systems still depend on a ‎common data format. We can remove this dependency by allowing for ‎data format transformations inside the channel. If the format of one ‎system changes we only have to change the transformer and not the other ‎participating systems. This is particularly useful if many applications send ‎data to the same channel.

Okay, I guess I had answered your question about the Loose Coupling. in the next article I will exaplin the Application Integration Options.
So Be there.

BizTalk 2009 and ESB 2.0 Now Available for Public Beta

Some of the key enhancements to BizTalk 2009 is support for the latest Microsoft offerings (including Windows 2008, Visual Studios 2008 SP1, .Net 3.5 SP1, SQL 2008, and TFS), support for RFID Mobile, and new LOB Adapters (including a new SQL Adapter).

A new release of the Enterprise Service Bus Guidance is available on CodePlex (http://www.codeplex.com/esb).

Wow.. seem that we are going to enojoy biztalking :D

New BizTalk Business Activity Monitoring (BAM) Guidance

BizTalk Server Team Blog post this guidance, If you are looking into implementing a BAM solution using the BizTalk Server infrastructure I strongly recommend reading this paper and testing the sample code provided with it. The online version is available on MSDN, however for the full Word version and sample code you may want to download it.

Do you want high throughput?

The combination of BizTalk Server 2006 and SQL Server 2005 running on a Windows Server 2003 platform offers the industry’s best mission-critical functionality and availability, and achieves very high levels of performance. This document presents a customer that achieved unprecedented performance, scalability, flexibility and cost-effectiveness for a business-critical application within a large financial institution.

download from here 

BizTalk Server 2009 Beta Availability and beyond

The first public beta release of BizTalk Server 2009 is now available for download at http://connect.microsoft.com.
Some of the major themes of this version are the enhanced developer productivity and the significantly improved developer team experience through the integration with Microsoft’s application life-cycle management (ALM) solution: Visual Studio Team System and Team Foundation Server.
For more details, please refer to the press release.

Check this article  by David Worthington describes the main facntinality of Biztalk 2009

Due to Muslims around Gaza

At least 632 Palestinians, including women and children, have been killed in an Israeli aerial bombardment on Hamas security installations.
Emergency services said that at least 2845 people had been wounded.
we call on imams, speakers, journalists, poets, writers, and literary authors to enrich the religious, cultural, and media pulpits with resistance literature and to send out a message of hope that the more ferocious events get, the more hope we lay on Allah who promised to exterminate the tyrants and grant victory to the believers and Who made it a duty on Him in His grandeur saying in the Qur'an: {وَكَانَ حَقّاً عَلَيْنَا نَصْرُ الْمُؤْمِنِينَ} Transliteration: Wa Kāna Ĥaqqāan `Alaynā Naşru Al-Mu'uminīna[and it was due (to them) upon Us to help the believers.] [Ar-Rum 30:47].
:: This Is a Message for Humankind ::
:: Did you know ? ::
:: The situation of the Muslims in Palestine ::
:: What can we do about the slaughter of Muslims in Palestine and around the world? ::
:: For more news ::

Microsoft BizTalk Server Performance Optimization Guide

Microsoft published  this guide to provide in depth information for optimizing the performance of a BizTalk Server solution. Full end-to-end performance testing is frequently overlooked during enterprise application deployment. Knowing that Microsoft has built a scalable messaging infrastructure, many organizations that use BizTalk Server spend little or no time conducting performance testing of their own applications. BizTalk Server applications consist of many parts, which may include custom-built components as well as those provided by Microsoft. It is impossible for Microsoft to performance test every possible combination of these components. Therefore, fully and properly conducting a performance test of your application is a critical step of any deployment. The purpose of this guide is to consolidate and provide prescriptive guidance on the best practices and techniques that should be followed to optimize BizTalk Server performance.
Check this guide and follow the right steps to optimize your solution:

Microsoft talks BizTalk 2009 and beyond

Microsoft is preparing a major release of BizTalk Server that will take advantage of functionality added to the company’s application platform stack since BizTalk Server 2003 R2 shipped.BizTalk Server 2009 will become generally available in the first half of 2009, said Burley Kawasaki, director of product management in the Connected Systems Division at Microsoft, and a Community Technology Preview will be release before the end of this year.Application platform support will be extended to the .NET Framework 3.5 SP1, SQL Server 2008, Visual Studio 2008 SP1 and Windows Server 2008. BizTalk Server will take advantage of failover clustering features from Windows Server and will support Hyper-V, Microsoft’s hypervisor.Integration with Visual Studio Team Foundation Server adds ALM functions such as automated builds, bug tracking, source controls and multi-party development. The underlying BizTalk project system for Visual Studio will also be improved, the company says.The new features require Visual Studio 2008, said Kawasaki. “A lot of integration[s] we are doing builds on top of 2008 wave,” he added. What’s more, BizTalk will have out-of-the-box Business Activity Monitoring capabilities with SQL Server 2008 Analysis Services, as well as supporting Unified Dimensional Model cubes and real-time aggregations that are used in Microsoft’s PerformancePoint Server 2007 business intelligence product.“I believe the planned improvements that will become available in [BizTalk Server 2009] signal Microsoft's continuing intent to make this product a leading solution in the integration space, while ensuring that it's development is synchronized with other major initiatives inside of Microsoft. That's a good strategy,” said Ken Vollmer, a principal analyst for Forrester Research.Outside of the Microsoft stack and beyond the firewall, the 2009 release is slated to have a UDDI 3.0 registry that will provide for extended discovery services, registry affiliation, a subscription API and support for digital certificates for authentication.For business-to-business integrations, Microsoft is broadening its support for AS2 and EDI data sharing protocols. New capabilities will include support for multiple message attachments, configurable auto message resend and file name preservation.BizTalk Server 2009 updates all business rules and message schemas for compliance with the SWIFTReady Financial EAI Gold certification, increasing functionality as a result.The company also vows to offer broader guidance for applying Enterprise Service Bus usage patterns and other tasks.Inside the firewall, BizTalk will include implementations of more recent versions of CICS, CICS HTPP Transport, DB2, DB2/400, DB2 Universal Database, IMS and WebSphere MQ, according to Kawasaki.In addition, the company is developing a Line of Business Adapter for Oracle E-Business Suites and SQL Server, as well as improving its existing set of adapters, said Kawasaki. Adapters are available for MySAP ERP, Oracle Database and Siebel CRM business applications.The company is also updating BizTalk’s RFID standards support to adhere to industry standards, including LLRP, TDS, TDT and WS Discovery. Another standard, EPCIS, will not be fully implemented.Customers should expect a follow-up release within two years, Kawasaki noted. Microsoft will continue to update BizTalk to remain aligned with its application platform, he said. In the release beyond BizTalk 2009, Microsoft intends to collaborate with its research division to develop productivity enhancements like complex mapping, said Kawasaki. Other features will includes complex trading partner management, low-latency messaging enhancements and additional ESB guidance.Another goal is to extend the reach of BizTalk’s enterprise asset tracking to portable devices and laptops by leveraging its existing RFID platform.Customers will also be able to utilize its “Oslo” wave of modeling tools to create composite applications. Services exposed through Windows Communication Foundation by previous versions of BizTalk may also be used with Oslo modeling technologies, said Kawasaki.

BizTalk Server 2006 or WF? Choosing the Right Workflow Tool for Your Project

I have seen the questions about the differences between BizTalk and WF. I think that you should have a good knowledge in both of technologies in order to make an accurate comparison, thus I had searched within the internet to find an official comparison, I had found it at http://msdn.microsoft.com/en-us/library/cc303238.aspx .

Presenting @ DotNetWork.org: Introduction to EAI , B2B and BPM



Date
Saturday, August 30th 2008,12:00 - 16:00
Location
Canadian International College, @ "El-Tagamo3 El-5ames"

The Biztalk Adapter Pack Poster

The BizTalk Server Team Blog announce the availability of the new BizTalk Adapter Pack Poster.
This poster covers interoperability with Line-of-Business Applications using the BizTalk Adapter Pack and the Windows Communication Foundation (WCF) LOB Adapter SDK.
This poster depicts the functionality, components, architecture, and usage/hosting scenarios of BizTalk Adapter Pack 2.0 and of the WCF LOB Adapter SDK. When printed in full scale, this poster size is 38”x 26”.


Download it from HERE

Write your first Singleton orchestration

As you know In BizTalk Orchestrations, a new instance of an Orchestration is created every time a message arrives at the Receive Port. So what will be the number of orchestrations if we developed an orchestration responsible to login the user ? WOW. for each user want to login, there will be an instance of orchestration .

So How can I solve this?

Do you know that there is a singleton pattern which is a design pattern that is used to restrict instantiation of a class to one object?

Depending in this term we will build a singleton orchestration which will always be only one instance of the Orchestration

HOW !!!!!!!!





Could you please explain more?

The first message that arrives at the Receive port creates a new instance of the Orchestration (initialize the correlation). All other messages would use the same Orchestration for processing (following the correlation), all the messages which arrive at the orchestration must have common property. In our example it will the receive port name

The recipe is very simple, do the following and you will get a singleton orchestration :




1)Create a correlation type with the property BTS.ReceivePortName.

2)Create a correlation set with the correlation type created in the previous step

3)In the first Receive Shape (Activate = true), set the property "Initializing Correlation Sets" as the correlation set ,and in the Second receive , set the property "Following Correlation Sets" as the correlation type .

4)Do you remember while(true) loop , we will create a similar loop similar to this loop in our orchestration

5)The listen and delay shapes is just for Massage , in order not to let the orchestration being within the memory all the time .

You can download the whole project from here

BizTalk Advanced Orchestration Concepts and Best Practices Slides and Demos

http://www.biztalkgurus.com/media/p/15575.aspx