Windows Communication Foundation (WCF) Service Governance and a Virtual Intermediary
As a.NET development group transitions toward an SOA or Service-Oriented Architecture, service governance, versioning, and health monitoring will become future issues that must be analyzed in detail. Service governance refers to the processes used to oversee and control the transition to and implementation of a service-oriented architecture in accordance with recognized practices, principles, and policies. SOA governance should optimize service quality, consistency, predictability and performance. SOA governance should ensure that WCF programmers follow prescribed policies and correct system problems or standards infractions as they occur.
SOA governance consists of three major components: a registry or catalog of services, best-practice policies, and testing or health monitoring procedures:
An SOA registry is an evolving catalog of information about the available services in the SOA service tier. The catalog should allow consumers and service producers to understand how to interface and use services encapsulated in the service layer.
SOA policies consist of a set of standards to ensure that service implementations remain consistent and do not interfere with each other. These constraints also ensure that standards, best-practice principles, and Microsoft best-practice guidelines are followed. Only the a Governance Committee can authorize policy exceptions.
SOA testing is a comprehensive schedule of health and performance monitoring procedures intended to ensure that each service in the entire collection of SOA services is efficient, secure, up-to-date, and, in some cases, meet State or Federal regulations (e.g., HIPAA laws). The SOA policies above should provide a checklist and schedule for testing and health monitoring.
One of the main promises of the service-oriented architectural style is its ability to efficiently deal with changes. SOA is based on a decomposition of enterprise IT assets and separation of stable IT artifacts (services) from changeable artifacts (business requirements), and an orchestration of services consumed by IT solutions (applications). As a result, some functional requirement changes can often be satisfied either by changes to existing processes or by creation of new processes based on an orchestration of existing services. This approach allows for much better support for required changes through assembling a solution based on the reusable enterprise services. Some business services become resources, shareable by multiple applications, enabling parallel autonomous development of services by multiple disparate teams, each with its own delivery and maintenance schedule.
Because shared services will be consumed by multiple applications, a change in a service can have a significant impact on many existing solutions and may, consequently, require changes in each of them. This refactoring is not only expensive, but it often requires time-consuming coordination between development teams and QA testing. This coordination ensures that none of the multiple service consumers are affected, but it also goes against one of the fundamental SOA tenets: services should be autonomous. Services should be deployed, modified, and maintained independently of each other and the applications that consume them.
Because of the problems introduced when services must be updated or changed in some way, an SOA implementation must adopt a strategy for versioning. service versioning will involve one of two procedures: add an operation contract (new interface method) to a service without changing any of the other service methods, or add a completely new service. Fortunately, most .NET development teamshad not implemented a number of services before Microsoft developed the Windows Communications Foundation framework that encapsulates a variety of service standards and transports under one umbrella platform. The problems associated with versioning and updating services have not been an issue to date.
Since most .NET development teams now are poised to implement several services, however, these problems must be anticipated and planned for in advance. One solution to service versioning and updating is virtualization. At its most basic form, virtualization is an abstraction layer or intermediary that decouples a resource consumer from the provider of those resources. Virtualized resources allow a single service implementation layer to support multiple consumers and providers to achieve a higher of separation of concerns for these service resources.
Service virtualization realizes many of the professed SOA advantages by abstracting the true service address, binding, and contract (termed service ABCs) necessary to invoke business functionality at a given endpoint. An abstraction layer is also a potential provider for additional behaviors, policies, and security not in the service itself, but in the service intermediary. Service governance then has a single point of enforcement – the virtual service layer or intermediary.
Address virtualization abstracts the Uniform Resource Identifier (URI ) where the endpoint actually sits from the service consumers. This abstraction allows messages to be delivered per database metadata to different endpoints of service providers that might include an old and a new version of a zip code service, for example. Coupled with content-based routing behavior, address virtualization could be used to manage a service level agreement (SLA) discussed later. Content-based routing behavior provides the appropriate service provider consistent with the consuming application’s requests for a service.
Binding virtualization abstracts the actual communication protocols supported by the endpoint (e.g., http or tcp) to more easily allow providers to adopt newer WCF service versions while still supporting existing service consumers. For example, a service may be dynamically comprised of operations from disparate services to form a single service that may serve as a better representation of a task or entity service. Such seemingly straightforward capabilities are extremely powerful and organizations may be willing to undergo major infrastructure upgrades to better support a virtualized service inventory.
Service Level Agreements (SLAs) address situations where compliance to one or more defined service level benchmarks needs to be verified. Metrics like internal processing time or latency, messages per minute, and rejected transaction totals are examples of defined service levels which could be measured by a virtualized intermediary. These measurements are then typically compared by enforcement policies to the level desired, and the resulting SLA violations drive some form of action by a development group. This action can be simply gathering and reporting results, identifying and forwarding SLA violations to a data center for action, or changing service behavior based on current SLA performance compared to a guaranteed level.
Another virtualization function that an intermediary could provide is integration with the Windows Server Operating System Universal Description, Discovery, and Integration (UDDI) services. This standards-based Windows Server service enables organizations to run their own UDDI directory for intranet use. Enterprise UDDI Services could help groups organize and catalog Web services and other programmatic resources by applying categorization schemes such as geography, Quality of Service (QoS), or organization of UDDI services. A development team could establish a structured and standardized way to describe services if the intermediary could enforce UDDI policies.
Microsoft released an open source intermediary called the Managed Services Engine (MSE) as one approach to facilitating Enterprise SOA through service virtualization. Microsoft built MSE upon the Windows Communication Foundation (WCF) and the Microsoft Windows Server 2003 and 2008 platforms. The Microsoft Services developed MSE to provide support for versioning, abstraction, management, routing, and runtime policy enforcement.
The MSE Technical Guide from a CodePlex download describes how this intermediary implements policies and other functionality:
“With the MSE, policy enforcement can be decoupled from service development. So the policies can be created and applied independent of how the services are being developed. The MSE supports an extensible policy model built upon WCF-behaviors with XAML based syntax. It allows stakeholders to create new polices using existing based on standard WCF behaviors or custom behaviors. These polices which are based upon standard and custom WCF behaviors can provide enforcement not just limited to security but can cover a wide range of topics such as metadata publishing, performance tracking, and business activity monitoring etc. These polices can be enforced at the EndPoint level to affect all the service operations available at that EndPoint or at the service operation level where only that particular service operation is affected.
The Managed Services architecture defines these entities through virtual services represented by a contract and consisting of many components:
End Points – the location a service request is received
Messages – the data exchanged between the service and the consumer
Policies – the internal and external rules for how a service can and will be run
Context – the relevant information peripheral to the actual service
Implementation – the logic or data that is involved in providing the functionality of the service
The Managed Services architecture was developed to support this approach to SOA through a three-layer architecture that derives its capabilities and responsibilities through a metadata repository called the Service Catalog. Since each layer has its own area of responsibility, there is no redundancy in work effort as is typically the case with a multiple intermediary model. This reduces the number of message inspections and evaluations required to process a request, allowing the architecture to maintain optimal performance while providing all the benefits of a dynamic, content-driven solution.
The first layer of this architecture is the messenger. This layer is responsible for hosting endpoints, receiving service requests, executing all message-level tasks, and supporting message-level policy. The messenger has no awareness of specific services as its task is to simply validate that a message is acceptable and could be making a valid service request. For example, if an endpoint in the DMZ requires an X-509 certificate, the messenger will verify that, but will not verify that it is invoking a valid service.
The next layer of the Managed Services architecture is the broker, which is responsible for the service control functions, including operation rationalization, request validation, service context, and operation policy enforcement. The broker is not concerned with any message level policies, but rather focuses on whether the message is valid for a specific operation listed in the service catalog. This includes the ability to determine the appropriate version, either implicitly or explicitly. Once the broker is done processing, the message is processed as a valid service request.
The final layer of this architecture is the dispatcher, which is responsible for the execution of the service logic. This architecture allows an operation to have multiple implementations and each could have its own dispatcher implementation. The dispatcher is based on a factory model, where each type of communication pattern could have its own implementation. The execution specifics are again defined by metadata in the Service Catalog which the broker provides to the appropriate host. This is where execution details, such as timeouts, re-tries, and asynchronous routines would be handled.
The Service Catalog is one of the MSE’s core components. It is responsible for storing the service metadata. In its physical implementation, it consists of three (3) modules:
SQL Server 2005 database named “MSE6DB” which stores services metadata
A WCF-based service façade which provide access to this metadata
A windows service named “Microsoft.MSE.Repository.Service” which hosts the above WCF-based service façade
Access to this metadata is only possible through this WCF-based service facade. Please note that this repository only stores metadata related to services
such as: message schema, service location, applicable policies etc. There is only one “Service Catalog” per MSE implementation, but that implementation can support multiple instances of the service façade. It is also possible for different departments or groups within an organization to host their own MSE implementations.”
A summary block diagram of the above description from the MSE Technical Guide provides an overview of the intermediary’s layered functionality:
Figure 1: The MSE Virtual Intermediary routing service consumers to providers
In the figure above, an older application (Service Consumer 1) requests zip code information from an older http zip code service (Old Http Zip Code Service 1) through the Virtual Service Interface and the Messenger endpoint. The virtual service Broker determines the request validity and context from Catalog information contained in the Service Catalog Database. The appropriate Virtual Service Dispatcher (Dispatcher 1) uses the Virtual Service Logic and metadata from the Broker to route the request to the correct legacy service (Old Http Service 1 Provider). In a similar manner, Service Consumer 2 could make a service request, and the Virtual Service Intermediary would route the request to New Tcp Service 1 Provider.
The Service Catalog Database stores service metadata currently in a SQL Server 2005 database named “MSE6DB.” This metadata repository stores information in a cache for the services like message schema, service location, applicable policies, and other data. By caching the information during runtime startup, the MSE virtual intermediary operates much faster routing the service consumer to the appropriate service provider.
MSE has a UI called the Management Console that serves as an administrative UI for the Managed Services Engine implemented underneath with WCF technologies (See the UI figure below).
Figure 2: The MSE Management Console administrative UI
The console allows:
importing multiple operations of a service;
specifying a different name for any operations imported;
indicating the binding type of a service;
associating service operations with a new or an existing endpoint;
importing schemas;
shaping request/response messages;
defining a channel to define the model for connecting to the service implementation;
providing XSL transformations needed to transform the request and response messages;
creating XAML service policies;
configuring events with the Publisher/Subscriber pattern that implements changes to the Service Catalog immediately without restarting MSE
The UI has wizards for the above operations or allows the user to set up the virtual service implementations manually.
MSE also provides a testing and health monitoring application shown below:
Figure 3: The MSE Universal Service Tester Client UI
The MSE Service Tester installs on Windows XP SP2, Vista, 2003 R2 Server, and 2008 Server and facilitates status testing of WCF services. The MSE Service Tester allows the user to test many different types of services with multiple protocols and transports.