Data Format Transformation
How can services interact
with programs that communicate with different data formats?
|
Problem
A service may be
incompatible with resources it needs to access due to data format disparity.
Furthermore, a service consumer that communicates using a data format
different from a target service will be incompatible and therefore unable to
invoke the service.
|
|
Solution
Intermediary data format
transformation logic needs to be introduced in order to dynamically translate
one data format into another.
|
|
Application
Data format transformation
can be incorporated by adding internal service logic, service agents, or a
dedicated transformation service.
|
Impacts
The use of data format transformation
logic inevitably adds development effort, design complexity, and performance
overhead.
|
|
Principles
Standardized Service Contracts,
Service Loose Coupling
|
Architecture
Inventory,
Composition,
Service
|
Table 6.7 – Profile summary for
the Data Format Transformation pattern.
Problem
Services can be fundamentally
incompatible with resources or programs that only support disparate data
formats. For example, the service may have been standardized to send and
receive XML-formatted data, but is required to also retrieve data from a legacy
environment that only supports the CSV format (Figure 6.x).
Figure 6.33 – Three different
services accustomed to processing XML formatted data need to access a legacy
application API that only accepts CSV formatted data.
Solution
A layer of data format transformation logic is introduced.
This logic is specifically designed to convert one or more data formats into
one or more different data formats (Figure 6.x).
Figure 6.34 – A Format Conversion
utility service is added to the architecture. This service abstracts the Legacy
Application API and provides XML-to-CSV and CSV-to-XML functions. Note that in
the depicted architecture, the Format Conversion service exists as component
being reused by multiple components that are part of Web services (as per the
Dual Protocols pattern).
Application
Data format transformation logic can exist within a
services internal logic, within a service agent, or (as shown earlier in
Figure 6.x) as a separate service altogether. Middleware with brokerage
features will often provide out-of-the-box services and agents that perform a
variety of data format conversions.
Furthermore, because the conversion of one data format to
another will naturally transform the source data model, the application of this
design pattern can be seen as also applying the Data Model Transformation
pattern. However, as explained in the corresponding pattern profile, the Data
Model Transformation pattern is most commonly used separately for the
conversion of one formal data model to another when both source and destination
formats are the same.
Impacts
As with any of the three design patterns associated with the
parent Broker compound pattern, the introduction of a transformation layer for
data format conversion will introduce the following impacts:
• An
increase in solution development effort when the required transformation logic
needs to be custom-programmed. However, custom development effort may not be
necessary when using a middleware platform already equipped with the required
conversion functionality.
• An
increase in design complexity, as this pattern will introduce a new layer in an
already distributed environment.
• An
increase in performance requirements because the format conversion will need to
be executed every time interaction between disparate format sources is
required. This is especially a concern with the transformation of data formats
because of the tendency to carry out bulk or batch format conversions.
These impacts are often considered natural requirements that
come with introducing a service layer into an enterprise with legacy
applications and resources.
Relationships
As shown in Figure 6.x, Data Format Transformation is
commonly utilized to help integrate custom service logic with legacy systems,
which makes it commonly required when applying the Legacy Wrapper pattern. The
actual transformation logic may be the responsibility of a faade component
that is tasked with converting proprietary to standardized formats, as per the
Service Faade pattern.
When protocols are standardized using the Canonical Protocol
pattern, data formats are usually common as well. This is because most
communication protocols are associated with common data formats. For example
the CSV format is often used with FTP and SCP protocols, and XML is primarily
associated with HTTP. Therefore, when Data Format Transformation is required,
it is often in conjunction with the Protocol Bridging pattern.
The service established by the Inventory Endpoint pattern
may very well be required to carry out the conversion of data formats and the
overall logic introduced by the Data Format Transformation pattern can be
encapsulated by event-driven agents produced by the Service Agent pattern.
Figure 6.35 – The Data Format
Transformation pattern is commonly employed to deal with traditional
integration issues resulting from legacy encapsulation, which is why this
pattern relates to Legacy Wrapper and other broker-related patterns..
Also worth noting is that this pattern represents one of the
three core patterns that represent the Broker compound pattern. Because Broker
is a also a core part of the Enterprise Service Bus compound pattern, ESB
platforms are fully expected to support the conversion of different data
formats.
Figure 6.36 – The
Data Format Transformation pattern is one of three transformation-related
design patterns that comprise the Broker and Enterprise Service Bus compound
patterns.
Case Study Example
Case study content is not available on this Web site.
