Businesses today are actively improving and streamlining their core business processes by integrating internal business and application silos across their extended ecosystems. They seek to integrate processes, people, and information seamlessly. Connectivity is the vital underpinning of such business integration and process improvement. Companies also need to ensure compliance with multiplying regulatory demands, and again, connectivity is central to compliance delivery.
Many enterprises have adopted Service Oriented Architecture (SOA), the game-changing “software as services” architecture for flexible and adaptable business applications. In fact, the rate of deployment for SOA solutions is increasing rapidly. A major strength of an SOA implementation is a messaging backbone that can reliably transport data across an integrated enterprise to get the right information to the right person at the right time for better decision making. However, many enterprises still produce “roll your own” home-grown types of integration solutions, often using unreliable methods, such as FTP, that can expose the business to hidden risks.
To avoid the inadequacies of these methods, businesses can implement an SOA messaging backbone for transporting all types of data, while using their existing hardware and software. The power of the SOA messaging background is so broad today that it serves to connect and unify file-oriented, message-oriented, service-oriented, and event-oriented applications and Web 2.0 traffic across the enterprise. Business can realize the advantages of infrastructure security, enterprise integration, efficiency, and regulatory compliance by relying on an SOA messaging background.
A look at what is driving the need for SOA connectivity
Businesses face increasing regulatory compliance and government legislation requiring them to demonstrate the accuracy and timeliness of financial reporting or the secure handling of sensitive information. Organizations must be able to show that data contained in files and documents is transferred with integrity around their organization. Examples of compliance standards include Sarbanes-Oxley (SOX), Markets in Financial Instruments Directive (MiFID), and industry-specific directives such as the Health Insurance Portability and Accountability Act (HIPAA).
Organizations need ways to keep accurate, up-to-date records of where and how files and documents are being transferred within and between businesses to demonstrate good handling of information in audits and to resolve issues that would compromise the integrity of transactions and disrupt business processes.
Other drivers affecting businesses include globalization requiring organizations to reduce the batch window, which is the amount of time needed to collect or synchronize information between locations. This would ensure that an up-to-date view of the business is always accessible and would better cope with the increasing volume of data that must be processed overnight. Additionally, supply chain integration drivers intensify the need for reliable, auditable exchange between companies to ensure partner agreements are kept and errors can be minimized.
Using an SOA messaging backbone to improve file transport
As an example of the value of the SOA messaging backbone, let’s look at one of its connectivity capabilities: managed file transfer.
How do organizations determine if their enterprise can benefit from a SOA messaging backbone that efficiently handles managed file transfer? When evaluating their infrastructure, the IT staff must ask these questions:
In the past, organizations have often chosen File Transfer Protocol (FTP) to transfer files and documents, combined with in-house software engineering. FTP is conceptually simple and intuitive; it provides a quick fix; and is often free, integrated into many operating systems. While it appears that FTP will save on development time and maintenance, the true cost may be hidden in the consequences of not being able to manage and track file transfers, and in the in-house engineering needed to develop and maintain additional software to improve it.
"Numerous factors cause companies to re-examine how they manage the movement of information from system to system, partner to partner and person to person. FTP alone is not a viable option to give you the insight, security, performance and, ultimately, the risk mitigation necessary to responsibly conduct business," state Gartner analysts Frank Kenney and James Lennard in "Magic Quadrant for Managed File Transfer," June 23, 2008.
FTP can burden your IT department and affect your organization in many ways. These are some of the disadvantages:
"Our staggering and somewhat shocking research finding is that custom-built, in-house, hard-coded integration solutions (the majority using free FTP software) are much the most widely-used approach. These often take 2 to 4 times the time and effort to build as Enterprise Integration middleware-supported integration projects, require a similar multiple of ongoing maintenance and support effort, and are insecure, fragile and vulnerable to several serious risks," states Ian Bramley, Software Strategies analyst, in the white paper, "Application Integration Challenges," November 2006.
Organizations have struggled with such issues relating to managed business data for more than 15 years. An SOA messaging backbone can help resolve the wide range of connectivity challenges facing businesses today; for example, the WebSphere MQ product family is the IBM SOA messaging backbone. Starting in July, the new WebSphere MQ File Transfer Edition is being previewed. It will add critical file transfer services to the current data connectivity capabilities of the SOA messaging backbone.
The SOA messaging backbone provides the transport layer for moving all kinds of data between IT systems, connecting SOA assets, and supporting the business with more relevant, timely information. Part of the Smart SOA™ approach, the SOA messaging backbone eliminates the need for a custom-built, hard-coded application integration solution which may spend excessive resources rewriting and patching FTP code, resulting in an unreliable, fragile and vulnerable infrastructure.
As many organizations look to SOA to increase the flexibility of IT, it becomes increasingly important to be able to connect the new (service-oriented applications and assets) and the now (existing IT assets that are the lifeblood of your organization). As SOA enables greater interoperability to IT assets and increases the ability to dynamically interconnect these, the need to provide a robust, reliable SOA messaging backbone for these interactions is even more acute. It can preserve a bet-the-business quality of robustness whenever services and non-service-oriented assets interact.
"Companies should consider managed file transfer (MFT) suite vendors that meet their tactical and strategic needs. It’s all too easy to act reactively and deploy technology that only supports one protocol or security standard. It’s much more prudent to deploy a product that can be easily expanded and upgraded to handle multiple protocols and multiple standards in a well-managed way that’s fully auditable, " note Frank Kenney and James Lennard in their Gartner paper, "Magic Quadrant for Managed File Transfer," June 23, 2008.
Customer scenarios with two different connectivity environments
Two common managed file transfer scenarios are described next. They are based on the critical business needs of auditory compliance and multiple transports consolidation.
Need to audit file transfers
A very common business need is to audit file transfers. For instance, a business wants to moves files around their organization. The company may not have an SOA messaging backbone or similar messaging, an ESB, or any other SOA enabling software, and may instead rely on home-grown software built on FTP.
This often results in multiple pain points. The file transfers can be impossible to audit — exposing risks of non-compliance with industry regulations. The unreliable nature of FTP means transfer failures need to be detected somehow (by writing application code) and re-sent, increasing the risk of duplicated transactions. This level of inflexibility can increase development time, maintenance costs and business disruption, with ever-growing costs for additional changes.
A solution using an SOA messaging backbone provides the ability to track movement of files end-to-end around the organization. It can improve an organization’s ability to prepare for their regulatory compliance obligations by helping demonstrate the integrity of data in motion, whether this is sensitive, private information or business transaction data used to compile financial reports.
With file transfers taking place in a reliable environment, applications no longer need to detect transfer failures and corrupted or partially transmitted files, thus eliminating the need to always re-transfer entire batches of files when failures occur. The infrastructure flexibility reduces the burden on the hardware, operating system, applications and networks to lessen the complex distribution of transfer code. A managed file transfer solution, such as WebSphere MQ File Transfer Edition, can meet these challenges with reliable software that logs transfers from source to target IT system, regardless of whether these systems are directly connected or communicate across the backbone.
Need to consolidate multiple transports
Another business scenario involves an organization that is developing and maintaining an entire infrastructure for files — typically built on FTP — alongside their infrastructure for application messaging, such as one based on WebSphere MQ. Maintaining the FTP infrastructure devours time and resources from the IT team and inhibits progress with other SOA initiatives. At the same time, operations teams are duplicating administration of both infrastructures, with poor traceability for file transfers and for data passing between these infrastructures. The duplication of efforts from IT and operations can be a drain on the company’s resources.
In this case, the organizaton can upgrade their existing messaging backbone by adding file transfer services. Now their SOA messaging backbone provides managed file transfer as well as application messaging in one combined multi-purpose infrastructure. Their SOA messaging backbone now delivers a single universal transport for both kinds of traffic. The company benefits include:
An SOA messaging backbone assists customers with a broad spectrum of data movement requirements, enabling connectivity without limits to link virtually all IT platforms, applications and services across the enterprise.
However you plan to use SOA for greater IT flexibility, a messaging backbone provides compelling value for your organization, regardless of its size. If your organization wants to deploy an ESB, the SOA messaging backbone provides a foundational connectivity layer to build on. Small or midsize businesses with immediate needs to connect applications can still realize substantial benefits from implementing a messaging solution. Whatever business you are in, your file transport needs are supported by SOA.