Enterprise Service Bus

Enterprise Service Bus

by David A Chappell
Released June 2004
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781449391096

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Large IT organizations increasingly face the challenge of integrating various web services, applications, and other technologies into a single network. The solution to finding a meaningful large-scale architecture that is capable of spanning a global enterprise appears to have been met in ESB, or Enterprise Service Bus. Rather than conform to the hub-and-spoke architecture of traditional enterprise application integration products, ESB provides a highly distributed approach to integration, with unique capabilities that allow individual departments or business units to build out their integration projects in incremental, digestible chunks, maintaining their own local control and autonomy, while still being able to connect together each integration project into a larger, more global integration fabric, or grid.Enterprise Service Bus offers a thorough introduction and overview for systems architects, system integrators, technical project leads, and CTO/CIO level managers who need to understand, assess, and evaluate this new approach. Written by Dave Chappell, one of the best known and authoritative voices in the field of enterprise middleware and standards-based integration, the book drills down into the technical details of the major components of ESB, showing how it can utilize an event-driven SOA to bring a variety of enterprise applications and services built on J2EE, .NET, C/C++, and other legacy environments into the reach of the everyday IT professional.With Enterprise Service Bus, readers become well versed in the problems faced by IT organizations today, gaining an understanding of how current technology deficiencies impact business issues. Through the study of real-world use cases and integration patterns drawn from several industries using ESB--including Telcos, financial services, retail, B2B exchanges, energy, manufacturing, and more--the book clearly and coherently outlines the benefits of moving toward this integration strategy. The book also compares ESB to other integration architectures, contrasting their inherent strengths and limitations.If you are charged with understanding, assessing, or implementing an integration architecture, Enterprise Service Bus will provide the straightforward information you need to draw your conclusions about this important disruptive technology.

Publisher resources

View/Submit Errata

Table of contents

  1. Enterprise Service Bus
  2. A Note Regarding Supplemental Files
  3. Foreword
  4. Preface
    1. About This Book
      1. Overview of the Chapters
    2. Notational Conventions for ESB Integration Patterns
      1. Diagramming Notations
    3. Conventions Used in This Book
    4. We’d Like to Hear from You
    5. Acknowledgments
  5. 1. Introduction to the Enterprise Service Bus
    1. 1.1. SOA in an Event-Driven Enterprise
    2. 1.2. A New Approach to Pervasive Integration
    3. 1.3. SOA for Web Services, Available Today
    4. 1.4. Conventional Integration Approaches
    5. 1.5. Requirements Driven by IT Needs
    6. 1.6. Industry Traction
      1. 1.6.1. Vendors Adopting the ESB
    7. 1.7. Characteristics of an ESB
      1. 1.7.1. Pervasiveness
      2. 1.7.2. Standards-Based Integration
      3. 1.7.3. Highly Distributed Integration and Selective Deployment
      4. 1.7.4. Distributed Data Transformation
      5. 1.7.5. Extensibility Through Layered Services
      6. 1.7.6. Event-Driven SOA
      7. 1.7.7. Process Flow
      8. 1.7.8. Security and Reliability
      9. 1.7.9. Autonomous but Federated Environment
      10. 1.7.10. Remote Configuration and Management
      11. 1.7.11. XML as the “Native” Datatype of the ESB
      12. 1.7.12. Real-Time Throughput of Business Data
      13. 1.7.13. Operational Awareness
      14. 1.7.14. Incremental Adoption
    8. 1.8. Adoption of ESB by Industry
      1. 1.8.1. Financial Services
      2. 1.8.2. Insurance
      3. 1.8.3. Manufacturing
      4. 1.8.4. Retail
      5. 1.8.5. Telecom
      6. 1.8.6. Energy/Utility
      7. 1.8.7. Food Distribution Network
      8. 1.8.8. Government
    9. 1.9. Summary
  6. 2. The State of Integration
    1. 2.1. Business Drivers Motivating Integration
      1. 2.1.1. IT Spending Trends
      2. 2.1.2. Integration as a High-Ranking Priority
      3. 2.1.3. Regulatory Compliance
        1. 2.1.3.1. Telecom
        2. 2.1.3.2. Sarbanes-Oxley
        3. 2.1.3.3. Government
      4. 2.1.4. Straight-Through Processing (STP)
      5. 2.1.5. Radio Frequency Identification (RFID)
    2. 2.2. The Current State of Enterprise Integration
      1. 2.2.1. The Enterprise Is Not Well Connected
      2. 2.2.2. The Accidental Architecture
        1. 2.2.2.1. Departmental and organizational issues
        2. 2.2.2.2. Moving forward with an accidental architecture
      3. 2.2.3. ETL, Batch Transfers, and FTP
      4. 2.2.4. Integration Brokers
    3. 2.3. Leveraging Best Practices from EAI and SOA
      1. 2.3.1. Adopt XML
      2. 2.3.2. Adopt Web Services and Implement SOA
    4. 2.4. Refactoring to an ESB
      1. 2.4.1. Introduce the ESB at an Individual Project Level
      2. 2.4.2. Propagate the ESB Across a Widely Distributed Enterprise
      3. 2.4.3. Leave and Layer: Connecting into the Existing EAI Broker
      4. 2.4.4. Partner Integration
    5. 2.5. Summary
  7. 3. Necessity Is the Mother of Invention
    1. 3.1. The Evolution of the ESB
    2. 3.2. The ESB in Global Manufacturing
      1. 3.2.1. Putting It to the Test
    3. 3.3. Finding the Edge of the Extended Enterprise
      1. 3.3.1. e-Commerce Trading Hubs
      2. 3.3.2. The Extended Enterprise: The Ever-Changing Edge
      3. 3.3.3. Corralling the Ever-Changing Edge of the Network
    4. 3.4. Standards-Based Integration
      1. 3.4.1. The New Economics of Integration
      2. 3.4.2. Driving Down the Cost of Technology
    5. 3.5. Case Study: Manufacturing
      1. 3.5.1. Building a Real-Time Business
        1. 3.5.1.1. Inventory management
        2. 3.5.1.2. Technical challenges
        3. 3.5.1.3. Availability To Promise (ATP)
      2. 3.5.2. Flexible Partner Integration
    6. 3.6. Summary
  8. 4. XML: The Foundation for Business Data Integration
    1. 4.1. The Language of Integration
      1. 4.1.1. XML Is Human-Readable
    2. 4.2. Applications Bend, but Don’t Break
    3. 4.3. Content-Based Routing and Transformation
    4. 4.4. A Generic Data Exchange Architecture
      1. 4.4.1. Data Translation to and from a Canonical Format
      2. 4.4.2. Adopting a Canonical Data Exchange
      3. 4.4.3. Alternate Approaches
    5. 4.5. Summary
  9. 5. Message Oriented Middleware (MOM)
    1. 5.1. Tightly Coupled Versus Loosely Coupled Interfaces
      1. 5.1.1. Remote Procedure Call (RPC)-Style Programming
      2. 5.1.2. Tightly Coupled Interfaces
      3. 5.1.3. Loosely Coupled Interactions
      4. 5.1.4. Loosely Coupled Interfaces
      5. 5.1.5. N-Squared Data Formats: It’s Really About the Data Transformations
      6. 5.1.6. Get on the Bus
      7. 5.1.7. Loosely Coupled Web Services Standards
    2. 5.2. MOM Concepts
      1. 5.2.1. Abstract Decoupling
      2. 5.2.2. Messaging Models: Publish-and-Subscribe and Point-to-Point
      3. 5.2.3. Topic Hierarchies
      4. 5.2.4. Access Control Lists (ACLs)
      5. 5.2.5. What’s in a Message?
    3. 5.3. Asynchronous Reliability
      1. 5.3.1. Message Autonomy
      2. 5.3.2. Store and Forward
      3. 5.3.3. Message Persistence
      4. 5.3.4. Message Acknowledgments
    4. 5.4. Reliable Messaging Models
      1. 5.4.1. Reliable Publish-and-Subscribe
      2. 5.4.2. Reliable Point-to-Point Queues
      3. 5.4.3. Store-and-Forward Across Multiple Message Servers
    5. 5.5. Transacted Messages
      1. 5.5.1. Local Transactions
      2. 5.5.2. Transactions with Multiple Resources
      3. 5.5.3. An ESB Removes the Low-Level Complexities
    6. 5.6. The Request/Reply Messaging Pattern
      1. 5.6.1. The Reply-Forward Pattern
    7. 5.7. Messaging Standards
      1. 5.7.1. The Java Message Service (JMS)
        1. 5.7.1.1. What about the “J” in JMS?
      2. 5.7.2. Reliable Messaging with SOAP
      3. 5.7.3. Web Services Events and Notifications
    8. 5.8. Summary
  10. 6. Service Containers and Abstract Endpoints
    1. 6.1. SOA Through Abstract Endpoints
    2. 6.2. Messaging and Connectivity at the Core
    3. 6.3. Diverse Connection Choices
    4. 6.4. Diagramming Notations
    5. 6.5. Independently Deployable Integration Services
    6. 6.6. The ESB Service Container
      1. 6.6.1. The Management Interface of the Service Container
      2. 6.6.2. The ESB Service Interface
      3. 6.6.3. Auditing, Logging, and Error Handling
      4. 6.6.4. ESB Service Container Facilities
        1. 6.6.4.1. QoP and QoS
        2. 6.6.4.2. Administration of ESB facilities
      5. 6.6.5. Standardizing ESB Container Connectivity
    7. 6.7. Service Containers, Application Servers, and Integration Brokers
      1. 6.7.1. Adherence to Standards
      2. 6.7.2. Centralized Hub-and-Spoke Processing
      3. 6.7.3. What’s Hosted in the Container
      4. 6.7.4. Management Substrate
      5. 6.7.5. Compiled Class Files Versus Declarative Rules
      6. 6.7.6. Abandon Application Servers?
    8. 6.8. Summary
  11. 7. ESB Service Invocations, Routing, and SOA
    1. 7.1. Find, Bind, and Invoke
    2. 7.2. ESB Service Invocation
    3. 7.3. Itinerary-Based Routing: Highly Distributed SOA
    4. 7.4. Content-Based Routing (CBR)
      1. 7.4.1. Conditional Routing Using BPEL4WS
    5. 7.5. Service Reusability
      1. 7.5.1. Parameterization and Configuration
      2. 7.5.2. Reuse via Composition
    6. 7.6. Specialized Services of the ESB
      1. 7.6.1. Routing Patterns Using Itineraries and Services
      2. 7.6.2. Sophisticated Process Flow Using an Orchestration Service (BPEL4WS)
        1. 7.6.2.1. Recovery of process state
        2. 7.6.2.2. Correlation of asynchronous conversations
        3. 7.6.2.3. BPEL4WS for scripting of orchestration services
      3. 7.6.3. XML Storage and Caching Services
        1. 7.6.3.1. Data caching
        2. 7.6.3.2. Aggregation and reporting
        3. 7.6.3.3. Implementation choices
      4. 7.6.4. Convergence in Process Modeling
    7. 7.7. Summary
  12. 8. Protocols, Messaging, Custom Adapters, and Services
    1. 8.1. The ESB MOM Core
      1. 8.1.1. MOM Interoperability
      2. 8.1.2. The 80/20 Rule of MOM Backbone Versus External Protocols
    2. 8.2. A Generic Message Invocation Framework
      1. 8.2.1. Protocol Bridging
      2. 8.2.2. MOM Bridging
      3. 8.2.3. Direct Protocol Handlers
        1. 8.2.3.1. Synchronous request/reply: invoking an ESB service via HTTP
        2. 8.2.3.2. Asynchronous request/reply
      4. 8.2.4. SOAP Protocol Handler
      5. 8.2.5. Where Do Asynchronous Errors Go?
    3. 8.3. Case Study: Partner Integration
      1. 8.3.1. Secure DMZ Deployment
      2. 8.3.2. Availability To Promise (ATP)
      3. 8.3.3. The SAP Adapter: A Custom ESB Service
      4. 8.3.4. EDI Transportation and Transformation
      5. 8.3.5. Removing the Dependency on the EDI VAN
      6. 8.3.6. Looking Ahead
    4. 8.4. Summary
  13. 9. Batch Transfer Latency
    1. 9.1. Drawbacks of ETL
      1. 9.1.1. ETL Reliability and Data Validity
      2. 9.1.2. Undesired Downtime and the Logistics of Data Synchronization
      3. 9.1.3. Overall Latency of Data Gathering
    2. 9.2. The Typical Solution: Overbloat the Inventory
    3. 9.3. Case Study: Migrating Toward Real-Time Integration
      1. 9.3.1. Inserting the ESB
      2. 9.3.2. Transforming and Routing the Data
        1. 9.3.2.1. Structured message channels
        2. 9.3.2.2. Assigning process definitions
      3. 9.3.3. Considerations
        1. 9.3.3.1. Streamlined data flow
        2. 9.3.3.2. Migrating to XML
        3. 9.3.3.3. Message-based atomicity
        4. 9.3.3.4. Think asynch
        5. 9.3.3.5. Removing the file interface
    4. 9.4. Summary
  14. 10. Java Components in an ESB
    1. 10.1. Java Business Integration (JBI)
      1. 10.1.1. JBI as an ESB Container
      2. 10.1.2. JBI and Other Java Specifications
    2. 10.2. The J2EE Connector Architecture (JCA)
      1. 10.2.1. Domain Expertise
      2. 10.2.2. ESB to Application Server Connectivity
    3. 10.3. Java Management eXtensions (JMX)
      1. 10.3.1. A JMX Primer
        1. 10.3.1.1. JMX server environment
        2. 10.3.1.2. JMX client protocol adapters
        3. 10.3.1.3. JMX connector interface
        4. 10.3.1.4. JMX MBean interfaces
      2. 10.3.2. JMX Management in an ESB Architecture
      3. 10.3.3. JMX-Managed Components in an ESB
      4. 10.3.4. Directory Cache
      5. 10.3.5. Template-Based Remote Configuration Replication
    4. 10.4. Summary
  15. 11. ESB Integration Patterns and Recurring Design Solutions
    1. 11.1. The VETO Pattern
      1. 11.1.1. Variations: The VETRO Pattern
    2. 11.2. The Two-Step XRef Pattern
    3. 11.3. Portal Server Integration Patterns
      1. 11.3.1. The Portal Server Deployment Architecture
      2. 11.3.2. Portal Server Challenges
        1. 11.3.2.1. Synchronous sequential aggregation
        2. 11.3.2.2. Web services
        3. 11.3.2.3. Concurrency via multithreaded request
        4. 11.3.2.4. Long-duration conversation management
        5. 11.3.2.5. Asynchrony and reliability using JMS
        6. 11.3.2.6. Connectivity using application adapters
      3. 11.3.3. Inserting an ESB as the Integration Backbone
    4. 11.4. The Forward Cache Integration Pattern
      1. 11.4.1. Data Forwarding Using Publish-and-Subscribe
      2. 11.4.2. Data Forwarding Using Itinerary-Based Routing
      3. 11.4.3. Other Considerations of the Forward Cache Pattern
        1. 11.4.3.1. Integration first, portal second
    5. 11.5. Federated Query Patterns
      1. 11.5.1. Cache Push Versus Pull Patterns
      2. 11.5.2. The Real-Time Request Pattern
      3. 11.5.3. The Long-Duration Request Pattern
      4. 11.5.4. Federated Query Pattern Variations
        1. 11.5.4.1. Federated query using itinerary processes
        2. 11.5.4.2. Federated query using BPEL4WS and an orchestration service
    6. 11.6. Summary
  16. 12. ESB and the Evolution of Web Services
    1. 12.1. Composability Among Specifications
    2. 12.2. Summary of WS-* Specifications
    3. 12.3. Adopting the WS-* Specifications in an ESB
      1. 12.3.1. WS-* Does Not Imply Enterprise Capabilities
      2. 12.3.2. WS-* Does Not Imply ESB
    4. 12.4. Conclusion
  17. A. Appendix: List of ESB Vendors
  18. 13. Bibliography
    1. Analyst Reports
    2. Books
    3. Miscellaneous
    4. Web Services Specifications
    5. Java Specifications
  19. Index
  20. About the Author
  21. Colophon
  22. Copyright

Product information

  • Title: Enterprise Service Bus
  • Author(s): David A Chappell
  • Release date: June 2004
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781449391096