Introduction

Why Is Integration So Difficult?

 Even with the knowledge of how machines, platforms, and applications
exchange data, the integration process is still complex for the following
reasons:
 Business has to be supported
 Numerous open and proprietary protocols and standards, including:
◦ CORBA (interoperability)
◦ Copybook (legacy system)
 Workflow implementation
◦ Real-time
◦ Straight-through processing
◦ Aggregation
◦ Asynchronous or synchronous
Enterprises are typically comprised of hundreds, if not thousands, of
applica- tions that are custom built, acquired from a third party, part of a
legacy system, or a combination thereof, operating in multiple tiers of
different operating sys- tem platforms. It is not uncommon to find an
enterprise that has 30 different Web sites, three instances of SAP, and
countless departmental solutions.

The Enterprise Resource Planning (ERP) vendors have had some success at
creating larger-than-ever business applications. The reality, though, is that
even the heavyweights like SAP, Oracle, Peoplesoft, and the like perform
only a fraction of the business functions required in a typical enterprise. We
can see this easily by the fact that ERP systems are one of the most popular
integration points in today's enterprises.

Second, spreading business functions across multiple applications provides

the business with the flexibility to select the "best" accounting package, the
"best" customer relationship management software, as well as the "best"
order- processing system for its needs. Usually, IT organizations are not
interested in a single enterprise application that does it all, nor is such an
application possible given the number of individual business requirements.

Over time, developers have overcome these challenges with four main
approaches:
1. File Transfer --One application writes a file that another later reads. The
applications need to agree on the filename and location, the format of
 the file, the timing of when it will be written and read, and who will
delete the file.
2. Shared Database --Multiple applications share the same database
schema, located in a single physical database. Because there is no
duplicate data storage, no data has to be transferred from one
application to the other.
3. Remote Procedure Invocation --One application exposes some of its
functionality so that it can be accessed remotely by other applications
as a remote procedure. The communication occurs in real time and
synchronously.
4. Messaging --One application publishes a message to a common
mes- sage channel. Other applications can read the message from the
channel at a later time. The applications must agree on a channel as
well as on the for- mat of the message. The communication is
asynchronous.

Why Is Integration So Important?

As companies expand from local to global operations, acquire other
companies or are themselves acquired, they need to integrate systems and
applications becomes a necessity to maintain efficient operations and be
able to do the following:
 Support the business and company growth
 Trade with external and internal business partners
 Exchange data and information between applications and back end
systems (e.g. send instructions, payments, etc.)
 Automate IT processes to become more efficient and agile

How Can the Integration Process Be Simplified?

The architecture design that Red Hat proposes to simplify the development
of integration projects is based on the following key concepts:
 Loose coupling (decouple the producer from the consumer)
 Expose the flows and processes by using standard-based
components/endpoints
 Mediate and route the information using a service bus layer to
propagate data exchanges
 Design the project as a collection of modules and package them as
individual archive files
 b u s _ la y e r.m p 4
Decoupling the Producer and the Consumer
 Use a message to decouple the producer from the consumer and to
send an event containing the information about the action to be
performed.
 The message is a unit of transport that contains three elements:

Mediation and Routing

 A message is not enough when designing for complex architectural use
cases.
 A traditional network layer transports information, but this layer does
not make data routing decisions.
 Data routing decisions should be handled by a specific integration
technology or framework called a mediation and routing engine.

m e d ia tio n _ ro u t in g .m p 4
What a Java Integration Framework Looks Like
 Supports all concepts presented previously
 Simplifies the following:
◦ How endpoints are exposed
◦ How messages are processed
◦ How applications are integrated
 Also supports:
◦ Enterprise integration patterns (EIPs)
◦ A standards-based application program interface (API)
 A domain specific language (DSL)
Enterprise Integration Patterns
 Provide solutions to application integration problems
 Like Java patterns used by Java developers for Java application design
problems
 Provide vocabulary and grammar that analysts, architects, and
developers can use to design solutions for complex integration projects
 Book by Gregor Hohpe and Bobby Woolf covers 65 patterns, from
simple to complex, including:
◦ Splitter
◦ Content-based router
◦ Aggregator
◦ Filter

DSL
 The DSL is not a new programming language. It is a vocabulary used to
represent and implement EIPs.
 Most of the time, a DSL is expressed using a fluent API or fluent
interface.
  DSL is mini-language designed for a special purpose.
 Routes are described using DSL languages such as Java, XML (Spring,
Blueprint), Scala, or Groovy
 Wordings and vocabularies are similar to English
 The model used relies on the same Java classes

The first line in the above example creates an object which is an instance of
an anonymous subclass of RouteBuilder with the specified configure()
method.
The CamelContext.addRoutes(RouterBuilder builder) method invokes
builder.setContext(this) – so the RouteBuilder object knows which
CamelContext object it is associated with – and then invokes
builder.configure(). The body of configure() invokes methods such as from(),
filter(), choice(), when(), isEqualTo(), otherwise() and to().
The RouteBuilder.from(String uri) method invokes getEndpoint(uri) on the
CamelContext associated with the RouteBuilder object to get the specified
Endpoint and then puts a FromBuilder "wrapper" around this Endpoint. The
FromBuilder.filter(Predicate predicate) method creates a FilterProcessor
object for the Predicate (that is, condition) object built from the
header("foo").isEqualTo("bar") expression. In this way, these operations
incrementally build up a Route object (with a RouteBuilder wrapper around
it) and add it to the CamelContext object associated with the RouteBuilder.

Difference of Java DSL with XML

In particular, Java DSL is less verbose than its XML counterpart. In addition,
many integrated development environments (IDEs) provide an auto-
completion feature in their editors. This auto-completion feature works with
Java DSL, thereby making it easier for developers to write Java DSL.

Currently, Camel does not provide such a DSL parser

Camel Framework
 Open source projects use Java and various design frameworks for
application integration:
◦ Apache Camel
◦ Spring Integration
◦ Mule Software
 Apache Camel project
◦ Began in 2007
◦ Only framework that implements and extends all EIPs using an API,
including:
▪ RecipientList
▪ Throttler, delayer
▪ Loadbalancer
◦ Deploys standalone or inside any Java container used with Red Hat®
JBoss® Fuse (JBoss Fuse), Red Hat® JBoss® Enterprise Application
Platform (JBoss EAP), Red Hat® JBoss® Web Server, Apache Tomcat,
and Wildfly
◦ Works directly with most transport or messaging models to rapidly
integrate existing services and applications

Reference : http://camel.apache.org/maven/current/

Summary
This module introduced you to the following key integration concepts:
  Messaging and the decoupling of the producer and consumer
 How to simplify integration projects
 EIP and DSL
 Apache Camel framework

Core Engine
Messaging & Payload
A Camel Exchange object is a wrapper that encapsulates a message and its
metadata. It consists of the following elements:
 Body
 Headers and Properties
 Attachment (optional)
 Exception
 Exchange ID

p ip e lin e .m p 4
Processor
 Processors encapsulate logic that manipulates an Exchange.
 Route definitions start with from() and are followed by a list of
.to()processors: from().to().to()
 Route definitions create two lists:
◦ Inputs = list of from()
◦ Outputs = list of .to()
 The first processor acts as a Consumer and initiates Exchange creation.
 (Optional) The pipeline can contain processor(s) acting as a producer.

Using a processor in a route

Component and Endpoint

 The Component class generates endpoint objects based on parameters
 The consumer or producer endpoints are created from the factory
endpoint class
 Consumer: Polls, schedules a call, or waits (event-driven)
 Producer: Sends a message synchronously or asynchronously

Endpoint
The term endpoint is often used when talking about inter-process
communication. For example, in client-server communication, the client is
one endpoint and the server is the other endpoint. Depending on the
context, an endpoint might refer to an address, such as a host:port pair for
TCP-based communication, or it might refer to a software entity that is
contactable at that address.

Camel provides out-of-the-box support for endpoints implemented with

many different communication technologies. Here are some examples of the
Camel-supported endpoint technologies.
 A JMS queue.
 A web service.
 A file. A file may sound like an unlikely type of endpoint, until you
realize that in some systems one application might write information to
a file and, later, another application might read that file.
 An FTP server.
 An email address. A client can send a message to an email address,
and a server can read an incoming message from a mail server.
 A POJO (plain old Java object).
In a Camel-based application, you create (Camel wrappers around) some
endpoints and connect these endpoints with routes. Camel defines a Java
interface called Endpoint. Each Camel-supported endpoint has a class that
implements this Endpoint interface.

Camel provides a separate Javadoc hierarchy for each communications

technology supported by Camel. Because of this, you will find
documentation on, say, the JmsEndpoint class in the JMS Javadoc
hierarchy, while documentation for, say, the FtpEndpoint class is in the
FTP Javadoc hierarchy.

Components
Component is confusing terminology; EndpointFactory would have been
more appropriate because a Component is a factory for creating Endpoint
instances. For example, if a Camel-based application uses several JMS
queues then the application will create one instance of the JmsComponent
class (which implements the Component interface), and then the application
invokes the createEndpoint() operation on this JmsComponent object
several times. Each invocation of JmsComponent.createEndpoint() creates
an instance of the JmsEndpoint class (which implements the Endpoint
interface). Actually, application-level code does not invoke
Component.createEndpoint() directly. Instead, application-level code
normally invokes CamelContext.getEndpoint(); internally, the CamelContext
object finds the desired Component object (as I will discuss shortly) and
then invokes createEndpoint() on it.
Consider the following code.

myCamelContext.getEndpoint("pop3://john.smith@mailserv.example.com?
password=myPassword");

The parameter to getEndpoint() is a URI. The URI prefix (that is, the part
before ":") specifies the name of a component. Internally, the CamelContext
object maintains a mapping from names of components to Component
objects. For the URI given in the above example, the CamelContext object
would probably map the pop3 prefix to an instance of the MailComponent
class. Then the CamelContext object invokes
createEndpoint("pop3://john.smith@mailserv.example.com?
password=myPassword") on that MailComponent object. The
createEndpoint() operation splits the URI into its component parts and uses
these parts to create and configure an Endpoint object.
In the previous paragraph, I mentioned that a CamelContext object
maintains a mapping from component names to Component objects. This
raises the question of how this map is populated with named Component
objects. There are two ways of populating the map. The first way is for
application-level code to invoke CamelContext.addComponent(String
componentName, Component component). The example below shows a
single MailComponent object being registered in the map under 3 different
names.

Component mailComponent = new

org.apache.camel.component.mail.MailComponent();
myCamelContext.addComponent("pop3", mailComponent);
myCamelContext.addComponent("imap", mailComponent);
myCamelContext.addComponent("smtp", mailComponent);

The second (and preferred) way to populate the map of named Component
objects in the CamelContext object is to let the CamelContext object
perform lazy initialization.
CamelContext
A CamelContext object represents the Camel runtime system.
The CamelContext represents a single Camel routing rulebase.

public interface CamelContext

extends SuspendableService, RuntimeConfiguration

Interface used to represent the context used to configure routes and the
policies to use during message exchanges between endpoints.
The context offers the following methods to control the lifecycle:

start() - to start (important: the start method is not blocked, see more details here)
stop() - to shutdown (will stop all routes/components/endpoints etc and clear internal state/cache)
SuspendableService.suspend() - to pause routing messages
SuspendableService.resume() - to resume after a suspend

You typically have one CamelContext object in an application. A typical

application executes the following steps.
 Create a CamelContext object.
 Add endpoints – and possibly Components, which are discussed in
Section 4.5 ("Components") – to the CamelContext object.
 Add routes to the CamelContext object to connect the endpoints.
 Invoke the start() operation on the CamelContext object. This starts
Camel-internal threads that are used to process the sending, receiving
and processing of messages in the endpoints.
 Eventually invoke the stop() operation on the CamelContext object.
Doing this gracefully stops all the endpoints and Camel-internal
threads.

Note that the CamelContext.start() operation does not block indefinitely.

Rather, it starts threads internal to each Component and Endpoint and then
start() returns. Conversely, CamelContext.stop() waits for all the threads
internal to each Endpoint and Component to terminate and then stop()
returns.
If you neglect to call CamelContext.start() in your application then
messages will not be processed because internal threads will not have been
created.
If you neglect to call CamelContext.stop() before terminating your
application then the application may terminate in an inconsistent state. If
you neglect to call CamelContext.stop() in a JUnit test then the test may fail
due to messages not having had a chance to be fully processed.

Camel uses a Java based Routing Domain Specific Language (DSL) or an Xml
Configuration to configure routing and mediation rules which are added to a
CamelContext to implement the various Enterprise Integration Patterns.

At a high level Camel consists of a CamelContext which contains a collection

of Component instances. A Component is essentially a factory of Endpoint
instances.

An Endpoint acts rather like a URI or URL in a web application or a

Destination in a JMS system; you can communicate with an endpoint; either
sending messages to it or consuming messages from it. You can then create
a Producer or Consumer on an Endpoint to exchange messages with it.

CamelTemplate
Camel used to have a class called CamelClient, but this was renamed to be
CamelTemplate to be similar to a naming convention used in some other
open-source projects, such as the TransactionTemplate and JmsTemplate
classes in Spring.
The CamelTemplate class is a thin wrapper around the CamelContext class.
It has methods that send a Message or Exchange – to an Endpoint .This
provides a way to enter messages into source endpoints, so that the
messages will move along routes – to destination endpoints.

URI-Scheme
 Endpoints are defined using this URI syntax: “scheme://contextPath?
key=value&key=value …”
 Scheme is the name of a component
 Camel checks the classpath to discover the component to be used

A URI (uniform resource identifier) is a URL or a URN. So, to fully understand

what URI means, you need to first understand what is a URN.
URN is an acronym for uniform resource name. There are may "unique
identifier" schemes in the world, for example, ISBNs (globally unique for
books), social security numbers (unique within a country), customer
numbers (unique within a company's customers database) and telephone
numbers. Each "unique identifier" scheme has its own notation. A URN is a
wrapper for different "unique identifier" schemes. The syntax of a URN is
"urn:<scheme-name>:<unique-identifier>". A URN uniquely identifies a
resource, such as a book, person or piece of equipment. By itself, a URN
does not specify the location of the resource. Instead, it is assumed that a
registry provides a mapping from a resource's URN to its location. The URN
specification does not state what form a registry takes, but it might be a
database, a server application, a wall chart or anything else that is
convenient. Some hypothetical examples of URNs are
"urn:employee:08765245", "urn:customer:uk:3458:hul8" and "urn:foo:0000-
0000-9E59-0000-5E-2". The <scheme-name> ("employee", "customer" and
"foo" in these examples) part of a URN implicitly defines how to parse and
interpret the <unique-identifier> that follows it. An arbitrary URN is
meaningless unless: (1) you know the semantics implied by the <scheme-
name>, and (2) you have access to the registry appropriate for the
<scheme-name>. A registry does not have to be public or globally
accessible. For example, "urn:employee:08765245" might be meaningful
only within a specific company.
To date, URNs are not (yet) as popular as URLs. For this reason, URI is widely
misused as a synonym for URL.
IRI is an acronym for internationalized resource identifier. An IRI is simply an
internationalized version of a URI. In particular, a URI can contain letters and
digits in the US-ASCII character set, while a IRI can contain those same
letters and digits, and also European accented characters, Greek letters,
Chinese ideograms and so on.
Exchange Pattern
 Apache Camel supports two topologies:
◦ Fire Forget (= InOnly)
◦ Request / Reply (= InOut)
 Pattern InOnly
◦ Only In compartment is used
◦ No response is returned to the consumer
 Pattern InOut
◦ The In object transports the incoming message
◦ The response is assigned to the Out compartment
 Exchange: Always returned to the consumer (InOnly/InOut)
 Pattern InOut
◦ The In object transports the incoming message
◦ The response is assigned to the Out compartment
 Exchange: Always returned to the consumer (InOnly/InOut)

c h a rle s _ in o u t.m p 4
Message and Exchange
The Message interface provides an abstraction for a single message, such
as a request, reply or exception message.
There are concrete classes that implement the Message interface for each
Camel-supported communications technology. For example, the JmsMessage
class provides a JMS-specific implementation of the Message interface. The
public API of the Message interface provides get- and set-style methods to
access the message id, body and individual header fields of a messge.
The Exchange interface provides an abstraction for an exchange of
messages, that is, a request message and its corresponding reply or
exception message. In Camel terminology, the request, reply and exception
messages are called in, out and fault messages.
There are concrete classes that implement the Exchange interface for each
Camel-supported communications technology. For example, the
JmsExchange class provides a JMS-specific implementation of the Exchange
interface. The public API of the Exchange interface is quite limited. This is
intentional, and it is expected that each class that implements this interface
will provide its own technology-specific operations.
Application-level programmers rarely access the Exchange interface (or
classes that implement it) directly. However, many classes in Camel are
generic types that are instantiated on (a class that implements) Exchange.
Because of this, the Exchange interface appears a lot in the generic
signatures of classes and methods.

Processor
The Processor interface represents a class that processes a message.

Notice that the parameter to the process() method is an Exchange rather

than a Message. This provides flexibility. For example, an implementation of
this method initially might call exchange.getIn() to get the input message
and process it. If an error occurs during processing then the method can call
exchange.setException().
An application-level developer might implement the Processor interface with
a class that executes some business logic. However, there are many classes
in the Camel library that implement the Processor interface in a way that
provides support for a design pattern in the EIP book.

package org.apache.camel;
public interface Processor {
 void process(Exchange exchange) throws Exception;
}

Routes, RouteBuilders and Java DSL

A route is the step-by-step movement of a Message from an input queue,

through arbitrary types of decision making (such as filters and routers) to a
destination queue (if any). Camel provides two ways for an application
developer to specify routes. One way is to specify route information in an
XML file.

The other way is through what Camel calls a Java DSL (domain-specific
language).

Camel supports the definition of routing rules using a Java DSL (domain
specific language) which avoids the need for cumbersome XML using a
RouteBuilder.

Filters
You can combine simple routes with filters.
Choices
With a choice you provide a list of predicates and outcomes along with an
optional default otherwise clause.

Camel Container
 CamelContext is the runtime environment
 Registry for endpoints, routes
 Acts as a server runtime
 Can be deployed in any kind of Java Container: JBoss Fuse/JBoss A-MQ,
JBoss EAP, Wildfly, Apache Tomcat, Apache ActiveMQ or in standalone
 CamelContext is created programmatically in Java, using Dependency
injection (Spring, Blueprint) or using Java Annotation (CDI)
Route Registration - JAVA DSL
The RouteBuilder is a base class which is derived from to create routing
rules using the DSL. Instances of RouteBuilder are then added to the
CamelContext.
Route Registration - XML DSL

Route Registration - Scan

LifeCycle

Camel uses a simple lifecycle interface called Service which has a single
start() and stop() method.
Various classes implement Service such as CamelContext along with a
number of Component and Endpoint classes.
When you use Camel you typically have to start the CamelContext which
will start all the various components and endpoints and activate the routing
rules until the context is stopped again.

public interface Service

Represents the core lifecycle API for POJOs which can be started and
stopped
The CamelContext provides methods to control its lifecycle:
 start
 stop
 suspend Camel 2.5
 resume Camel 2.5
The operations is paired: start/stop and suspend/resume.
Stop is performing a Graceful shutdown which means all its internal state,
cache, etc is cleared. And the routes is being stopped in a graceful manner
to ensure messages is given time to complete. If you start a CamelContext
after a stop, then its performing a cold start, recreating all the state, cache
etc. again.
Instead you can use the suspend/resume operations. They will keep the
CamelContext warm and only suspend/stop routes using the same Graceful
shutdown feature. This ensures messages is given time to complete.
End users is encouraged to use suspend/resume if you are temporary
stopping a Camel application.
Service lifecycle

A service (org.apache.camel.Service) in Camel adheres to the following

lifecycle states as illustrated in the diagram below:
A service can optimally support suspend/resume by the
org.apache.camel.SuspendableService. This means not all services in Camel
supports suspension. It's encouraged that consumers support suspension
which allows to suspend/resume routes.

The org.apache.camel.impl.ServiceSupport is a good base class to extend

for custom services as it offers the basic functionally to keep track of state.

Routes lifecycle
Routes in Camel have the following operations to control its lifecycle
 start
 stop
 suspend
 resume
 remove (previously named shutdown)
The remove operation will remove the route, for example in JMX the route
will then be unregistered and its gone. So only use remove if you really want
to remove the route. The route must have been stopped before you can
remove.
The start and resume operations in JMX checks the state beforehand. So if a
route is stopped and you click resume, it will know to invoke start. And
likewise if a route has been suspended and you click start it knows to
resume instead. This makes management a bit easier.

Summary
This module introduced you to the following topics:
 Apache Camel core engine
 Key concepts of the Apache Camel framework
 Container where the integration project is deployed
 Development of an Apache Camel project in JBoss Developer Studio