Você está na página 1de 202

Webs

ens
e
TRI
TONSol
ut
i
on

Techni
calEnabl
ementPr
ogr
am

E
S
GAv
7.
6Cour
s
e201
S
t
udentGui
de
Rev
.
691267

2011 Websense, Inc. All rights reserved.

Websense and the Websense logo are registered trademarks of Websense, Inc. in the United States and
other countries. TRITON is a trademark of Websense, Inc. in the United States and other countries. All
other trademarks are properties of their respective owners.

Table of Contents
Websense Email Security Gateway Initial Setup. . . . . . . . . . . . . . . 7
ESG Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
ESG Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Personal Email Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Policy Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Advanced Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Clustering Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Archive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
ESG Log and Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Troubleshooting Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

ESGA v7.6 201 5

6 ESGA v7.6 201

Websense Email Security


Gateway Initial Setup

This chapter focuses on the initial setup and configuration of the Email Security
Gateway (ESG) module in a Websense V-Series appliance.
Prior to the physical and logical deployment of the appliance within the network, you
need to understand the fundamental concepts behind email security. Much of the
initial setup has to do with your email domains and ensuring only authorized users can
send and receive email through the ESG appliance.
During the initial setup of ESG, the wizard requires you to define:
1. Email server IP address: The IP address of the mailboxes server, which stores,
receives and sends email to and from local users.
You can add additional email server after the initial setup.
2. Protected domain address: This is the email domain where the mailboxes reside.
For example, if the email is salestraining@websense.com then the protected
domain is websense.com.
You can add additional protected domain after the initial setup.
3. Trusted IP address: In this field you enter the IP address of the hosts which are
allowed to send email (inbound connection) through ESG and not be subject to IP
filtering rules.
4. Data security server IP address: In this field you enter the IP address of the DSS
server so you can take advantage of the DLP integration offered by ESG.
5. Log server IP address: In order for ESG to be able to generate reports, you need
to configure it to talk to an off-box log server. The log server should be installed
before you commence the ESG initial setup.
6. Notification email address: A valid email address is required which will serve as
the default address for all notifications from ESG.
This email address should be a shared mailbox that is available to multiple
administrator level individuals. This ensures maximum supportability for your
environment.

ESGA v7.6 201 7

Websense Email Security Gateway Initial Setup

Slide 1:1

ESG Initial Setup


Before you begin the configuration and installation of the ESG appliance you need to
gather some key information needed during the initial setup process.

Email server IP address

Protected domain

Trusted IP address

IP address and administrator account information for the DSS server

IP address of the log server

Shared mailbox (or account) where ESG can send all email alerts and notifications

Steps of ESG Initial Setup


1. Install the Websense V-Series appliance, including setting up the rack, connecting
network cables, connecting the input device for command line interface, etc.
2. Run the first boot wizard through command line interface. Security mode and
Appliance Controller interface (C) are configured during this step.
a. Security mode - the appliance performs email security protection (ESG
mode), or web security protection (WSG mode), or both (ESG+WSG).
Well talk about security mode with more details later.

8 ESGA v7.6 201

Websense Email Security Gateway Initial Setup

b. Appliance Controller interface - IP address of Appliance Manager logon


portal.
Note
This is NOT ESG configuration wizard.
3. Access Appliance Manager via browser to configure ESG DomU IP address and
other information.
4. Install TRITON Unified Security Center, also called TRITON console, including
Email Security, Data Security, etc; install log database (if not pre-installed) and
log server.
5. Login to TRITON console, switch to Email Security tab, then run the ESG
configuration wizard. Email server, protected domain, trusted IP, IP addresses of
DSS server, log server, and shared mailbox for notifications are set during this
step.

ESGA v7.6 201 9

Websense Email Security Gateway Initial Setup

Slide 1:2

Protected Domains
The concept of a protected domain has a slightly different meaning when you refer to
inbound messages and outbound messages.
An inbound message is defined as a message sent from outside your company domain
to an account within your company domain. The filtering of inbound messages is
fairly simple. The ESG reads the recipient address and if it does not match any of the
configured protected domains, the email is immediately rejected.
For example, if an ESG appliance configured with websense.com and surfcontrol.com
as protected domains receives an message sent from johndoe@gmail.com to
pippo@yahoo.com, it will immediately reject the message because pippo@yahoo.com
does not match either websense.com or surfcontrol.com.
In ESG, there are four directions defined:

Internal: From protected domain to protected domain

Inbound: From other domain to protected domain

Outbound: From protected domain to other domain

Open Relay: From other domain to other domain

10 ESGA v7.6 201

Websense Email Security Gateway Initial Setup

Slide 1:3

Email Relay
During the ESG configuration wizard, you have entered some key data, which makes
the ESG a closed relay. Only the following email messages are routed (relayed):

From any sender (coming from any IP) to the protected domain

From permitted IP (typically the internal email servers) and from a sender in the
protected domain to any recipient

The idea is to ensure the ESG only routes valid email and does not become an open
relay. Hackers and spammers often use open relays and misconfigured email servers
to launch an attack or send out massive amounts of emails, in relative security. All
traffic points back to the compromised email server.

ESGA v7.6 201 11

Websense Email Security Gateway Initial Setup

Slide 1:4

Required Components
Two domains inside ESG appliance
Any appliance running ESG has at least two domains configured:

ESG domain: contains an SSL interface for communicating with TRITON


Management Server and the Personal Email Manager (PEM) module for end users
to manage their isolated email according to the policies defined by administrators.
Both TRITON console and the PEM module store data on Log and Report Server.

Appliance Manager: provides the facilities for administrators to connect and


configure the appliance.

Well bring more details about the ESG domains in the ESG Architecture chapter.

Two types of users


There are two types of users who can access the various modules inside the two
domains:

Administrator

can connect and configure the appliance directly

cant access ESG domain directly, but through TRITON console running on
an off-box server to set ESG configuration and policies

12 ESGA v7.6 201

Websense Email Security Gateway Initial Setup

End user with a valid mailbox

can access the PEM module in ESG domain directly without TRITON
console

Important
There is no direct administrative access to ESG module.
Administrators have to install TRITON console off-box to
be able to configure ESG domain.

Lets assume the following scenario:

ESG domain has IP address 172.16.90.100

Appliance Manager has IP address 172.16.90.101

TRITON Management Server has IP address 172.16.90.200

The following statements are correct:

The administrator logs to the TRITON manager, on IP address 172.16.90.200, to


configure the ESG domain. You need to point your browser to the IP address
172.16.90.200

The TRITON Management Server can communicate directly with the ESG
module at IP address 172.16.90.100

The administrator can configure the appliance settings by pointing the browser the
IP address 172.16.90.101

An end user can access the PEM module by pointing the browser directly to the IP
address of the ESG module at 172.16.90.100

Tip
Any appliance running ESG needs at least two interfaces
with two separate IP addresses: one bound to ESG domain,
the other to Appliance Manager.

ESGA v7.6 201 13

Websense Email Security Gateway Initial Setup

Slide 1:5

Network Interfaces
The V5000 appliance has 4 NIC and the V10000 has 6 NIC. The Appliance Manager
domain always binds to the C interface.

On the V10000

N interface is used for the Network Agent domain.

E1 and E2 interfaces are bound to the ESG domain. You can use one or both
interfaces depending on your environment. You could only use E1; and you could
use E1 as internal interface and E2 as external interface too.

P1 and P2 are bound to the WSG domain.


Tip
On a V10000 you can run both WSG and ESG at the same
time.

On the V5000 (running ESG only)

P1 and P2 interface are bound to the ESG domain.

C interface is bound to the Appliance Manager.

14 ESGA v7.6 201

Websense Email Security Gateway Initial Setup

On the V5000 (running WSG only)

N interface is bound to the Network Agent domain.

C interface is bound to the Appliance Manager.

P1 and P2 are bound to the WSG domain.


Tip
On a V5000 you can run either WSG or ESG.

ESGA v7.6 201 15

Websense Email Security Gateway Initial Setup

16 ESGA v7.6 201

ESG Deployment

ESG is typically deployed in DMZ zone. While other deployment options are
possible, this is the most common and the one that Websense recommends.
By design, ESG can process both incoming and outgoing messages. Some companies
may decide to process only incoming messages; however it is most common to
process messages in both directions.
When processing inbound messages, the ESG appliance receives the email from the
Internet (or possibly the Intranet, for large scale complex deployments), scans the
messages, and forwards them to company internal mail server.
The incoming messages can be directed to the ESG appliances in a variety of ways,
depending on the customer preferences and final objectives:

MX Record re-configuration

Firewall routing policies

In order to process outbound messages from inside the company, there are two main
possible configuration options. The options mostly depend on the mail client settings:

Clients not using SMTP - the mail administrator configures the SMTP server to
relay all outbound traffic to the ESG appliance. This is the most common
configuration.

Clients with configured SMTP server - the mail administrator can configure all
clients to point to ESG directly. Then the ESG routes the messages as necessary.
This setting, while not very common enables filtering of all internal email
messages.
Typically, it is easier to assign the IP address of the SMTP server to the ESG
appliance and then assign a new IP address to the SMTP server.

Warning
All configurations require firewall re-configuration.

ESGA v7.6 201 17

ESG Deployment

Slide 2:1

ESG Deployment
The simplest deployment for an email appliance is to put in on premise and configure
the firewalls or the DNS (or both) to direct incoming mail to the newly deployed
appliance.
You then configure the existing email server (or the client, in rare cases) to send all the
outbound email through the ESG appliance.
In addition to the on premises deployment, you can integrate ESG with the cloud
solution and you can pre-filter all incoming email in the cloud; this reduces
significantly the amount of email that the ESG has to process on premises.
To scale to any size environment, the ESG supports clustering. This advanced feature
also ensures that email filtering is always up and running in your network.

18 ESGA v7.6 201

ESG Deployment

Slide 2:2

Standard Network Deployment


This network diagram represents a very simple network configuration. Regardless of
the complexity of the actual network where you plan to deploy ESG, you can always
reduce it, at least conceptually, to the one shown above.
Inbound email messages flow according to the following steps:
1. The email messages arrive from the Internet to the perimeter firewall. Permitted
traffic is allowed through.
2. The firewall permits the messages to reach the V10000 appliance in the DMZ.
Tip
Typically you should install the V-Series appliances
running ESG in the DMZ. On appliances configured in
dual mode, you can have the E2 and P2 interface in the
DMZ and the E1 and P1 interfaces in the LAN.
3. The ESG processes the messages and matches them against the running policies.
Permitted messages flow to the email server.

ESGA v7.6 201 19

ESG Deployment

4. The email server receives the messages which passed the ESG filtering. This
email server can have actual mailbox on it or be a bridgehead server, which
distributes the messages to other mail servers based on internal routing policies.
The messages are now available for pick/delivery by the end users.
For outbound messages, you can reverse the process. The users send the messages to
the mail server that is configured to forward all outbound messages to the ESG
appliance. All messages that pass the filtering policies on the ESG, flow from the VSeries appliance, through the firewall to the Internet, where they reach their final
destination.

20 ESGA v7.6 201

ESG Deployment

Slide 2:3

Standard Network Deployment with DNS


This network diagram is very similar to the previous one. The added steps focus on the
role of the DNS lookup by a sending email server.
Lets assume that a Gmail user wants to send a message to a Websense employee. The
process of delivering email messages consists of the following steps:
1. The Gmail email server queries the DNS for the Websense MX record. The MX
record contains the information about the first hop where to send the email bound
for the websense.com domain.
Tip
When sending an email, the transmitting server only learns
about the first relay host, which processes email for the
recipient. There can be any number of relay hosts between
sender and recipient. Each host only knows about the next
one. None has the fully visibility of the source-todestination path.
The DNS server responds with the information about the next hop where to send
the messages for the Websense domain. The reply may contain one or more

ESGA v7.6 201 21

ESG Deployment

values. The sending server should always pick the one with lowest preference
value. When more entries have the same preference value the sending server
should process them in the order received, till find the one responding.
2. The email server connects to the next hop in the chain. This can be:
- A server in the cloud
- A perimeter firewall
- An ESG appliance
In the this example, the MX record returns the IP address of one of the routable
interfaces on the firewall.
3. The messages arrive to the premise firewall. Permitted traffic is allowed through.
4. The firewall permits the messages to reach the V10000 appliance in the DMZ.
5. The ESG processes the messages and matches them against the running policies.
Permitted messages flow to the email server.
6. The email server received the messages which passed the ESG filtering. This
email server can have actual mailbox on it or be a bridgehead server, which
distributes the messages to other mail servers based on internal routing policies.
The messages are now available for pick/delivery by the end users.

22 ESGA v7.6 201

ESG Deployment

Slide 2:4

ESG Anywhere Deployment


In this scenario, the customer has enabled the Hybrid filtering option. All incoming
email is pre-filtered by the cloud service. This deployment scenario dramatically
reduces the amount of inbound messages that the ESG on premises has to process.
Considering that over 80% of all incoming messages (by number not size) is spam,
using the cloud service is a very efficient and effective approach.
For incoming messages, the flow is as follows:
1. The sender email server queries the DNS for the recipient MX record. The DNS
server responds with the information about the next hop, in this case the cloud
service.
2. The email server connects to the cloud service.
3. The Hybrid service sends the messages that passed the filtering policies, to the
next hop in the chain.
4. The messages arrive to the premise firewall. Permitted traffic is allowed through.
5. The firewall permits the messages to reach the V10000 appliance in the DMZ.
6. The ESG processes the messages and matches them against the running policies.
Permitted messages flow to the email server.
7. The email server received the messages which passed the ESG filtering. The
messages are now available for pick/delivery by the end users.

ESGA v7.6 201 23

ESG Deployment

Slide 2:5

ESG Components
This slide represents a V10000 appliance installed in dual mode; both WSG and ESG
domains are running.
The various components operate as follows:
1. Admin

Administrators can access Appliance Manager by pointing the browser to the


IP address of the C interface on the appliance.

Administrators can configure the WSG domain through the same IP address
(different port).

Administrators need to login TRITON console running on an off-box server


to configure the ESG domain. The ESG domain communicates with TRITON
console through SSL interface.

2. TRITON Management Server

TRITON Management Server enables administrators to manage all of the


domains on the V10000 appliance: ESG, WSG, and Appliance Manager.

TRITON Management Server reads/writes SQL Database Server to perform


log and report functionalities.

3. ESG domain

24 ESGA v7.6 201

ESG Deployment

ESG domain sends and receives email messages on the E1 and E2 (if
configured) interfaces. You can use, in most cases the E1 interface for both
sending and receiving email. Alternatively, you can configure E1 to send and
receive on the LAN side and E2 to send and receive on the DMZ side.

ESG domain stores logs and statistics into SQL Database Server in order to
generate reports and dashboard charts.

ESGA v7.6 201 25

ESG Deployment

Slide 2:6

Clustering Deployment with Load Balancer


This network deployment shows load balancer with three ESG appliances connected
to it. The load balancer redirects mail traffic to different ESGs with certain traffic
balancing algorithm. The balancing algorithm is part of the load balancer
configuration and not ESG.
The flow of traffic is as follows:
1. The firewall receives the email from the Internet.
2. If the messages are permitted, the firewall forwards them to the IP address of the
load balancer.
3. The load balancer, using the specified algorithm, decides to which of the ESG
appliance send the mail message.
4. The ESG, which receives the email from the load balancer, processes the message.
Log information is stored about the message. The permitted message is forwarded
to the email server.
5. The email server receives the messages which passed the ESG filtering. The
messages are now available for pick/delivery by the end users.

Clustering details
Each ESG in a cluster has one of three roles:

26 ESGA v7.6 201

ESG Deployment

Primary

The primary node when working in cluster mode. Every cluster can have only
1 primary ESG. The primary is the only node in cluster to be configured via
TRITON console.

Administrators configure the policies on the primary node through TRITON


console. The configuration is then synchronized to all the secondary node.

Secondary

The secondary node when working in cluster mode.

Every cluster can have 1 to 7 secondary ESGs.

The secondary node can not be configured via TRITON console directly.

The secondary nodes synchronize the configuration from the primary node.

The status and logs are synchronized time to time as well.

The secondary nodes and the primary process mail traffic based on the same
policies.

Standalone

Any appliance that is not set as primary and not negotiated to be a secondary
stays in the role of standalone.

Important
One cluster has a maximum of 8 nodes.
All nodes of cluster must be the same appliance type: you
cannot mix V10000 with V5000 in the same cluster.
All nodes of cluster should have same applications
deployed on the appliance: all appliance are either single
mode (ESG only) or dual mode (ESG and WSG).
Tip
When cluster negotiation does not succeed, the primary
node will not become standalone automatically. The
administrator will need to change this manually.
Important
When you have more than one ESG appliances, clustering
is not a requirement. You can have all ESG appliances be
in standalone mode; however, you will have to configure
each ESG individually. This is not recommended and leads
easily to inconsistent policy settings.

ESGA v7.6 201 27

ESG Deployment

Slide 2:7

Clustering Deployment with MX Record


This scenario does not have a visible load balancer deployed. It loads and balances
email traffic on multiple ESG appliances using MX record.
Lets take the diagram as an example, in which you have three ESG appliances in your
cluster. You can re-configure your DNS as one of the following:

Configure the MX record in DNS to return three routable IP addresses which bind
to the external interface of the firewall. Each record should have the same
preference value, so that the traffic is evenly distributed on the basis of number of
incoming connection requests. The firewall maps the external IP addresses to the
internal one of the corresponding E1 interfaces on the ESG appliances (1-to-1
NAT)

Configure the MX record in DNS to return three routable IP addresses which bind
to each of the E2 interfaces of the ESG. The E2 interfaces are in the DMZ. The E1
interfaces are on the LAN side.

The flow of traffic is as follows


1. The sending email server queries the DNS MX record for the destination domain.

The DNS returns three IP addresses with the same preference value.

The order of the IP addresses are set to round robin mode.

28 ESGA v7.6 201

ESG Deployment

The email server connects to the first IP address in the list (the other addresses
are used if the first one does not respond).

The DNS returns the information about the next hop as well.

2. The email server connects to the next hop in the chain.


3. The messages arrive to the premise firewall. Permitted traffic is allowed through.
4. The firewall receives the incoming SMTP request and passes it to the
corresponding ESG appliance (using one of the two techniques listed above).
5. The layer 2 switch receives the connection request for the address of the selected
appliance. This appliance receives and processes the email message. If the policy
filter allows the email through, the appliance sends the email to the email server.
6. The email server receives the email that has passed filtering, which is now
available for pickup or delivery by the end use.

ESGA v7.6 201 29

ESG Deployment

Slide 2:8

Clustering Operating Mode


The only operating mode for the ESG cluster is Active/Active. All machines in the
cluster can process email at any time. The cluster is considered active and available if
at least one appliance is functional.
When the primary is down, all secondary nodes will keep working and filtering as
secondaries; no one will switch to the primary automatically; the primary needs to be
restored manually. Furthermore, you can NOT modify the policies because the
primary is the only node through which policies are configured and it is not available
at the moment.
If you need to make emergency changes to the policies, you can either re-configure
the cluster to pick up a secondary acting as the new primary via TRITON console, or
break the cluster and turn the secondaries to standalone machines.
As the configuration is synchronized among clusters nodes, the new cluster will work
in the same manner. The only lost is the email in the primary message queue, which
can not be recovered.
We will cover clustering with more details in the later chapter.

30 ESGA v7.6 201

ESG Deployment

Slide 2:9

Firewall Configuration
You need to configure your firewall as the slides on this and next page show.

Port 25: permit incoming and outgoing SMTP traffic on 25

On-box message processor listens on 25 to receive messages


On-box message processor sends messages to 25
Note
For Hybrid Service, the route of your email may
occasionally be altered within the Hosted Cluster. To
enable this without requiring you to make further changes,
you must allow SMTP requests from all the IP ranges
listed on the firewall page to port 25.

Port 80: permit outgoing traffic on 80

ESGA v7.6 201 31

ESG Deployment

On-box update module and feedback module connect to 80 listened by offbox DDS Server
Tip
DDS, developed by Websense, performs utilized
downloadable databases for various Websense products.
The databases include URL filtering database, real time
update database, real time security scanning database, etc.
Well talk about DDS again in ESG Architecture chapter.

Port 443: permit outgoing traffic on 443

On-box log, quarantine, PEM, and configuration components connect to


1443 listened by off-box MSSQL database

Port 2525: permit incoming traffic on 2525

On-box authentication component connects to 389 and 636 listened by offbox LDAP Server. 636 is the secure LDAP port.

Port 1443: permit outgoing traffic on 1443

On-box hybrid component connects to 443 listened by Hybrid Service

Port 389 and 636: permit outgoing traffic on 389 and 636

On-box DLP component connects to 80 listened by off-box TRITON DSS


Manager

On-box message processor listens on 2525 to receive encrypted messages


released by TRITON DSS Manager

Port 6671: permit incoming and outgoing traffic on 6671

32 ESGA v7.6 201

On-box SSL Proxy component listens on 6671 to communicate with offbox TRITON Unified Security Center
On-box PEM component connects to 6671 listened by SSL Proxy
component on another appliance

ESG Deployment

Slide 2:10

Firewall Configuration

Port 6643: permit incoming traffic on 6643

Port 9443: permit outgoing traffic on 9443

On-box DLP component listens on these ports

Port 5819, 5821, 8443, 56992, 18404: permit outgoing traffic

On-box PEM component listens on 9449 for end users to manage their
personal messages

Port 5820, 8888, 8889, 9080: permit incoming traffic

Off-box TRITON Unified Security Center listens on 9443


On-box DLP component connects to 9443 listened by off-box TRITON
DSS Manager

Port 9449: permit incoming traffic on 9449

On-box PEM Load Balance component listens on 6643 to dispatch PEM


traffic

On-box DLP component connects to these ports listened by off-box


TRITON DSS Manager

Port 50800: permit outgoing traffic on 50800

Off-box Log Server listens on 50800

ESGA v7.6 201 33

ESG Deployment

Slide 2:11

Sample Deployment
The slides shows two of our sample deployments:
A. ESG with Hybrid service
B. ESG without Hybrid service

34 ESGA v7.6 201

ESG Architecture

ESG is optimized to handle large volumes of email messages. It also integrates


seamlessly with the DLP engine with Websense Web Filter. The TRITON Console can
manage all the components. The internal architecture of ESG is very flexible to
achieve this level of scalability and integration.
This chapter focuses on the differences between the V10000 and V5000 appliances
relative to hardware configuration and software installation.

V10000:

ESG and WSG (dual mode)

ESG only (single mode)

WSG only (single mode)


Tip
The hardware resources on the V10000 are allocated
differently depending on which security mode you select.

V5000:

ESG only (single mode)

WSG only (single mode)

ESGA v7.6 201 35

ESG Architecture

Slide 3:1

Architecture and Message Flow


The ESG hardware settings (RAM, HDD, and CPU) can vary depending on whether it
is running on a V10000 in dual mode, a V10000 in single mode, or a V5000.
Consequently, the amount of traffic it can process depends on these parameters.
One of the strong points of ESG is its natural integration with the DLP module. This
integration is available for all installation options. You can also integrate ESG with the
Web Filter which requires dual mode (V10000 only) or an external server running the
Web Filter off-box.
This chapter covers the actual message processing or the lifecycle of a message within
the ESG module. Understanding these details enables you to better configure and
position the ESG solution.

36 ESGA v7.6 201

ESG Architecture

Slide 3:2

Appliance Modules
Both V10000 and V5000 run the Appliance Manager module, which is the domain to
configure hardware settings for appliances.
The difference between V10000 and V5000 is that a V10000 appliance can run ESG
and WSG modules at the same time, while a V5000 appliance can run either ESG or
WSG.
The WSG module consists of three sub-modules:
1. Websense Content Gateway (WCG): the web proxy component of WSG.
2. Web Security Suite (WSS): the content filtering component of WSG, also called
Web Security Enterprise (WSE).
3. Network Agent (NA): the component used to monitor and control protocols other
than HTTP, HTTPS, and FTP.
Except for the Appliance Manager module runs on Dom0 in V-Series appliance, all
other modules run on DomU.

ESGA v7.6 201 37

ESG Architecture

Slide 3:3

CPU Allocation
A V10000 G2 appliance can have three different configuration options when it has the
ESG module running:
1. ESG only
2. ESG plus WSG without Network Agent
3. ESG plus WSG with Network Agent running
In option 1, you have 12 CPUs dedicated to ESG. In options 2 and 3 you are limited to
four CPUs for ESG as the other eight are allocated to WSG. Depending on the volume
of web and email traffic, dual mode may or may not be the preferred option.
From a CPU standpoint, having Network Agent running or not has no impact on the
ESG module; it only affects the WSG module.

38 ESGA v7.6 201

ESG Architecture

Slide 3:4

Memory Allocation
Similar to CPU allocation, the amount of RAM assigned to each module depends from
the specific appliance configuration. When running in dual mode, the V10000
allocates the same amount of memory as the V5000 has.
The Network Agent configuration has no impact on the amount of memory allocated
to the ESG; it only affects the balance of memory allocated within the specific WSG
components.

ESGA v7.6 201 39

ESG Architecture

Slide 3:5

Hard Disk Allocation


The difference between single mode and dual mode is the amount of hard disk space
allocated to Mail Transfer Agent (MTA)
A V10000 running in dual mode allocates 70GB of HDD to MTA; while both the
V10000 single more and the V5000 allocate 100 GB of HDD to MTA.

40 ESGA v7.6 201

ESG Architecture

Slide 3:6

ESG Internal Architecture


The ESG has many internal components. Some of the most important ones appear in
this slide. The idea is to help you understand that each message that arrives to the ESG
flows through a series of components. At the end of the lifecycle, one of the following
will happen to the message:

Delivered to the intended recipient

Delivered to the intended recipient but modified (i.e. attachment stripped)

Quarantined

Dropped/Discarded

In addition to the messages processing components, the ESG needs to communicate


with the systems administrators, email users, and external components.
The ESG provides two management UIs for two types of users:

TRITON - Email Security, incorporated UI into TRITON architecture, running on


the off-box TRITON Manager Server, is for administrators to manage multiple
ESG appliances concurrently.

PEM, running on the ESG appliance, is for end users to manage their quarantined
emails.

ESGA v7.6 201 41

ESG Architecture

TRITON - Email Security and PEM communicate directly with the off-box database
to fetch log information.
Appliance Interface receives and processes the requests from Appliance Manager,
dom0 of the appliance.
SSL Interface is a SSL proxy which provides secure communication between on-box
module and off-box module. The ESG management UI and the on-box Config module
connect to SSL Interface.
Authentication provides four functions: maintaining different types of user groups,
authenticating message sender, message recipient, and end user to access PEM.
Config consists of a policy module and a cluster module. The policy module receives
configuration requirements from the ESG management UI and simple command line
from back end, and stores the configuration files on the appliance. The cluster module
synchronizes configuration files from primary ESG to secondary ESG in cluster
mode.
PEM loads and balances traffic to multiple PEM UIs, and reduce the stress for single
PEM UI. It also provides the single entry for all end users to manage quarantined
messages.
Quarantine isolates messages which failed in going through the antivirus engine or
antispam engine.
Alert provides multiple methods (dashboard, email, pop-up and SNMP) for system
services to report exceptions.
Log receives all kinds of logs from other on-box modules, stores the system logs and
audit logs to the on-box log database, and forwards the message logs to the off-box
Log Server.
PostgreSQL Database, the on-box log database server, saves system logs and audit
logs from the Log module.
The off-box Log Server runs on windows Operation System. It receives the message
logs from the on-box Log module save them to the database.
Hybrid provides the supports for the protected domain to register to Hybrid service
automatically, synchronizes the communication token and pulls hybrid logs to the log
database.
Policy Engine is the component installed on ESG which talks to the DSS Manager for
registration and receives the policy pushed out by the DSS Manager. DLP Filter
enables Policy Engine register to the DSS Manager, which is integrated with TRITON
Unified Security Center.
Scan Engine performs five types of filtering: hybrid, antivirus, antispam, disclaimer
and DLP.

42 ESGA v7.6 201

ESG Architecture

Slide 3:7

External Components
As mentioned before, ESG interacts with administrators, end users, and various offbox components. The slide displays the key components that are responsible for
communication between inside and outside ESG.
1. The Log Database and the Log Server are two different entities, which can be
installed on the same physical server.
2. The Encryption Service can be on-premises or on-demand in the cloud.
Currently the ESG supports integration with the Voltage encryption gateway. It
also supports natively TLS encryption. Well cover the encryption with more
details later in the specific chapter.
3. Messages go through a pre-filtering analysis where the email sender (IP address
and domain) runs through the Real-time Blackhole List Server (RBL Server)
lookup and the Websense Reputation Service Server (WRS Server). The
objective is to reduce the amount of messages that need to be analyzed early on in
the process. RBL Server and WRS Reputation Server can easily identify and
reject a large number of spam messages.
4. The ESG administrator needs connect to the ESG through the TRITON
Management Server, which runs on an off-box server. The TRITON
Management Server communicates with the SSL interface as described earlier.

ESGA v7.6 201 43

ESG Architecture

5. DDS Update Server is designed for utilized downloadable database within


Websense products. The database covers URL filtering database, real time update
database, real time security scanning database, license information database, etc.

44 ESGA v7.6 201

ESG Architecture

Slide 3:8

DLP Integration
The DLP Filter receives the messages for scanning in standard email format (RFC 822
compliant). The PE parses the message before scanning according to the policy.
Integrating DLP into ESG:
1. Register ESG with the DSS Manager and transfer DLP incident logs to the DSS
Manager.
2. Configure and deploy DLP policies, e.g. fingerprint, on the DSS Manager. The
policies are pushed out to the DLP Filter.
3. Configure DLP policies (active/disable/block/monitor) on ESG.
Your organization will leverage the TruEmail DLP capability with patented PreciseID
fingerprinting and natural language processing technologies.

Note
To let the DLP Filter work, you need to make sure ESG
subscription for DLP is valid; otherwise the message will
not be send to the DLP Filter for scanning.

ESGA v7.6 201 45

ESG Architecture

The DSS Manager itself needs a subscription for normal operations, which is an XML
file. However for the DLP Filter, the DSS subscription is not a prerequisite. That is,
even if no subscription is imported into the DSS Manager, or the DSS subscription is
expired, the ESG registered into DSS manager could still be able to make scanning. In
another word, the DLP scanning of ESG does NOT need the DSS subscription, but
needs the ESG subscription for DLP.

46 ESGA v7.6 201

ESG Architecture

Slide 3:9

Hybrid Integration
ESG hybrid mode allows the user to integrate the on-premise Email Security Gateway
with Websense Hosted Email Security (HES) service, to deliver easy deployment and
high-capacity email security. The workflow sequences are shown in the slide above.
HES service to each ESG box is based on hybrid account. All ESG hybrid
configuration will be sent to HES cluster and stored as the hybrid account settings.
The account ID is ESG license key, and the account password is transparent to the
customer and is maintained by ESG/HES internally. Whenever ESG talks to HES
Sync Server (HSS), the account and password is required for authentication.
If the hybrid mode is enabled, an HES account will be created.
HES can provide hourly refreshed statistical data in last week. The statistical data is
used to show the Value of Hybrid Mode on the dashboard reports.
The data includes:

Number of spam messages blocked in the cloud

Number of total messages processed in the cloud

Total size of blocked and processed messages

ESG will pull the data hourly from HES cluster.

ESGA v7.6 201 47

ESG Architecture

The inbound email traffic flows the sequence below:


1. Emails sending from external domains are supposed to reach the protected
domains. All messages go through the Hybrid service at first.
2. Filtered messages are sent to the ESG, where to be processed again.
3. Messages detected as spams, viruses, malwares, etc are discarded or isolated.
4. Allowed messages are sent to the email server for delivery.

The outbound email traffic flows the following steps:


A. Outbound messages are sent from the email server to the ESG.
B.

The ESG filters out the messages that do not comply with the policies.

C.

The messages that comply with policies are delivered to the intended recipient.

Note
In this scenario, Hybrid service filters the inbound traffic
only.

48 ESGA v7.6 201

ESG Architecture

Slide 3:10

Policy Processing
ESG listens on port 25, receives SMTP messages, and filters the message according to
its configuration.
If the message is not coming from a protected domain (inbound message), the ESG
first checks the sender IP address, domain, and other information against the RBL,
WRS and any custom Always Block List. This step already filters a majority of the
messages.
Next, the recipients are validated against the directory service. This is an important
step to prevent Directory Harvest Attacks (DHA).
The last three steps are the most typical of any email filtering product. The messages
is checked against size limits, spam content, virus content, and other malicious threats
(file types of the attachments, compressed files, password protected archives, etc.)
Outbound messages do not have to be checked against the RBL, WRS, and other
block lists; by definition, they come from trusted source.
However, outbound messages need be checked for possible viruses. While viruses
should not be there (most company run multilayer AV in their infrastructure), it is
good practice to ensure that you are not responsible for spreading an outbreak.

ESGA v7.6 201 49

ESG Architecture

More importantly, and mostly unique to the ESG solution, you implement DLP
policies for all outbound messages. The cost of information leakage, both financially
and in terms of a tarnished reputation, can be devastating.

50 ESGA v7.6 201

ESG Architecture

Slide 3:11

Message Processing Flow


The incoming messages (inbound or outbound alike) are received by the SMTP
Server, which listens for incoming connections in TCP port 25.
SMTP Server is in charge of dropping as many as possible of the incoming messages
that maybe threats, which is called pre-filtering analysis. During this process, the
SMTP service uses multiple techniques, such as Real-time Blackhole List (RBL)
lookup, Reverse DNS (RDNS) lookup, Websense Reputation Service (WRS), etc to
perform the analysis fast and accurately. Moreover, it greatly reduces the number of
messages that need further analysis.
Filter Server receives the messages after the SMTP Server processing them,
facilitates the Policy Engine and the Scan Engine to filter out the additional
incompatible emails based on the policies and filters set by administrators. This is the
content filtering, with the goal of only delivering messages that are appropriate in
content and do not have any threats in them.
At the end of the message processing, three outcomes are possible for the messages:
1. Dropped/Discarded: The message is deemed to be a serious threat. It cannot and
should not be ever released to the recipient. This is a not recoverable action.
2. Delivered to the recipient: The message passed all the security and policy
scanning. It is appropriate in content and does not have any other threats.

ESGA v7.6 201 51

ESG Architecture

3. Quarantined: The message did fail one or more of the security and policy
scanning. However the content is not such that is should not ever be released to
the recipient, but may need to be retrieved by administrators or end users.
Quarantined messages are stored in the local (ESG file system) quarantine queue.
Information about quarantine is stored in the log database.
Tip
Recipients of a quarantined message can retrieve it through
PEM management UI.

52 ESGA v7.6 201

ESG Architecture

Slide 3:12

Mail Filtering Framework


ESG adopts 4 queues when processing the messages:

Filter queue
The messages that pass through pre-filtering will be placed into priority based
filter queues.
If quarantined messages are to be re-processed (scanned again), the messages will
be put back to filter queue.

Incoming queue
The messages processed by Filter Daemon are moved from filter queue to
incoming queue.
Besides, the messages that should not be filtered (bounce/delivery request from
PEM) are put in the incoming queue.

Deferred queue
If a message failed in being delivered for some transient reasons (it might succeed
later), the message is placed in the deferred queue.

Active queue
The messages in the active queue will be sent to SMTP Client and delivered.

The messages are processed in the following procedure within ESG:

ESGA v7.6 201 53

ESG Architecture

1. First layer of filtering is what we called pre-filtering, handled by SMTP Server.


Pre-filtering rejects most zombie connections. Clean Up Filter inserts the
permitted messages into priority based filter queues according to the scores and
informs Filter Daemon.
2. Filter Daemon asks Policy Server about the policies of the message, call the
related Scan Engines, and execute the configured actions. After filter daemon
scans the message, the message is moved from filter queue to incoming queue.
3. The message that fail in filtering is dropped or quarantined.
4. The acceptable message is sent to Queue Manager. Queue Manager moves the
message from incoming/deferred queue to active queue. After a message is
scheduled, Queue Manager informs SMTP Client with meta info of the message.
Queue Manager scans the deferred queue periodically, brings messages in
deferred queue to active queue and schedules the messages according to system
resource.
After the messages have been tried for max times, the messages will be bounced
to sender. If the first bounce failed, the message will be bounced to configured
postmaster address.
5. SMTP Client gets the message from active queue and deliver the message, and
replies the status of the message to Queue Manager after delivering.

54 ESGA v7.6 201

ESG Architecture

Slide 3:13

RBL and WRS


The slides shows the detail processing of Real-time Blackhole List (RBL) check and
Websense Reputation Service (WRS) lookup. The IP address and domain of the email
sender go through the external RBL Server at first then the WRS Server and get an
reputation score. Based on different scores, messages will be placed into priority
based filter queues.

ESGA v7.6 201 55

ESG Architecture

56 ESGA v7.6 201

Personal Email Manager

Personal Email Manager (PEM) is a component of Websense Email Security.


PEM enables a user to look at isolated spam messages and decide whether to delete
the messages or treat them as legitimate email messages. It also identifies the IP
address of the appliance on which an end user can access PEM. You can also authorize
an end user to manage personal Always Block List and Always Permit List, configure
the appearance of the PEM end user display, and deploy appliances in groups for load
balancing purposes.
PEM monitors specially configured Websense Email Security queues and notifies
users if they have blocked inbound email. It also provides end-users a facility for
managing their blocked email. To achieve the desired behavior and results, PEM must
be administered on an ongoing basis. These tasks include:

Configuring queues

Customizing the notification email templates

Language support
PEM includes adaptive language support for Spanish, Portuguese, Italian, and
German.
When an end-user or administrator logs into the PEM facility, they can determine the
language displayed in the user interface.

ESGA v7.6 201 57

Personal Email Manager

Slide 4:1

PEM Key Features


PEM is a built-in feature of the ESG appliance. There is no need for the administrator
to install additional software or hardware.
The PEM module is fully supported in a clustered environment. In a clustered
environment, PEM can be accessed through the primary ESG or the secondary ESG. If
the primary ESG is not available, users can also view their isolated message list
through the secondary ESG. However, isolated messages stored on the primary ESG
cannot be viewed, delivered, or deleted.
The PEM can be successfully implemented and used any environment regardless of
size. To support even greater scalability, the ESG can store the isolated messages on a
remote file system. ESG can connect to a NFS share or a CIFS one (via SAMBA). In
environment where more than ESG exists, even if all appliances use the same share,
each ESG still creates it own separate and independent queues.
One of the key benefits of the PEM is greatly reduced administrator load; users are in
charge of setting their own Always Permit List and Always Block List and release
messages which may have been isolated by mistake. With tight and granular
administrator control, only messages that do not present a threat to the company can
be available for user release.

58 ESGA v7.6 201

Personal Email Manager

Slide 4:2

Quarantine System Overview


The ESG process the messages; when it finds that an email is a spam, virus, or the
message failed to be processed, it could not be encrypted or could not be decrypted the
ESG informs the quarantine daemon to quarantine the email. The email can be
quarantined locally or on a remote storage mounted on ESG through NFS or SAMBA
(CIFS).
Administrators and end users can view and manage the quarantined emails and the
quarantine daemon will record corresponding quarantine logs to ESG log database.
Quarantine logs are sent directly to log database without being relayed by the ESG log
server. These logs need to be sent out one by one instead of in bulk for high reliability
requirement.

ESGA v7.6 201 59

Personal Email Manager

Slide 4:3

Mail Queues
Quarantine messages are isolated into different quarantine queues (folders) in terms of
their different attributes. By default, there are six built-in queues:
1. Virus: Used to store messages, which contain a virus.
2. Spam: Used to store messages, which triggered one of the spam filters.
3. Exception: Used to store exceptional emails (failed several non specific filters).
4. Encryption-fail: Used to store messages that did not encrypt correctly.
5. Decryption-fail: Used to store messages that did not decrypt successfully.
6. Archive: Used to store all incoming messages.
Tip
These 6 queues cannot be deleted. However, user can edit
their properties or add new queues.
As mentioned before, messages in each queue can be stored locally (on ESG box) or
on a remote storage. ESG supports two remote storage types:

NFS

CIFS (through Samba)

60 ESGA v7.6 201

Personal Email Manager

On the Blocked Message page, users can see the quarantine queue list.

ESGA v7.6 201 61

Personal Email Manager

Slide 4:4

Queues Monitor
Every 30 seconds, ESG will perform a checking on queue status, including remote
queue availability and available disk space check.

If the remote queue is found unavailable, the queue storage status will be marked
as "mounted error", and ESG will generate a corresponding alert. If the remote
queue is available again, the "mounted error" status will be cleared as "ok" and the
alert will be cleared too.

If the remote storage is unavailable, trying to move the queue with copying the
messages will fail.

If the disk is available, ESG will check the space usage of the queue. If the
percentage of free disk space is less than 10%, ESG will mark the queue status as
"disk space low" and send an alert. Also, if a queue's free disk space is return to
more than 10%, the status will be cleared and alert will also be cleared.

62 ESGA v7.6 201

Personal Email Manager

Slide 4:5

Queues Configuration
In cluster mode, the "Queue list" page is similar to standalone mode. The "volume" is
the sum of all machines' volumes, and the "size/total" is the sum of all machines' size
and size limit. For example, there a 2 machines in a cluster, both of which has a 10GB
size limit (Their limits are always same because the configuration is synced between
cluster).
Machine 1 has 10 files in it and its size is 5MB, machine 2 has 5 files in it and its size
is 3MB. In this case, the "volume" and "size/total" displayed in the UI will be "15" and
"8MB/20GB".
Compared with standalone mode, the error status description of the queue is more
complex. In standalone mode, if the hard disk space of the queue is low, there will be
an icon displayed which shows "hard disk low". In cluster mode, if the storage type is
locally, the content displayed will be a list of "Device X.X.X.X: hard disk low",
because maybe more than 1 device has no enough free disk space; if the storage type
is remote, the content displayed will be "Remote hard disk low".
Regarding the "remote storage invalid" status, if a queue of at least one machine is in
this status, the WebUI should show "remote storage invalid".

ESGA v7.6 201 63

Personal Email Manager

Slide 4:6

PEM Components
PEM is composed by two sides: the admin side and the end user side:

Admin side - PEM general configuration: the PEM digest settings, the end-user
access control and the user authentication mode.

End user side - PEM management: the end user can manage the messages
processed by ESG and take the actions allowed he/she prefer.

64 ESGA v7.6 201

Personal Email Manager

Slide 4:7

Notification Message
The ESG PEM enables end users to configure the notification message that will be
sent out from the ESG when any of the emails, with the recipients or senders as
themselves, are blocked.
End users are able to configure the notification message in the following aspects:

Max messages
The maximum number of the message digests in the notification email to the end
user. Such as, if set to 10, then, only 10 message digests will be in the notification
message even if the end user has more than 10 quarantine or spam messages. The
maximum value of this setting is 100 and the default is 10.

Digest send interval


This interval decides when the digest message will be sent out according to a time
and day setting. Times are set by hour and minute while days can be

Every day

Every weekday

A specific day in Every Week


For example, 3:00 Monday, Every week or 6:00, 12:00, 20:00, Every day.

Digest message template


Message operation options define the operations which will be included in the
digest message and the end user can see and execute.

ESGA v7.6 201 65

Personal Email Manager

The slide show the screen when setting the notification message:
1. Set the max number of messages per notification and the actions available in the
notification that the end user are able to take.
2. Configure the information of the notification:

Company
Define the brief info to the customer company displayed in the digest message

Description
Define the brief info to the ESG which the customer prefers.

Sender
Define the sender email address of the digest notification message to the end
user. The address format is username@domaininfo, such as
postmaster@websense.com, meaning that the sender of the digest message
will be this address. This is ONLY one email address.

Subject
Define the subject of the digest message. The maximum length of this value is
100 characters.

3. Set the header


The header info will be displayed in the digest message. It can provide some key
words to the end user before he/she open the message text.
4. Set the footer
The footer info can provide information that the end user can use, like, the contact
telephone number of email address when the end user gets a problem.
The end-user access list allows the ESG administrator to define the user list who can
use the PEM to manage the quarantine or spam messages. The user account will only
be chosen from the existed "User Directory" objects. If the ESG wants to use one new
user directory, first, it's required to add it in the page at which the user directory can be
defined. This setting will list all the existed un selected user directorires in the left
compound box and all the checked user directories in the right box.

66 ESGA v7.6 201

Personal Email Manager

Slide 4:8

Queues Actions
Quarantine message logs will be stored on a remote SQL server database, together
with ESG message logs.
By clicking the queue name in "Queue list" page, user can view all messages of the
queue. User can view the messages by date, and can also search messages by sender,
recipients, subject and policy.
User can delete, download, reprocess, continue reprocess, deliver (release) the
message(s) or forward the message(s) to one or more email addresses. User can also
report a false-positive to Websense, or add the sender of a message into ESG global
Always Permit/Block List.
Sometimes when a queue operation is ongoing, some of the message operations will
return a failure of "retry later".
The following actions are supported:

Delete: Delete the message

Deliver: Deliver the message to its recipients

Reprocess: Delete the message from the queue and reprocess it, just like it has
been received by ESG again. It's useful when sometimes technical support team
and development team want to debug false-positives of filter. They can turn on the
debug, reprocess the message and find why this message is captured by filter.

ESGA v7.6 201 67

Personal Email Manager

Forward: Forward the message to one or more recipients, the forwarded message
will be embedded in the forward mail as an attachment. The subject of the forward
mail is "Fw: Forward Message(s)".

Add to Always Block List: Will add the message sender to Always Block List.

Add to Always Permit List: Will add the message sender to Always Permit List.

Download: Download the message in .eml format. If more than one message is
downloaded together, they will be compressed in a zip file.

Clear queue: Clear the whole queue, all messages in the queue will be deleted.

Continue Process: Sometimes a message contains both virus and spam info.
Assume it's captured by spam filter and quarantined, then administrator finds that
it's a false-positive and decides to deliver (release) it. In this case, the message will
bypass the virus scanning. To prevent this, there is another action "Continue
Process" provided to make sure the message will be scanned by other filters
before it is delivered.

Not Spam: Clicking this button will cause two actions: to send a copy of this email
to esg-fp@websense.com for false positive research, then release the email to its
original recipients.

68 ESGA v7.6 201

Personal Email Manager

Slide 4:9

PEM in Clustered ESG


In a cluster environment there is always one and only one appliance which operates as
the primary.
For the purpose of the PEM notifications, only the primary sends out the notification.
In the notification, sent to the user, the IP address (or hostname) for the isolated mail
link is the primary's IP (or hostname). If the primary is not available (down), the users
do not receive notifications and they cannot retrieve isolated emails. The secondary
will not send notification and will not participate in any election process. Once the
primary returns operational, the users receive once again the notification and are able
to manage the isolated messages.
Similarly to the queue management, also the login is only available through the
primary.
During normal operations, the process of retrieving isolated messages via PEM
proceeds as follows:
1. ESG #1 (not the primary) receives a message, which is going to be isolated.
2. The ESG writes an entry in the log about this message. As mentioned before, a
key field in the database is the ID of the ESG who isolated the message.

ESGA v7.6 201 69

Personal Email Manager

3. The recipient, which has been notified that she has received an message being
isolated, connects to the PEM module. The link in the notification points to the
TRITON - Email Security.
4. The primary ESG looks up the message details in the log database. Using the ID
of the machine, which isolated the message, can connect to that ESG (in this case
ESG #1)
5. The primary ESG communicates with the ESG #1, retrieves the message, and
sends it over to the user who requested it.

70 ESGA v7.6 201

Policy Management

Policy is the pipeline that message flows go through. ESG defines 3 types of policies
based on traffic directions:

Inbound policy

Outbound policy

Internal policy

Inbound/Outbound/Internal policy have multiple sub-policies which will applies to


the messages that match the policys matching condition. The matching condition
defines the entry criteria of the policy pipeline, based on the envelope sender and
recipient address of the messages.
Each policy can contain a series of rules. Rules define how the message in the policy
pipeline is to be scanned and how the message is processed when it triggers the filters.
Each rule is combined with action and filter. Filter is individual scanning module
which defines how the message in the policy pipeline is to be scanned. Action is
individual module which defines how to process message when filter is matched.

How the ESG policy works could be described in the following steps:
1. Messages are divided into 3 categories: inbound, outbound, and internal.
2. In each category, messages match the condition based on the addresses of the
senders and recipients respectively.
3. The specific filter, defined in the rule of the matched policy, scans the messages.
4. The messages filtered out are processed according to the action.

ESGA v7.6 201 71

Policy Management

Slide 5:1

Key Features
Policy rules comprise the filters and filter actions that determine how a message that
matches a policys sender/recipient conditions is handled.
Filters provide the basis for email scanning for viruses and spam, and filter actions
determine the final disposition of a message when it triggers a particular filter. Define
new filters, or copy and edit an existing filter to suit your organizations needs.
Add and define new actions or customize default actions as needed. After you have
created and configured filters and filter actions, they are available for inclusion in your
policies.

72 ESGA v7.6 201

Policy Management

Slide 5:2

Policy Components Flow


1. Policies are directional-based and depend on the following factors:

Protected domain

Email sender address

Email recipient address

2. Policies contain conditions; when matched, rules are applied.


3. Rules contain both filters and actions:

Filters are individual scanning modules which define how messages in the
policy pipeline are scanned.

Actions are individual modules which define how a message is processed.


Note
Policy order is important. ESG policies are similar to
firewall rules. If one policy is matched, then the rest will
be skipped.

In ESG, there can be up to 32 policies per mail route direction.

ESGA v7.6 201 73

Policy Management

Slide 5:3

Policy Directions
An Email Security Gateway policy is applied based on defined sender/recipient
conditions and the direction of the email. You can apply a different policy to different
groups of senders and recipients. For example, you might apply a policy to a
marketing department group in your organization and a different policy to a human
resources group. After you define a set of senders and recipients in a policy, you can
add the policy rules (a combination of filter and filter action) to apply when the
sender/recipient conditions of the email match the policy. Email Security has 3 general
types of policies, depending on the direction of the email inbound, outbound, or
internal.
Message direction is determined on the basis of an organizations protected domains:

Inbound - The sender address is not from a protected domain, and the recipient
address is in a protected domain

Outbound - The sender address is from a protected domain, and the recipient
address is not in a protected domain

Internal - Both the sender and recipient addresses are in a protected domain.

ESG has one predefined default policy for each email direction, as well as a default
Data Security policy for each direction. Data Security policies may be applied to email
in any direction. These policies are configured in the Data Security module of
TRITON Unified Security Center and can only be enabled or disabled in ESG. You

74 ESGA v7.6 201

Policy Management

need to register Email Security with the Data Security management server and click
Deploy in the Data Security module for the policies to be active.

The inbound policy logic is as follows:


1. Pre-filtering/Reputation
2. Always Permit/Block List
3. Antivirus (sender/recipient/filter-option/action)
4. Antispam (sender/recipient/filter-option/action)
5. Content Guardian (sender/recipient/filters/action)
6. DLP

The outbound policy logic is as follows:


1. Antivirus (sender/recipient/filter-option/action)
2. Antispam (sender/recipient/filter-option/action)
3. DLP

The internal policy logic is as follows:


1. Antivirus (sender/recipient/filter-option/action)
2. Antispam (sender/recipient/filter-option/action)
3. DLP or bypass DLP

ESGA v7.6 201 75

Policy Management

Slide 5:4

Policy Conditions
Match conditions define the entry criteria of a policy.
Each match condition consists of a pair of sender and recipient envelope addresses,
marked as From address and To address. The address can be a simple address like
john@company.com, or can be a wildcard address *@somewhere.com, or can be just
* for any address. When the envelope sender address of the message matches the
From address of a match condition and the recipient address matches To address of a
match condition, then this match condition is considered matched and the message
goes into this policy.
If the message has multiple envelope recipient addresses, it is possible that a message
is split into different policies. The address in match condition can also be from certain
User Directory, which is logic group of addresses. A policy can have multiple match
conditions, and messages will try to match these match conditions one by one, from
top to bottom.
The order of match conditions in policy is not important. When match conditions are
empty, the policy will be skipped directly.
There are at most 32 match conditions per policy.

76 ESGA v7.6 201

Policy Management

Slide 5:5

Rules
Rules define message scanning process of policy. If the sender and the recipients of a
message match any match condition of a policy, then messages will try to match rules
one by one, from top to bottom. If one rule is matched, the rest will be skipped. Rules
cannot be deleted/added, but can be edited.
Rules contain the following:

General configuration, including Name, Enable/Disable, Order

Filters: scanning modules to use

Actions: what action to take if a filter scan is matched

There are 3 default rules

Antivirus

Antispam

Disclaimer

ESGA v7.6 201 77

Policy Management

Slide 5:6

Filters
Email Security Gateway has three predefined default filter types: virus, spam, and
disclaimer.
The virus filter analyzes an email message and its attachments for the presence of
viruses and other threats. The spam filter scans email content and compares it against
a database of known spam characteristics. You can select from a variety of antispam
tools, including the Websense Web Security URL scanning tool. If you want to add
text at the beginning or end of a message, use the disclaimer filter.
Filters are created and managed via the Main > Policy Management > Filters page.
Click Add to open the Add Filter page and set the properties of your new filter.
You can also copy a filter whether or not it is in use by a policy. A filter can be deleted,
as long as it is not in use by any policy. However, you cannot copy or delete an Email
Security default filter.
Up to 32 filters can be assigned per policy.

78 ESGA v7.6 201

Policy Management

Slide 5:7

Actions
A filter action determines a messages final disposition. ESG scans messages and their
attachments then performs an action based on applicable policy settings. Actions are
created in the Main > Policy Management > Actions page. You can add a defined
action to a policy rule when you configure your email policies.
Create a new filter action by clicking Add and selecting action properties.
You can remove a filter action by marking the check box to the left of the filter name
to select it and clicking Delete. You can delete a filter action only if its current status is
Not in use. A filter action that is in use does not have a check box for selection.
Note
You cannot remove an Email Security default filter action.
There are at most 32 actions per policy.

ESGA v7.6 201 79

Policy Management

Slide 5:8

Summary
1. All traffics are divided into 3 directions: inbound, outbound, and internal.
2. Messages match the specific conditions.
3. Filters are triggered for the messages of the conditions.
4. Actions are performed if the filters are matched.

80 ESGA v7.6 201

Advanced Encryption

ESG encryption enables the secure delivery of sensitive email communications to


business partners and individuals. Easy administration, no complex key management,
no additional hardware or software requirements eliminate the traditional cost and
complexity barriers to implementing email encryption.

The ESG provides 3 types of message encryption:

Transport Layer Security (TLS): to encrypt/decrypt emails by ESG locally

Hybrid service encryption: to encrypt/decrypt emails by Hybrid service

Third party encryption: to encrypt/decrypt emails by integrated third party


encryption gateway

ESGA v7.6 201 81

Advanced Encryption

Slide 6:1

Advanced Email Encryption


Email Security supports the following 3 types of message encryption:

Transport Layer Security (TLS)

Hybrid service encryption

Third party encryption gateway

Well talk about the 3 encryptions in detail!

82 ESGA v7.6 201

Advanced Encryption

Slide 6:2

TLS introduction
Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL), are
cryptographic protocols that provide communications security over the Internet. TLS
and SSL encrypt the segments of network connections above the Transport Layer,
using symmetric cryptography for privacy and a keyed message authentication code
for message reliability.
Several versions of the protocols are in widespread use in applications such as web
browsing, electronic mail, Internet faxing, instant messaging and voice-over-IP
(VoIP).
TLS is an IETF standards track protocol, last updated in RFC 5246 and is based on the
earlier SSL specifications developed by Netscape Corporation.

ESGA v7.6 201 83

Advanced Encryption

Slide 6:3

TLS Encryption
TLS is an Internet protocol that provides security for all email transmissions
inbound, outbound, and internal. The client and server negotiate a secure connection
for the transmission to occur, provided both the client and the server support the same
version of TLS. The ESG uses a TLS encryption level of 128 bits.
The ESG uses 2 levels of TLS: Opportunistic TLS and Mandatory TLS.

Opportunistic TLS
It is used during the message routing process. You can enable TLS for a specified
message route on the Settings > Receive/Send > Mail Routing page. Create a new
route that uses the TLS delivery option or edit an existing mail route to add the TLS
option. At the bottom of the page, mark the Use Transport Layer Security (TLS) check
box to use opportunistic TLS for message routing.
With opportunistic TLS, the protocol must be implemented on both sides of a
connection for an encrypted transfer. If a connection attempt is made using the TLS
protocol, the connection recipient must provide appropriate TLS credentials for an
encrypted data transfer. If the TLS handshake fails, the data transfer is made via
plain text, rather than encrypted text. In either case, the data transfer is successfully
accomplished.

84 ESGA v7.6 201

Advanced Encryption

Mandatory TLS
Mandatory TLS is the default encryption method that is enabled on the Settings >
Inbound/Outbound > Encryption page.
For mandatory TLS, both sides of an email connection must use the protocol. An
encrypted data transfer occurs when the TLS handshake process is successful. Unlike
opportunistic TLS, if the handshake fails during the connection attempt, the
connection is terminated and no data transfer occurs.

A. If the next hop MTA has the ability of TLS, the encrypted email will be sent out.
B. Mandatory TLS: if the next hot MTA does not support TLS, the email is placed in a
deferred message queue for a later delivery attempt
B. Opportunistic TLS: if the client and server cannot negotiate a secure connection,
the message will be sent to the recipient.

ESGA v7.6 201 85

Advanced Encryption

Slide 6:4

Prerequisite of Hybrid Service Encryption


Prerequisites of Hybrid service encryption
A. Hybrid service encryption is available only if your subscription includes the Hybrid
service and the Hybrid service is registered and enabled. Otherwise, you can not select
Hybrid service encryption from TRITON - Email Security.
B. Hybrid service encryption requires a certificate from a hybrid service trusted CA.
In order to obtain a certificate from one of these CAs, you need Generating a
Certificate Signing Request (CSR) for the server on which you want to install the
certificate. After you have generated a CSR, follow the third-party CA acquisition
procedures for the certificate you want to purchase. Use the Settings > General >
TLS Certificate page for certificate import.

Generating a Certificate Signing Request (CSR)


You can generate a CSR using the following procedure:
1. Open the command-line interface on the V-Series appliance where Email Security
resides.

86 ESGA v7.6 201

Advanced Encryption

2. Execute the following commands:


openssl genrsa -out [keyfile] 1024
Where [keyfile] is used to import the certificate to the ESG.
openssl req -new -out reqfile -key keyfile subj /C=CN/
ST=Bj/L=Bj/O=Websense/CN=ESG\/
emailAddress=support@websense.com/
Where [keyfile] is the CSR file and /C=CN/ST=Bj/L=Bj/O=Websense/
CN=ESG\/emailAddress=support@websense. com/ is the subject of the
certificate.
The certificate subject can be modified if necessary.

List of trusted CAs for hybrid service


The Email Security Gateway hybrid service can use a certificate from any one of the
following trusted CAs:

Entrust.net Premium 2048 Secure Server CA

Entrust.net Secure Personal CA

Entrust.net Secure Server CA

Equifax Premium CA

Equifax Secure CA

GTE CyberTrust Global Root

GTE CyberTrust Root 2

GTE CyberTrust Root 3

GTE CyberTrust Root 4

GTE CyberTrust Root 5

GTE CyberTrust Root CA

GlobalSign Partners CA

GlobalSign Primary Class 1 CA

GlobalSign Primary Class 2 CA

GlobalSign Primary Class 3 CA

GlobalSign Root CA

Thawte Personal Basic CA

Thawte Personal Freemail CA

Thawte Personal Premium CA

Thawte Premium Server CA

Thawte Server CA

Thawte Universal CA Root

ValiCert Class 1 VA

ValiCert Class 2 VA

ValiCert Class 3 VA

ESGA v7.6 201 87

Advanced Encryption

VeriSign Class 4 Primary CA

Verisign Class 1 Public Primary Certification Authority

Verisign Class 1 Public Primary Certification Authority - G2

Verisign Class 1 Public Primary Certification Authority - G3

Verisign Class 2 Public Primary Certification Authority

Verisign Class 2 Public Primary Certification Authority - G2

Verisign Class 2 Public Primary Certification Authority - G3

Verisign Class 3 Public Primary Certification Authority

Verisign Class 3 Public Primary Certification Authority - G2

Verisign Class 3 Public Primary Certification Authority - G3

Verisign Class 4 Public Primary Certification Authority - G2

Verisign Class 4 Public Primary Certification Authority - G3

Verisign/RSA Commercial CA

Verisign/RSA Secure Server CA

88 ESGA v7.6 201

Advanced Encryption

Slide 6:5

Hybrid Service Encryption for Outbound Email


Hybrid service encryption is enabled on the Settings > Inbound/Outbound >
Encryption page.
Note
Hybrid service encryption is mandatory encryption. If the
encryption is failed, the message will be placed in a
deferred message queue for a later delivery attempt.
For mandatory TLS, both sides of an email connection must use the protocol. An
encrypted data transfer occurs when the TLS handshake process is successful. Unlike
opportunistic TLS, if the handshake fails during the connection attempt, the
connection is terminated and no data transfer occurs.
Hybrid service does not encrypt inbound or internal messages. Even though Data
Security policy identifies an inbound or internal message for encryption and Hybrid
service encryption is selected, the message will not be encrypted and an alert (HESencryption for inbound/internal messages) will be sent. Therefore Data Security
policy must be modified to designate only outbound messages for encryption when
Hybrid service is used.
Hybrid service encryption for the outbound email following the below steps:

ESGA v7.6 201 89

Advanced Encryption

1. The Email Server sends out an unencrypted email to the ESG.


2. Data Security policy labels the email for encryption.
3. The ESG sends the email to the Hybrid Filter for encryption on SMTP forced
TLS.
4. The Hybrid Filter encrypts the email and delivers the email to the recipients. The
Hybrid Filter address is configured during the Hybrid service setup process.

The Hybrid Filter identifies the message by

IP address of ESG

Sender domain

Encryption x-header: indicating the message should be encrypted

TLS state

If the message contains X-ESG-HYBRID header, but fails in any of the other 3
criterions, the Hybrid Filter will bounce the message. IP addresses of ESG are pushed
to the Hybrid Filter during Hybrid configuration; and later changes of IP addresses or
network topology are notified to the Hybrid Filter too.
If encryption fails, the message is placed in a deferred message queue for a later
delivery attempt.
If the Hybrid Filter detects a spam or virus in an encrypted outbound message, the
mail is bounced to the message sender.
Bounce means the Hybrid Filter returns a 5.x.x permanent failure to ESG on SMTP
conversation; or generates an NDR message (with return of content) depending on the
original message and delivers it to ESG on SMTP opportunistic TLS. ESG should
always advertise STARTTLS to the Hybrid Filter and accept TLS handshake from the
Hybrid Filter.
Messages sent from ESG to the Hybrid Filter for encryption may be put in the Hybrid
Filter queue for some time.

90 ESGA v7.6 201

Advanced Encryption

Slide 6:6

Hybrid Service Encryption for Inbound Email


For inbound messages, Hybrid service encryption attempts to decrypt inbound
encrypted mails.
The detailed process is described on the slide:
1. An encrypted email arrives at the Hybrid Filter.
2. The Hybrid Filter decrypts the email and adds an x-header to the email to indicate
whether the decryption succeeded, and scans the email and sends the permitted
unencrypted email to the ESG.
Note
Message scanning is performed regardless of whether
message decryption is successful.
3. The email may trigger DSS incidents.
4. The ESG scans and delivers the permitted unencrypted email to the recipients.
The Hybrid Filter will deliver the messages to the ESG on SMTP opportunistic TLS
with the following x-headers:

Decryption success/fail x-header, indicating decryption successful or failed

Virus x-header

ESGA v7.6 201 91

Advanced Encryption

Spam x-header

The ESG always advertise STARTTLS to the Hybrid Filter and accept TLS handshake
from the Hybrid Filter.

92 ESGA v7.6 201

Advanced Encryption

Slide 6:7

Third Party Encryption On-demand for Outbound Email


Third party encryption is enabled on the Settings > Inbound/Outbound >
Encryption page.
Note
Third-party encryption is opportunistic encryption. If the
encryption failed, the message will be sent to the recipient
in plain text.
When Data Security policy identifies an outbound message for encryption and third
party encryption is selected, the messages are sent to third party encryption gateway.
The process is described on the slide:
1. The Email Server sends out an unencrypted email to the ESG.
2. Data Security policy labels the email for encryption.
3. The ESG sends the email to third party encryption gateway for encryption on
SMTP opportunistic TLS, for third party gateway may not enable TLS to receive
messages.
4. Third party gateway encrypts the message and delivers it to the recipients directly.

ESGA v7.6 201 93

Advanced Encryption

Administrators configure x-headers to add to an outbound message via TRITON Email Security. These x-headers facilitate communications between the ESG
appliance and third party gateway and they must match the corresponding settings in
third party gateway.

Encryption x-header, indicating the message should be encrypted

Encryption success x-header, indicating the encryption is successful

Encryption fail x-header, indicating the encryption is failed

Administrators can require users to present credentials to view encrypted mails:


supply the desired user name and password in TRITON - Email Security.
Authentication must be supported and configured on third party gateway as well.
Besides x-header and credential configuration, administrators are able to set desired
encryption failure options:

Isolate messages to queue

Send notification to original sender

Deliver messages

Drop messages

If third party encryption gateway is not configured, ESG will generate an alert (invalid
third party encryption gateway).

94 ESGA v7.6 201

Advanced Encryption

Slide 6:8

Third Party Encryption On-premises for Outbound Email


The difference between on premises and on demand is that in this case, third party
encryption gateway routes the encrypted messages back to ESG instead of delivering
the messages out. ESG identifies the messages from third party gateways by
configured IP addresses of the gateway.
The sequence is listed on the slide:
1. The Email Server sends out an unencrypted email to the ESG.
2. Data Security policy labels the email for encryption.
3. The ESG sends the email to third party encryption gateway for encryption.
If encryption is successful, the third party gateway forwards the message (possible
multiple messages from multiple recipients) to ESG with encryption success xheader.
4. ESG will deliver the encrypted message without further scanning.
If encryption is failed, the third party gateway forwards the original message with
encryption fail x-header to ESG. ESG will apply the encryption fail configuration,
quarantine the message, notify senders, deliver or drop the message. An alert (third
party encryption gateway encrypt fail) will be generated.

ESGA v7.6 201 95

Advanced Encryption

If there is no encryption success/fail x-header, there may be errors in configuring ESG


and third party encryption gateway. An alert (third party encryption gateway does not
encrypt messages) will be generated.

96 ESGA v7.6 201

Advanced Encryption

Slide 6:9

Third Party Encryption On-premises for Inbound Email


In order to decrypt messages, third party encryption gateway should point their mail
route back to ESG and the decryption option is enabled on ESG configuration.
Decryption option can only be enabled when third party encryption gateway is
selected and IP address of third party gateway is configured.
The slide shows the process of third party encryption for inbound email:
1. The encrypted email arrives at the ESG.
ESG checks if the inbound email are encrypted- those with content-type or xheaders are encrypted messages, which will be send to third party encryption
gateway on SMTP opportunistic TLS to decrypt.
2. Third party gateway decrypts the email and routes it back to the ESG with
decryption success x-header.
3. The email may trigger DSS incidents.
4. The ESG scans and delivers the permitted unencrypted email to the recipients.
If decryption is failed, third party gateway will forward the original message with
decryption fail x-header back to ESG. ESG will apply the decryption fail
configuration: to quarantine the message, then scan the message. An alert (third party
encryption gateway decrypt fail) will be generated.

ESGA v7.6 201 97

Advanced Encryption

Administrators can configure x-headers for messages sent to third party encryption
gateway via TRITON - Email Security. These x-headers facilitate communications
between ESG and third party gateway and they must match the corresponding settings
in third party gateway.

X-header, indicating decryption is allowed

Decryption x-header, indicating the message should be decrypted

Decryption success x-header, indicating the decryption is successful

Decryption fail x-header, indicating the decryption is failed

If there is no decrypt success/fail x-header, there may be errors in configuring ESG or


third party encryption gateway. An alert (third party encryption gateway does not
decrypt messages).
Sandwich mode(decrypt, scan and then encrypt messages) is not supported. So
decrypt success and decrypt fail messages will not be encrypted again. If Data
Security returns encryption for decrypt success and decrypt fail messages, ESG will
treat it as unencryption.

98 ESGA v7.6 201

Clustering Details

In this chapter you will learn about the details of linking multiple ESG nodes together
to form a cluster to share traffic load or function as a single virtual ESG. Clustering
benefits you in balancing process work among different ESG nodes when traffic is
heavy, consolidating messages received in all separate ESG nodes for PEM, and
easing management and configuration.
If you have multiple ESG appliances running, clustering is not a requirement. You
could have all of the appliances be as standalone and managed by one TRITON
Management Server. In this way, the type of appliances and the running mode could
mix with each other. This is not recommended, for it leads to inconsistent policy
settings easily.

Each ESG has three types of roles:

Standalone: single ESG processes mail traffic. ESG works as standalone by


default. Standalone ESG can be configured via TRITON console.

Primary: the primary node when working in cluster mode. Every cluster can have
only 1 primary ESG. The primary is the only node in cluster to be configured via
TRITON console.

Secondary: the secondary node when working in cluster mode. Every cluster can
have 1 to 7 secondary ESGs. The secondary ESG can not be configured via
TRITON console directly. The configuration is synchronized from the primary
ESG.

After cluster negotiation succeeds, the primary and secondaries will work in cluster
mode. If cluster negotiation does not succeed, all nodes in cluster will be standalone.
One cluster is composed of up to 8 ESG nodes: one as the primary and others as
secondaries. All the nodes within one cluster must be the same type of appliance and
run in the same mode. For example, a V10000 and a V5000 cannot cluster together, a
V10000 in dual mode and a V10000 in ESG mode too.
All the ESG nodes within one cluster share the same configuration and same log
database, accept mail traffic and deliver to destination mail server according to the
same policies.

ESGA v7.6 201 99

Clustering Details

Slide 7:1

ESG Clustering Key Features


In ESG cluster mode, mail traffic is distributed to different ESG appliances based on
some traffic balancing algorithm. The traffic balancing algorithm is not on the ESG.
Well talk about it later.
All ESG nodes are managed centrally and can log to a central log server for
consolidated logging across the enterprise. Even in a case with multiple clusters, all
log events should be centrally stored, and reporting should be aggregated across all
boxes. Even in a case with multiple clusters, all quarantined emails can be stored
locally in each ESG appliance or centrally on an external storage device, but should be
accessed from the same interface(s).
ESG cluster is an Active/Active load balancing feature that provides real time
synchronization between the primary and all secondary nodes for device
configuration, which provides the most flexibility and maximizes system investment
as requests are load-balanced across all available processing capacity. In Active/
Active deployment, each node must stay in a consistent state and must reflect the
current state of the system

100 ESGA v7.6 201

Clustering Details

Slide 7:2

Clustering with Load Balancer


This network deployment shows load balancer with three ESG appliances connected
to it. The load balancer redirects mail traffic to different ESGs with certain traffic
balancing algorithm. The balancing algorithm is part of the load balancer
configuration and not ESG.
The flow of traffic is as follows:
1. The firewall receives the email from the Internet.
2. If the messages are permitted, the firewall forwards them to the IP address of the
load balancer.
3. The load balancer, using the specified algorithm, decides to which of the ESG
appliance send the mail message.
4. The ESG, which receives the email from the load balancer, processes the message.
Log information is stored about the message. The permitted message is forwarded
to the email server.
5. The email server receives the messages which passed the ESG filtering. The
messages are now available for pick/delivery by the end users.

Clustering details
Each ESG in a cluster has one of three roles:

ESGA v7.6 201 101

Clustering Details

Primary

The primary node when working in cluster mode. Every cluster can have only
1 primary ESG. The primary is the only node in cluster to be configured via
TRITON console.

Administrators configure the policies on the primary node through TRITON


console. The configuration is then synchronized to all the secondary node.

Secondary

The secondary node when working in cluster mode.

Every cluster can have 1 to 7 secondary ESGs.

The secondary node can not be configured via TRITON console directly.

The secondary nodes synchronize the configuration from the primary node.

The status and logs are synchronized time to time as well.

The secondary nodes and the primary process mail traffic based on the same
policies.

Standalone

Any appliance that is not set as primary and not negotiated to be a secondary
stays in the role of standalone.

Important
One cluster has a maximum of 8 nodes.
All nodes of cluster must be the same appliance type: you
cannot mix V10000 with V5000 in the same cluster.
All nodes of cluster should have same applications
deployed on the appliance: all appliance are either single
mode (ESG only) or dual mode (ESG and WSG).
Tip
When cluster negotiation does not succeed, the primary
node will not become standalone automatically. The
administrator will need to change this manually.
Important
When you have more than one ESG appliances, clustering
is not a requirement. You can have all ESG appliances be
in standalone mode; however, you will have to configure
each ESG individually. This is not recommended and leads
easily to inconsistent policy settings.

102 ESGA v7.6 201

Clustering Details

Slide 7:3

Clustering with MX Record


This scenario does not have a visible load balancer deployed. It loads and balances
email traffic on multiple ESG appliances using MX record.
Lets take the diagram as an example, in which you have three ESG appliances in your
cluster. You can re-configure your DNS as one of the following:

Configure the MX record in DNS to return three routable IP addresses which bind
to the external interface of the firewall. Each record should have the same
preference value, so that the traffic is evenly distributed on the basis of number of
incoming connection requests. The firewall maps the external IP addresses to the
internal one of the corresponding E1 interfaces on the ESG appliances (1-to-1
NAT)

Configure the MX record in DNS to return three routable IP addresses which bind
to each of the E2 interfaces of the ESG. The E2 interfaces are in the DMZ. The E1
interfaces are on the LAN side.

The flow of traffic is as follows


1. The sending email server queries the DNS MX record for the destination domain.

The DNS returns three IP addresses with the same preference value.

The order of the IP addresses are set to round robin mode.

ESGA v7.6 201 103

Clustering Details

The email server connects to the first IP address in the list (the other addresses
are used if the first one does not respond).

The DNS returns the information about the next hop as well.

2. The email server connects to the next hop in the chain.


3. The messages arrive to the premise firewall. Permitted traffic is allowed through.
4. The firewall receives the incoming SMTP request and passes it to the
corresponding ESG appliance (using one of the two techniques listed above).
5. The layer 2 switch receives the connection request for the address of the selected
appliance. This appliance receives and processes the email message. If the policy
filter allows the email through, the appliance sends the email to the email server.
The email server receives the email that has passed filtering, which is now available
for pickup or delivery by the end use.

104 ESGA v7.6 201

Clustering Details

Slide 7:4

Centralized Management
Under cluster mode, the primary and all secondary nodes are managed centrally.
1. The administrator log on TRITON Unified Security Center to configure the
primary node.
2. The configuration of the primary is synchronized to all secondaries.
Note
Only the primary has read/write privilege for ESG
configuration.
3. The primary and all secondaries of the cluster performs process inbound and
outbound emails based on the same policies.
4. All off-box logs generated from different ESG nodes will be sent to same log
server and stored to same database.
Note
Only the primary has privilege to view ESG dashboard/
logs and generates reporting, the scope of reporting data
covers all ESG nodes in cluster.

ESGA v7.6 201 105

Clustering Details

Note
The secondary only has the privilege to filter emails and
report everything to the primary.
When the primary is down, all secondaries will keep working and filtering as
secondaries; no one will switch to the primary automatically. Administrator can reconfigure the cluster to pick a secondary to act as the new the primary via TRITON
Console. As the configuration is synchronized among the cluster nodes, the new
cluster will work in the same manner. The only lost is the emails in primary work
queue, which can not be recovered.
When a secondary appliance is down, the cluster is not affected. Cluster machines
continue to process mail traffic and share a configuration.

106 ESGA v7.6 201

Clustering Details

Slide 7:5

Standard Deployment
A load balance device is deployed in front of ESG cluster. It redirects mail traffic to
different ESG nodes with certain traffic balancing algorithm. This needs to be done by
customer when ESG is deployed as cluster mode.

ESGA v7.6 201 107

Clustering Details

Slide 7:6

Configuration Synchronization
Configuration synchronization is handled by Config Daemon. The following tasks are
synchronized:

Configuration: All updates in configuration will be propagated from the primary


to secondaries.

Log: In-box logs are synchronized from secondaries to the primary; off-box logs
are saved on the same external systems, no need to synchronize.

Quarantine: All quarantine related database and files are synchronized from
secondaries to the primary.

DSS registration: The primary registers with DSS Manager; DSS registration is
synchronized to secondaries.

Not to be synchronized (only on the primary)

Signature database update

Hot fix/patch update

Hybrid service registration

108 ESGA v7.6 201

Clustering Details

Note
Only when the primary has the same version and platform,
and run in the same mode with the secondary,
configuration will be synchronized. Otherwise the node
status would be not synchronized with an alert being
sent.

License synchronization
The same license can be imported to each node of the cluster before negotiating, and
the license can be imported after negotiating too: to the primary at first then
synchronized to all the secondaries automatically.
Customers must assure licenses of each cluster node are either same or empty. If
different licenses are imported to the nodes of a cluster, the cluster negotiation will
fail.

ESGA v7.6 201 109

Clustering Details

Slide 7:7

Hybrid Service and DSS Integration


Hybrid registration only happens on the primary and is not synchronized with the
secondaries.
Hybrid registration result is synchronized to the secondary. When the primary is
down, the secondaries still process Hybrid mail traffic.
DSS registration happens on the primary and is synchronized to the secondaries.
ESG cluster works in below way:

The primary registers with the DSS manager.

The primary and secondaries point to same DSS manager and log & reporting
server

All configuration except cluster related configuration (IP, primary/secondary) are


synced from primary to all secondary appliances.

Inbound traffic: traffic will be load balanced by MX record or external L7 load


balancer.

Outbound traffic: DLP scanning is on secondaries locally. But if the mail


quarantine is required, it will send the message to primary DSS manager for
processing.

110 ESGA v7.6 201

Clustering Details

Slide 7:8

PEM
Under cluster mode, the end user can log on either the primary or secondaries to
utilize PEM functions. ESG cluster provides 2 options to deploy PEM:

End users log on PEM of the primary only: The primary will redirect to the
relative node for PEM, which is transparent for end users.

End users log on the specific ESG nodes for PEM: End users are assigned to
different PEM address, which point to respective nodes in cluster.

The slide takes the first option as an example. The process of retrieving isolated
messages via PEM proceeds as follows:
1. ESG #1 (not the primary) receives a message, which is going to be isolated.
2. The ESG #1 writes an entry in the log about this message. As mentioned before, a
key field in the database is the ID of the ESG who isolated the message.
3. The recipient, which has been notified that he/she has received an message being
isolated, connects to the PEM of the primary node. The link in the notification
points to the TRITON - Email Security.
4. The primary ESG looks up the message details in the log database. Using the ID
of the machine, which isolated the message, can connect to that ESG (in this case
ESG #1)

ESGA v7.6 201 111

Clustering Details

5. The primary ESG communicates with the ESG #1, retrieves the message, and
sends it over to the user who requested it.

112 ESGA v7.6 201

Archive

As businesses develop, many organizations face the email issues in storage, discovery,
and retention:
Storage Management: Quota limits on mailboxes frustrate end users- yet unlimited
mailbox storage is costly for IT to provide and will only get worse as email traffic is
expected to grow to 507 billion messages per day by March 2013.
Discovery: 87% of U.S. companies faced law suits and 56% of companies have been
ordered to produce employee email by a court or regulatory body- yet only one in
seven organizations can recover email that is older than one year.
Retention and Compliance: Many laws and regulations require retention of email for
extended periods of time. Only 31% of organizations are moderately prepared to
comply.

Websense Email Archive is an add-on feature to ESGA. It is a full software-as-aservice (SaaS) email archiving solution that integrates with email servers to
accumulate email messages and provides quick, searchable access to users. It features
comprehensive e-discovery and compliance capabilities, which administrators can
setup for select users to access within the system.

ESG has the basic archive feature: saving all incoming messages to an archive queue
locally or remotely. This feature is different with the add-on archie that we are going
to talk about in this chapter.

ESGA v7.6 201 113

Archive

Slide 8:1

Websense Email Archive Solution Add-Ons


Websense offers three email archiving add-ons that address unique challenges:

Websense Email Archive: Provides end users unlimited mailbox size and allows
them to quickly and easily access their personal archived email.

Websense Discovery Archive: Provides Legal counsel/HR/Administrators the


ability to investigate, search, tag, and export email across the entire organization
to satisfy discovery requests and email retention.

Websense Email Continuity: Provides end users/IT with standby mailboxes,


enabling them to send and receive email when their companys email server is
unavailable as if it were online.

The three add-ons are available for all Websense email security products (ESGA,
ESG, HES, WES). Customers must have one of the following products to purchase
email archive add-ons:

Websense Email Security Gateway Anywhere

Websense Email Security Gateway

Websense Hosted Email Security

Websense Email Security

114 ESGA v7.6 201

Archive

Slide 8:2

Email Archive Package and License


Email Archive Package
Each email archive add-on could be purchased individually or together in various
package combinations:

Websense Email Archive Suite: Websense Email Archive + Websense Discovery


Archive

Websense Email Archive with Hosted Continuity: Websense Email Archive +


Websense Email Continuity

Websense Email Archive Suite with Hosted Continuity: Websense Email Archive
+ Discovery Archive + Email Continuity

ESGA v7.6 201 115

Archive

Websense Email Archive Solutions add-ons are available as a service in the cloud and
does not require the cost and complexity of any on-premise equipment.
Note
Websense Email Continuity cannot be orderable as an
individual add-on. It must be combined with Websense
Email Archive (Websense Email Archive with Hosted
Continuity or Websense Email Archive Suite with Hosted
Continuity).

License
For each email archive add-on, customers purchase a subscription license based on the
number of mailboxes/users. Flat rate pricing for all user bands.
The number of license users for Websense Email Archive Solutions doesnt need to
match the number of users for the Websense email security product.
Customer could purchase a subset of users. For example, customers may purchase
licenses for a defined group of internal users (i.e. HR, Legal) rather than for the entire
email user population.
The minimum number of users/mailboxes that could be purchased is 25.

116 ESGA v7.6 201

Archive

Slide 8:3

Websense Email Archive - Personal Archive


The slide above shows the key features of Websense Email Archive:
1. Roles Management: Websense Personal Archive supports any number of
customized user roles, each with a unique and customizable set of permissions
associated with them.

User Account: All new users of the archive can be granted access to their own
Personal Archive which will give them the ability to search their own
historical emails.

Account Manager: This role can be customized to set up new accounts, reset
passwords, and have additional privileges.

Policy Manager: The Policy Manager can view and update policy settings
(e.g., set time zones, specify storage of emails based on email direction).

Role Manager: The Role Manager can view and modify privileges of other
users.

Administrator: The Administrator role has all the above/available


permissions.

2. Log in to the Personal Archive

ESGA v7.6 201 117

Archive

Users can access their archives from Microsoft Outlook, IBM Lotus Notes,
Outlook Web Access, Lotus Notes Web Client, BlackBerry devices and
through a browser-based, secure website, which is a convenient option for
employees working from home or on the go.

Users only have to enter the usernames and passwords one time when opening
Personal Archive through Outlook. After one successful login, they will
automatically be logged in on all subsequent visits to the archive.

3. Outlook Look and Feel


In addition to giving users the power to restore lost or deleted emails back to their
inboxes, Email Archive also allows them to compose, reply to and forward
messages directly from their archives, just like they would in Outlook or Notes.
4. Quick Search
With Quick Search, simply typing desired search terms into the Message field and
clicking on the magnifying glass icon will begin the search. The search results
appear in a list of messages, and the search terms are highlighted in yellow in the
preview pane.
5. Advanced search

The Advanced Search option gives the users the ability to customize their
searches based on a variety of criterion, such as message keywords, to, from,
subject, date(s), attachments, etc.

Search multiple archives directly from Outlook, enabling better knowledge


management and retention of archives from departed employees.

6. Search Filter
Search Filters help further hone searches, allowing users to drill into specific
categories, such as sender, date, tags and attachments.

Websense Email Archive benefits:

Unlimited Mailbox
Websense Personal Archive gives an unlimited mailbox and allows users to quickly
and easily access their archived emails directly from Microsoft Outlook or Lotus
Notes. For added convenience, Folder Sync keeps users Outlook folders intact in the
archive for easier search and retrieval.
Restoring lost or deleted emails gives end users the power to restore lost emails
themselves- even things they may have deleted from their desktop, laptop,
BlackBerry device or Windows Phone 7.
Reducing the need for PSTs/NSFs eliminates free-range email files (PST/NSF), and
minimizes the burden on IT staffs and/or help desk
Improving server performance improves performance by reducing the footprint on
customers mail servers, and shrink backup windows

118 ESGA v7.6 201

Archive

Boost Productivity
Do you know how much time people spend sorting and filing their emails? Almost 9
out of 10 Internet users spend seven hours a week managing email. Thats a lot of lost
productivity! Plus, 95% of the email sitting in your inbox will never be looked at
again.
With the Email Archive, users can delete email with confidence and stop wasting time
managing the inbox.

Familiar, Easy-to-Use Interface


Email Archive seamlessly integrates with Outlook or Notes, so its easy to complete
the most common day-to-day tasks, using a variety of familiar shortcut buttons.

Comprehensive Search Features


Email Archive makes retrieving old or deleted email easy. Users can search for
specific emails and attachments two ways: Quick Search and Advanced Search. The
Advanced Search feature allows users to fine-tune their searches with key parameters.
The Search Filters help further hone searches. Users can also save their searches for
future reference (which means they dont have to keep recreating their most common
searches).

Eliminate Local Archives


Removing local archives helps reclaim space on users shared drives, reduces data
leak protection issues when laptops are lost or stolen and eliminates the hassle
involved in searching individual desktops when they are responding to a legal
discovery request.

ESGA v7.6 201 119

Archive

Slide 8:4

Websense Discovery Archive


Motions to discover electronic data are commonplace in todays world of litigation;
yet, more often than not, companies are unprepared to find electronically stored
information, especially email and attachments. Discovery costs represent at least 50%
of the cost of litigation- most of which relates to the collection and review of email.
Websense Discovery Archive is a complete and affordable Security-as-a-Service
(SaaS) solution designed to seamlessly manage the e-discovery needs. Our fully
managed, web-based service meets and exceeds the e-discovery and mailbox
management needs of businesses and public organizations of all sizes. Discovery
Archive also helps significantly reduce the time it takes to respond to e-discovery
requests.

Store and index every email, attachment, IM and BlackBerry message (SMS
text, PIN-to-PIN, call log) in a centralized, online, and tamper-proof repository.

Accelerate legal discovery and HR inquiries with real-time search and retrieval
and multiple export capabilities. Search the entire contents of archived emails and
attachments.

Enforce email preservation for lawsuits/legal holds, and protect attorney-client


privileged communications.

120 ESGA v7.6 201

Archive

Websense Discovery Archive Features


Store & Index

Discription

Secure and Automated


Message Capture

Discovery Archive automatically captures all


messages that are sent or received without any user
intervention and securely transmits them using
TLS or VPN encryption to the LiveOffice data
centers for retention.

Single-Instance Storage

Once archived, all email is de-duplicated and


stored as a single instance to optimize storage
space and streamline the searches.

Unlimited Storage

Unlimited storage eliminates the need for mailbox


quotas and compounding storage costs.

Replicated Storage

All email messages and attachments are stored in


real time and then replicated to multiple,
geographically dispersed top-tier data centers.

Secure Storage

Websense enterprise-class infrastructure meets the


highest industry standards for secure data storage,
including SAS-70 Type II, and guarantees the
authenticity of the archived data.

Tamper-Proof Storage
(Optional)

For companies with stricter compliance


requirements, WORM (write once, read many)
storage is also available.

Search & Retrieval

Description

Lightning-Fast Search and


Retrieval

Quickly search the entire contents of archived


emails and attachments using a variety of search
criteria, including to, from, date, subject, message
body, message attachments and other message
properties.

Online Review and Tagging

Reviewers can efficiently navigate through search


results, identify highlighted search terms and tag
potentially harmful emails, so they are easily
retrievable for further review.

PST/NSF File Export

Once users have tagged all emails related to a


specific case or matter, they can use Discovery
Archives export feature to load the files into a case
management solution or other application for
further review and analysis.

EDRM XML Export

Approved IT and legal personnel can seamlessly


export messages from Websense Discovery
Archive to e-discovery solutions from Clearwell
Systems, CT Summation, FTI Technology and
iConect.

ESGA v7.6 201 121

Archive

Search & Retrieval

Description

Re-Forward or Print

Reviewers have the ability to print or re-forward


suspect emails to the appropriate parties for further
review and case submission.

Saved Searches

Reviewers can create and save customized email


searches based on the organizations email
policies, and re-run them as necessary.

Policy Alerts

Reviewers can setup Policy Alerts to notify them


when an email meets Saved Search criteria (e.g.,
contains specific words or phrases).

Access & Control

Description

Familiar Interface with Full


Outlook and Notes
Integration

Discovery Archives highly intuitive, Microsoft


Outlook-like interface helps users get up to speed
quickly. In addition, full Outlook and Notes
integration allows administrators to centrally
deploy Websense Personal Archive (optional addon service) to some or all users.

Authentication and Granular


Permissions

Users authenticate using SSL to login, ensuring


secure access to the archive. Discovery Archive
has two distinct permission levels: Administrators
(full access) and Reviewers (monitoring rights for
selected mailboxes).

Policy-Driven Retention and


Destruction

Discovery Archive retains the emails and


attachments as required to meet customers
companys specific retention or regulatory
requirements. We also purge and destroy older
email stores based on the custom criteria (e.g., any
emails five or more years old).

Automated Legal Hold


Enforcement

If customers company is in pending litigation,


they can place legal holds on specific
communications (based upon search criteria) to
safeguard the staff or automated deletion policies
from inadvertently deleting case-relevant emails.

Attorney-Client Privilege
Tagging

Administrators and reviewers can flag attorneyclient privileged communications, which are
excluded from e-discovery requests.

Full Audit Trail

Discovery Archives search log captures the


auditing activities of the reviewers, so
administrators can ensure that they are conducting
appropriate reviews and meeting compliance
requirements.

122 ESGA v7.6 201

Archive

Access & Control

Description

User Groups

Administrators have the ability to group users


based on functional, divisional or custom criteria.
Reviewers can then search across these groups.

Mail Servers

Discovery Archive supports Microsoft Exchange


2003, Microsoft Exchange 2007, Microsoft
Exchange 2010, IBM Lotus Domino 6.x 8.x,
Novell GroupWise and Linux/Unix mail servers
(SMTP).

ESGA v7.6 201 123

Archive

Slide 8:5

Websense Email Continuity


Websense Email Continuity is an always-on backup that provides customers with
standby mailboxes, enabling them to send and receive email when the primary
server is unavailable as if it were online. Email Continuity allows the IT resources to
focus on the root cause of the outage instead of wasting time implementing temporary
solutions (e.g. MX repoints, etc.).

Serves as standby mailbox when the primary email server is down

Users can easily create, reply to and forward messages during outage with
Outlook Look and Feel

Email Restore supports Microsoft Exchange, IBM Lotus Notes and Novell
Groupwise

All emails sent or received during the outage are automatically routed back to the
email server after the outage

30-days of email history

Overcome email server breakdowns, stay productive and protect customers business
with Websense Email Continuity.

124 ESGA v7.6 201

Archive

Slide 8:6

Inbound Email Flow


The slide above describes the inbound email flow with Websense Email Archive.
1. During normal operation
The inbound messages are transmitted via the Internet, filtered for spam and virus
by ESGA, routed to the email server and distributed to the recipients. At the same
time, the inbound messages are captured in the Websense Email Archive via
journaling. End users can access the these messages through Outlook, Lotus
Notes, or web-based archive interface anytime.
2. During the email server outage
Email messages are transmitted via the Internet, filtered for spam and virus by
ESGA and automatically routed to the Email Continuity archive, bypassing the
failed email server. End users can access the messages that are sent before and
during the outage. Once the email server is restored, messages are routed from the
archive back to the email server (no duplicate messages are stored).

ESGA v7.6 201 125

Archive

Slide 8:7

Outbound Email Flow


The slide above describes the outbound email flow with Websense Email Archive.
1. During normal operation
The outbound messages are sent from end users, routed to the email server,
filtered for virus and confidential data leakage by ESGA, and transmitted over the
Internet. At the same time, the outbound messages are captured in the Websense
Email Archive via journaling. End users can access the these messages through
Outlook, Lotus Notes, or web-based archive interface anytime.
2. During the email server outage
End users compose, reply to and forward messages directly from the Email
Continuity interface. The messages are then scanned for viruses and confidential
data leakage by ESGA prior to being transmitted over the Internet.

126 ESGA v7.6 201

Archive

Slide 8:8

Websense Ingestion Service


The diagram above illustrates the Websense ingestion service that help to imgrate the
legacy data from customer premises to Websense Email Archive server and preserve
chain of custody at the same time.

Legacy Data
Legacy data is a generally referred to as information stored in an old or obsolete
format (or computer system) that is, therefore, difficult to access or process. In the
email archiving world, legacy data refers to email data sitting on:

Email storages (e.g., Microsoft Exchange, Lotus Notes) on servers

Personal archives (e.g., PST/NSF files) on desktops, laptops and servers

Legacy email archives (e.g., Autonomy, EMC, CA)

Backup tapes

By ingesting your legacy email into Websenses archiving solutions, you have a
complete, living record of all your email history, both past and present for that
ingested email. But, it is critical to preserve the chain of custody when you move your
email data from one store to another.

ESGA v7.6 201 127

Archive

Chain of Custody
If chain of custody is a new or a vague concept to you, you are not alone. In the
simplest terms, a proper chain of custody establishes the integrity of a piece of
evidence, showing that it wasnt tampered with or otherwise altered since it was first
collected. Chain of custody has significant practical implications for IT managers and
other professionals.
Chain of custody plays a critical function in litigation, especially when opposing
counsel is challenging the authenticity of evidence particularly where digital
evidence and emails are involved. Weve found that very few organizations adequately
account for chain of custody or understand its importance until the authenticity is called
into question.

Two Optional of Ingestion Services


Websense Email Archive solutions do not require implementation/professional
services. However, there are 2 optional services that are available:

Websense Data Ingestion Service: Service generally purchased up front to import


all legacy email data from desktops, laptops, servers, into the Websense archive
solution.

Websense Data Export Service: Service generally purchased when a customer


terminates their Websense archive subscription so that they can re-capture the data
stored in our cloud.

The services are one-time fee based on a per GB basis. They arent performed at
customers sites. Customers merely ship all storage media desired to be archived to
Websense. Websense will then import/export the data.

Websense Data Ingestion Process


As the illustration above depicts, the Websense Data Ingestion Service process
involves 6 key steps:
1. Extraction
In the extraction phase, the customer extracts legacy email data from existing
repositories. This is a critical first step and one that provides the greatest
opportunity to break the chain of custody. So, its important to document the
number of original files/PSTs/NSFs sitting in customers existing stores in order
to compare to the total number of files/PSTs/NSFs imported into the archive.

128 ESGA v7.6 201

Archive

The following table lists the descriptions for the supported formats:

Description

PST

EML

Does not preserve folder structure

MSG

Longer conversion and ingestion time


Does not preserve folder structure

NSF

Ability to preserve folder structure upon request


Longer conversion and ingestion time
Additional cost

Files need to be 2 GB or less in size


PST files cannot be password protected
Files can be zipped with a password
Ability to preserve folder structure upon request

Physical Media:

Format

If the volume of data is less than 10 GB, send the data on CD/DVD, Flash
Drive or via S-FTP (secure FTP).
If the data volume is greater than 10 GB, send the data via a USB hard
drive (high speed USB 2.0 with external power recommended).

Encryption: Websense requires a basic level of encryption (i.e., passwordprotected zip file) for all email data transferred and strongly recommends
using TrueCrypt Freeware (truecrypt.org) or PGP (pgp.com) to encrypt data
before shipping it to us as a best practice. Zipping the data with a password is
also acceptable, but less desirable, since it delays the import. We do not accept
password-protected PSTs, since they are generally not secure and cannot be
processed by our legacy upload system.

2. Shipment
During this phase of the project, Websense outlines the key steps for the customer
project. This includes having the customer complete our Legacy Upload Order
Form, which captures key chain of custody information, such as the date and time
sent, number of files, number of emails, format of legacy data (e.g., PSTs),
amount of data and contact information. Then the customer ships the data along
with the Legacy Upload Order Form to us, addressed to the attention of the
Websense Data Import Center.
3. Verification
Upon receipt of the data, we log, initial and date the Chain of Custody Form (part
of the Legacy Upload Order Form) under received data. The case is then assigned
to one of our specialists, who reviews the details and adds the customer project to
our schedule. The completion date is based on the current volume of legacy

ESGA v7.6 201 129

Archive

uploads in our queue. In the meantime, Websense places the data in its current
format in our data vault for safe storage.
After the data is extracted and uploaded, the specialist from the customer verifies
that the data volumes match the information they provided on the Chain of
Custody Form and updates the customer any deviations. If there is a material
difference in the email received/extracted vs. the data on the form, we then
explore the reasons for the missing data.
Note
If the data does not match the Chain of Custody Form or
the data is corrupt, the customer will be notified to inform
our findings and determine the next steps.
4. Data Import
After the legacy data is prepped, the specialist from the customer imports it into
the archive. Websense securely captures and indexes all messages (and
attachments) imported into one of our archiving solutions Discovery Archive or
Email Archive. Once indexed, administrators and users can quickly and easily
search the entire contents of archived emails and attachments, using a variety of
search criteria, including to, from, date, subject, message body, message
attachments and other message properties. As a final check, the specialist from the
customer verifies the data and signs off on the Chain of Custody Form.
5. Chain of Custody Documentation
Once the import is complete the customer will be notified with the results. Any
deviations from the original Chain of Custody Form prior to ingestion are noted
and reviewed.
6. Returned Data/Destroyed Data
The final step in the process involves Websense either returning the original data
to the customer or destroying it. In either case, we document the process and note
the date the media was either returned or destroyed per the customer wishes and
retain the final copy of the Chain of Custody Form in our locked data vault for
records.

Key Points About Websense Data Ingestion Service

Use of FTP: Websense recommends using the FTP option when shipping data for
only small data sets (i.e., less than 10 GB).

Supported Formats: Websense supports a variety of email formats, including PST,


EML, MSG, NSF and MBOX. PST is the required format if preserving the folder
structure for the data that is imported.

130 ESGA v7.6 201

Archive

File Sizes: Websense can process PSTs up to 20 GB, but we prefer files that are 2
GB or less. Smaller PSTs have less chance of corruption and process faster.

Media Types: Though we accept DVDs, Websense recommends physical hard


drives or flash drives when the customer ships data to us.

Encryption: While Websense requires a basic level of encryption for email data
shipped to us, we do not accept password-protected PSTs, since they are generally
not secure and cannot be processed by our legacy upload system.

Mapping Archiving Accounts: When we import the PST files, we first scan them
to determine how to map the customer users to individual archive accounts. If a
senders email address is found in more than 80 percent of the messages in the
PST file, the emails are archived under the senders address. This process,
however, is error prone, so we recommend providing us with a CSV file for
mapping purposes. If a CSV file is not provided, the system automatically
determines the mapping.

Supported Message Types: We currently only archive post items. Non-post items,
such as calendars, contacts, system messages and voicemails, are not currently
supported.

Legacy Unassigned Archive: If we cannot match an email address or common


name to an archived account, we place the emails in the Unassigned Legacy
Archive. Emails in the Unassigned Legacy Archive must have a common name
(unmatched) or an email address with a recognized domain in the archived
account.

Corrupted Data: Before sending us data, we encourage the customer to run


scanpst.exe, a utility included with Microsoft Outlook, to determine whether or
not the PST file is corrupted. We cannot import corrupted PST files.

ESGA v7.6 201 131

Archive

Slide 8:9

Websense Email Archive - Folder Sync


Folder Sync is an add-on service to Websense Personal Archive. As end users create,
delete, and rename folders, as well as move email messages from one Outlook folder
to another, the scheduled synchronization service replicates the identical folders and
message location in Personal Archive. After initial synchronization, Folder Sync
sends only incremental mailbox folder changes, which can be scheduled hourly, daily
or weekly as users see fit.
Folder Sync does not require plug-ins or hardware. It is compatible with Microsoft
Exchange Server 2003, 2007 and 2010. Administrators can selectively deploy Folder
Sync on a per-user and/or Exchange Server basis.

132 ESGA v7.6 201

Archive

Slide 8:10

Quick Start - Configuration on Exchange Server


You need to activate Microsoft Exchange Journaling and direct emails to the
Websense Email Archive servers. As a result, every email that is sent to and from the
exchange server will be copied and transfered to the Websense Email Archive servers.
1. Log on to the exchange server with administrator account.
2. Follow the steps outlined in journaling guides, available on our website. The
instructions depend on the version of Exchange.
Exchange 2003
Standard Journaling
Guide

http://www.websense.com/content/support/
library/email/shared/exchjour_2003_std/
first.aspx

Exchange 2003
Envelope Journaling
Guide

http://www.websense.com/content/support/
library/email/shared/exchjour_2003_env/
first.aspx

Exchange 2007
Journaling Guide

http://www.websense.com/content/support/
library/email/shared/exchjour_2007/first.aspx

Exchange 2010
Journaling Guide

http://www.websense.com/content/support/
library/email/shared/exchjour_2010/first.aspx

ESGA v7.6 201 133

Archive

3. When prompted, point the email to the Websense address shown in the table.

134 ESGA v7.6 201

EMEA servers

<your-domain>@WebsenseJournal.archivesuite.com

U.S. and others

<your-domain>@WebsenseJournal.archivecloud.net

Archive

Slide 8:11

Quick Start - Configuration on Websense Archive Server


Enabling administrator view of archive
As a security measure, the default settings on the archive disallow administrator view
of end user email. To view all user messages in the archive, perform as follows:
1. Log on with the credentials provided by Websense.
2. Click the Administration bar at bottom left of the page.
3. Click the admin@... address.
4. Under the Privilege option, check View All Accounts.

Verifying the setup


After completing the setup process, test the archive as the follows:
1. Log on with the credentials provided by WEbsense.
2. Click Accounts on the left-hand side of the screen, and select the email address of
your choice.
3. To explore the features of archive, click Help on the left side of the page.

ESGA v7.6 201 135

Archive

Slide 8:12

Quick Start - Configuration on Client


The end users can use Personal Archive Access (PAA) to access their individual
archives directly from Microsoft Outlook. Provide the setup instructions to the end
users:
1. Create a new folder in Outlook (File > New > Folder).
2. Highlight the new folder, and select File > Folder > Properties.
3. Select the Home Page tab.
4. Enter the URL for Personal Archive Access:
EMEA

https://websense.archivesuite.com

U.S and others

https://websense.archivecloud.net

5. Mark the box Show home page by default for this folder.
6. Click OK.
7. Click the newly created folder.
After the Personal Archive is configured, end users are prompted only once for their
user name and password. Additional details for the end users are contained here:

136 ESGA v7.6 201

Archive

http://www.websense.com/content/support/library/email/shared/email_archive/
WEA_end_user_guide.pdf
Note
Websense offers Websense CloudLink, an application to
help to deploy Outlook Web folders to all of the end users.
CloudLink is included with the Websense service.

ESGA v7.6 201 137

Archive

Slide 8:13

Archive vs Backup
Backups are for disaster recovery, while archives are for retention and discovery.
Backups were never really intended to meet regulatory requirements and other
compliance needs. They most effectively serve as a short-term insurance policy to
facilitate disaster recovery (assuming they are kept off-site).
Archiving, on the other hand, is specifically designed to quickly and easily meet
regulatory requirements and other compliance needs. In addition, archiving serves
another important role. It can reduce the strain on customers in-house email servers.
With growing email volumes and skyrocketing storage costs, email archiving can:

Reduce the size of email stores

Reduce backup windows

Reduce the amount of IT budget earmarked for storage

For most modern organizations, the operational, legal and compliance challenges
of email necessitate going beyond simple backups.

138 ESGA v7.6 201

ESG Log and Reporting

ESG logs and collects data about email traffic and system activity to generate
presentation reports and dashboard charts and statistics for today and history. Records
of email filtering activities and system events are stored either in the on-box database,
that is, local PostgreSQL, or received by the off-box log server and then transferred to
the remote database, Microsoft SQL Server.
From TRITON - Email Security, ESG classifies the logs into: message log, audit log,
system log and console log; and catalogs the reports into: today dashboard, history
dashboard, and presentation report.
ESG log and reporting provide the facilities for administrators to configure the local
and remote log database and the database maintenance options, set report preferences,
and customize a report template, schedule and run a presentation report.
Now lets look at log and reporting in more details!

ESGA v7.6 201 139

ESG Log and Reporting

Slide 9:1

Primary Concepts
ESG logs

Message log records information about each inbound message received by ESG.
Message logs are collected from the ESG internal component: Filter Demon.

Audit log, only available for super administrators, shows which administrators
have accessed the Email Security module of TRITON console, as well as any
changes made to policies and settings. Audit logs are collected from the ESG
internal component: Config Daemon.

System log records all ESG system level events, along with any errors or
warnings produced. System logs are collected from each component inside the
ESG, such as the Filter Daemon, Update Daemon, Config Daemon, etc.

Console log records any administrator activities or changes made to the Email
Security module of TRITON console. Console logs are collected from TRITON Email Security, not from the ESG internal components.

ESG logs are searchable by predefined time periods or customized conditions. For
message logs, you can refine the search using conditions like email address, scanning

140 ESGA v7.6 201

ESG Log and Reporting

result, or message status and export the search results to an Excel, HTML, or XML
file. For other three types of log, you can exported to an Excel or HTML file.
Note
The maximum number of log entries exported cannot be
greater than 100,000.

ESG reporting

Today dashboard provides high level summary statistics of today to identify


potential security and email flow and delivery issues with high quality and
interactive graphics.

History dashboard of last 30 days, similar with today dashboard, get an overview
of email scanning activity for up to the past 30 days.

Presentation reports include a set of predefined charts and tabular report templates
with which you can generate graphical reports of Email Security Gateway
message traffic activities.

ESGA v7.6 201 141

ESG Log and Reporting

Slide 9:2

Log and Reporting System Architecture


The slide displays the main components of ESG log and reporting system. The log
system is built in ESG, including log collection, on-box log database, and Log
Daemon. The log system collects logs from ESG internal components then stores the
logs on-box or off-box. The reporting system is composed of Log Server and MS SQL
Server, which is similar with WSG reporting system.
1. A collection of raw data, in the form of logs from various ESG internal
components, are fed into the Log Daemon.
2. On-box logs are transferred to PostgreSQL.
3. Off-box logs are transferred to the Log Server.
4. If the Log Server is unreachable, a Cache Log is utilized. Once the logs are in MS
SQL Server, it utilizes an Extract/Transform/Load (ETL) process in every 20
seconds to convert original logs to reporting data.
5. Stored procedures are utilized to generate reports and dashboards via the WebUI.
As you will see, the ESG WebUI is very similar to WSG Reporting UI.

142 ESGA v7.6 201

ESG Log and Reporting

Slide 9:3

Message Log
The Message Log records information about each inbound message received by Email
Security. Access the Message Log on the Main > Status > Logs page. You can
configure the number of entries per log page, between 25 and 200, in the Per page
drop-down list in the log table banner. Scroll through Message Log pages by clicking
the back and next arrows in the banner, or enter a specific page number in the Page
field and click Go.
Message records are saved for 30 days. To preserve message records longer than 30
days, use the Export option to export the log on a regular basis. Exporting does not
remove records from the Message Log. It transfers log data to an Excel, HTML, or
XML file.
When the Message Log page appears, the most recent records are shown. Use the
View from/to fields to specify the date/time range for the log entries you want to see.
The calendar includes the following options:

Click the time in the lower left of the calendar to set the time in hours and
minutes.

Click Today to set the calendar date to todays date.

Click Apply to activate your date/time calendar changes.

If you click Clean and then Apply, the current date/time selection is removed.

ESGA v7.6 201 143

ESG Log and Reporting

The Message Log includes several search options, including date range or keyword
searches. Determine the date/time range for a search by selecting dates in the View
from/to field calendar controls. Default value for the from or to field is the date and
time that you open the log.

144 ESGA v7.6 201

ESG Log and Reporting

Slide 9:4

Audit Log
Websense Email Security provides an audit trail showing which administrators have
accessed TRITON - Email Security, as well as any changes made to policies and
settings. This information is available only to Super Administrators. Monitoring
administrator changes through the Audit Log enables you to ensure that system is
controlled responsibly and in accordance with your organization acceptable policies.
Click the Audit Log tab on the Main > Status > Logs page to view the Audit Log,
and to export selected portions of it to an Excel spreadsheet or an HTML file, if
desired. Audit records are saved for 30 days. To preserve audit records longer than 30
days, use the Export option to export the log on a regular basis. Exporting does not
remove records from the Audit Log. It transfers log data to an Excel or HTML file.
When the Audit Log page opens, the most recent records are shown. Use the View
drop-down list options located above the log to select the range of log entries you
want to see: All, One Day, One Week, One Month, or Custom. When you select
Custom, use the View from/to fields to specify the desired date/time range for the log
entries you want to see. The calendar includes the following options:

Click the time in the lower left of the calendar to set the time in hours and minutes

Click Today to set the calendar date to todays date.

Click Apply to activate your date/time calendar changes.

If you click Clean and then Apply, the current date/time selection is removed.

ESGA v7.6 201 145

ESG Log and Reporting

Slide 9:5

System Log
System Log records for Email Security Gateway reflect the current state of the system,
along with any errors or warnings produced. Click the System Log tab on the Main >
Status > Logs page to view the System Log, and to export selected portions of it to an
Excel spreadsheet or an HTML file, if desired. System Log records are saved for 30
days. To preserve System Log records longer than 30 days, use the Export option to
export the log on a regular basis. Exporting does not remove records from the System
Log. It transfers log data to an Excel or HTML file.
When the System Log page opens, the most recent records are shown. Use the View
drop-down list options located above the log to select the range of log entries you
want to see: All, One Day, One Week, One Month, or Custom. When you select
Custom, use the View from/to fields to specify the desired date/time range for the log
entries you want to see. The calendar includes the following options:

Click the time in the lower left of the calendar to set the time in hours and
minutes.

Click Today to set the calendar date to todays date.

Click Apply to activate your date/time calendar changes.

If you click Clean and then Apply, the current date/time selection is removed.

146 ESGA v7.6 201

ESG Log and Reporting

Slide 9:6

Console Log
The Console Log is a record of any administrator activities or changes made to the
Email Security module of the TRITON Unified Security Center. Click the Console
Log tab on the Main > Status > Logs page to view the Console Log, and to export
selected portions of it to an Excel spreadsheet or an HTML file, if desired.
Console Log records are saved for 30 days. To preserve Console Log records longer
than 30 days, use the Export option to export the log on a regular basis. Exporting
does not remove records from the Console Log. It transfers log data to an Excel or
HTML file.
When the Console Log page opens, the most recent records are shown. Use the View
drop-down list options located above the log to select the range of log entries you
want to see: All, One Day, One Week, One Month, or Custom. When you select
Custom, use the View from/to fields to specify the desired date/time range for the log
entries you want to see. The calendar includes the following options:

Click the time in the lower left of the calendar to set the time in hours and
minutes.

Click Today to set the calendar date to todays date.

Click Apply to activate your date/time calendar changes.

If you click Clean and then Apply, the current date/time selection is removed.

ESGA v7.6 201 147

ESG Log and Reporting

Slide 9:7

Dashboard
The Main > Status > Today: Health, Security and Value Since Midnight page
appears when you first log on to TRITON - Email Security. It displays alert messages
and graphical charts that show the current state of your email scanning software,
focusing on email traffic activity in your network. The charts on this page cover the
24-hour period beginning at 12:01 a.m. according to the time set on the Log Database
machine.
At the top of the page, two summary sections provide a quick overview of current
status:
1. The Health Alert Summary shows the status of your Websense software. Click an
error or warning alert message to open the Alerts page, where more detailed alert
information is available. Information in the Health Alert Summary is updated
every 30 seconds.
2. Under Business Value, see examples of how Websense Email Security has
protected your network today by blocking malicious email traffic, as well as the
total number of messages handled.
Below the summary information, up to four user-designated Flash charts provide
information about email scanning activities. These charts are available to Super
Administrators, and to other administrators who are granted permission to view
reports on the Today page. Click Customize to select the four charts you want

148 ESGA v7.6 201

ESG Log and Reporting

displayed. Information in these charts is updated every two minutes. You may need to
scroll down to see all of the charts.
Up to two buttons appear at the top of the Today page:
1. Customize, available to Super Administrators only, opens a page where you can
select which charts to display on the Today page.
2. Print, available to all administrators, opens a secondary window with a printerfriendly version of the charts on the Today page. Use browser options to print the
page.

ESGA v7.6 201 149

ESG Log and Reporting

Slide 9:8

Today Report
Use the Today > Customize page to select up to 4 charts for the Status > Today page.
Only Super Administrators with unconditional policy permissions can customize the
Today page.

150 ESGA v7.6 201

ESG Log and Reporting

Slide 9:9

History Report
Use the Status > History: Last 30 Days page to get an overview of email scanning
activity for up to the past 30 days. The four charts on the page are updated daily at
12:01 a.m. to incorporate data from the previous day, as determined by the time on the
Log Database machine. You may need to scroll down to see all the charts.
The exact time period covered by the charts and summary tables depends on how ESG
has been processing mail. During the first month that Websense software is installed,
the page shows data for the number of days since installation. After that, the reports
cover the 30 days prior to today. Depending on the reporting permissions granted to
the role, some administrators may not see the charts on the History page.

ESGA v7.6 201 151

ESG Log and Reporting

Slide 9:10

Presentation Reports
Presentation reports include a set of predefined charts and tabular report templates
with which you can generate graphical reports of Email Security Gateway message
traffic activities. You can run a report, customize a report template, or mark a
frequently used report as a Favorite. You can run any presentation report immediately,
or schedule it to run at a particular time or on a repeating cycle.
Not all report templates can be customized. Report templates that can be customized
display a different icon from reports that cannot be customized. If the Save As button
is enabled when you select a report name, then you can save and edit that report to suit
your needs. The Save As button is not enabled if you select a report that cannot be
customized.
Use the Main > Status > Presentation Reports page to generate charts and tabular
reports based on templates in the Report Catalog. The Report Catalog organizes a list
of predefined report templates and custom reports into groups. Expand a group to see
its corresponding templates and custom reports. Click on a template or report title to
see a brief description of what it includes.

152 ESGA v7.6 201

ESG Log and Reporting

Slide 9:11

Advanced Features
Log Database Option
Administrating the Log Database through TRITON Email Security involves
controlling many aspects of database operations, including the timing of maintenance,
the conditions for creating new database partitions, and which partitions are available
for reporting. Use the Settings > Reporting > Log Database page to manage Log
Database operations.
Database Partition
Database partition is the unit to store logs. You can create up to 70 partitions, with the
maximum size for each partition is 5120 Mega Bite.
Keep in mind that extremely large individual partitions are not recommended.
Reporting performance can slow if data is not divided into multiple, smaller partitions.
When a new database partition is created, reporting is automatically enabled for the
partition.
Database Roll over
Roll over is the process that the Log Database creates a new database partition. Use
the Roll over every option to indicate whether database partitions should roll over
based on size (MB) or date (weeks or months).

ESGA v7.6 201 153

ESG Log and Reporting

If the rollover begins during a busy part of the day, performance may slow during the
rollover process. To avoid this possibility, some environments choose to set the
automatic rollover to a long time period or large maximum size. Then, they perform
regular manual rollovers to prevent the automatic rollover from occurring.
Database Maintenance
Use the Maintenance Configuration to control certain aspects of database processing,
such as the time for running the database maintenance job, some of the maintenance
tasks performed, and deletion of database partitions and error logs.

Presentation Reports Drill down


With Presentation Reports, you can drill down into the underlying data of detailed
events, such as identifying the top 10 external addresses for outbound messages,
sorted by number of messages in a scheduled time range.
Presentation Reports provides comprehensive report catalogs covering from inbound
and outbound messages, hybrid and appliance, spam, virus and data security, message
transfer and system capacity. And it offers the flexibilities in defining the report time
schedule. You can get the expected reports through selecting the recurrence pattern,
scheduling time and data range, and specifying the output format.

The following are typical presentation reports available in ESG:


Overall message summary

Inbound Messages

Incoming message summary

Top Destination Domains by Message Size

Top N Internal Recipients by Volume

Outgoing message summary

Top Destination Domains by Message Size

Top Destination Domains by Volume

Data security summary

Top Data Security Policies

Spam and virus summary

Spam Quartile Chart

Top Virus Types

Message transfer summary

154 ESGA v7.6 201

ESG Log and Reporting

Delivery Failure

Encryption Gateway Routing

System capacity

Average message in work queue

ESGA v7.6 201 155

ESG Log and Reporting

156 ESGA v7.6 201

10

Troubleshooting Guide

This chapter talks about some basic troubleshooting techniques when working with
your ESGA deployments. The following topics are covered:
The latter sections will cover various areas of an ESGA implementation and discuss
issues in a question and answer format.

General Information: establishes a baseline to ensure that certain hardware and


software requirements are met.

License: explains the license expiration, counting, and reset issues.

ESG: tells about the possible errors when connecting the TRITON - Email
Security to the ESG and discusses potential reasons.

Log Server & Log Database: covers the Log Server and the Log Database
installation and troubleshooting methods.

Cluster: discusses the problems that may arise during the cluster implementation.

Hybrid: involves Q&A about the Hybrid connection, debug, and synchronization.

DLP: talks about the issues of DLP registration, DLP deployment, and policies.

Quarantine: includes the quarantine files location and quarantine Q&A.

System Counter: provides the detail descriptions about system counters.

Appendix: includes the references to ESG Shell, System Diagnosis, and Health
Daemon.

ESGA v7.6 201 157

Troubleshooting Guide

Slide 10:1

General Information
Before we start investigating the various areas of an ESGA implementation and
discuss various troubleshooting techniques, it is important to discuss certain
installation specifications.
The table above provides minimum requirements for each module. The information
below provides more detail surrounding some of the specific items such as database
versions and the location of certain log files.

Database version
Web (WSG) only:

SQL Server 2005, all editions except Express, all service packs, 32-bit and 64-bit,
not IA64

All products:

SQL Server 2008, all editions except Web, Express and Compact, all service
packs, 32-bit and 64-bit, not IA64

SQL Server 2008 R2, all editions except Web and Compact, all service packs, 32bit and 64-bit, not IA64

SQL Server Clustering, for all versions of SQL Server that we support

158 ESGA v7.6 201

Troubleshooting Guide

Slide 10:2

Troubleshooting
From the ESG appliance command line, you can perform the initial configuration,
modify settings, check appliance status, start/stop the modules via the commands like
firstboot, ip dns, module start, etc. Run help to acquire the command
descriptions that the ESG appliance provids.
Furthermore, you can enalbe SSH to connect with the ESG shell to check logs and
turn on debug mode. This is an effective way for troubleshooting and often used by
Technical Support.

A: Enable SSH
1. From the ESG appliance console, enter the following command to enable SSH
access:
ssh enable
2. Enter the following command to get a temporary passcode:
local-access

ESGA v7.6 201 159

Troubleshooting Guide

3. From a Windows computer, open Internet Explorer and browse to http://


passcode.websense.com.
Note
This website is only accessible within the Websense
intranet.
4. Input the temporary passcode and click Submit me! to convert it to a real SSH
password.
5. Back to the ESG appliance console, log in as websense-ts and enter the password
obtained in above step.
6. Run the command su and the password is websense#123.
7. Run the below command to access the ESG shell:
ssh esg

B: Log Files Location

Installation log: /var/log/esg_install.log

SSL Proxy log: /var/log/sslproxy.log

ESG statistic log: /var/logs

daily_counter: daily statistics since installation

counter_current: current days statistics

DLP logs: /opt/websense/PolicyEngine/Logs

Default ESG error log:

/var/log/maillog

/var/log/messages

ESG configuration processing log: /var/log/confd.log

ESG Log Server log:

To check the database issues during ESG Log Server installation:


$(installpath)\ESG Log Server\SQL\CreateDbInstall.log and *.txt output files

To check the ESG Log Server issues: Windows system event view

C: Turn on Debug Mode


1. From the ESG shell, enter the following command:
cd /etc/debug
2. Run the command below:
ls
The available modules, e.g. filter, scan_engine, mta, etc. are listed here.
3. Run the following command:
esgshell

160 ESGA v7.6 201

Troubleshooting Guide

4. Run the following command to debug:


debug module <module_name> event all
For example, to debug the scan_engine module, enter:
debug module scan_engine event all
5. Control the debug output location:
a. For standard output, enter:
debug output cli
b. For file output, enter:
debug file <path> <max_size> (wrap | nowrap)
path

If the path is a relative path, the file will be


placed in the /var/log folder.

max_size

Value: 1 - 256
Unit: MB

wrap | nowrap

To wraparound the file or not.

ESGA v7.6 201 161

Troubleshooting Guide

Slide 10:3

Troubleshooting - Example
The slide is an example to troubleshooting via SSH (ESG shell). As the diagram
shows, the message with confidential data shall be dropped based on the DLP policy;
while it is delivered to the destination directly. The administrator troubleshoots the
issue in the following ways:

Via TRITON Unified Security Center. Check logs, policies, and configurations.

Via SSH
a. Log into the ESG shell
b. Check the log files
c. Turn on Debug mode to track the filtering and delivering process.

162 ESGA v7.6 201

Troubleshooting Guide

Slide 10:4

License
Issue A: What is the behavior when the ESG user license is exceeded?
All features in a license key have the same expiration date with the key. E.g, if a key
has three features, AV/AS/DLP, and will expire by Dec 22, 2012, AV/AS/DLP will
expire on Dec 22, 2012 too. Even if customer adds another feature before Dec 22,
2012, the expiration date is still Dec 22, 2012.
UpdateD considers a key expired when DDS server returns the expired flag, which is
the unique way to judge the keys validation. So its useless to change system time.
ESG sends messages to ESG alert system according to the remaining days in service
period:

Message level in 7~30 days

Warning level in 0~7 days

Critical level after expired

Issue B: How are user licenses counted?


ESG counts the user licenses in this way: the total number of all the outbound email
senders in 7 days.

ESGA v7.6 201 163

Troubleshooting Guide

Issue C: How do view the current licensed user count?


From the ESG Shell, enter the following command:
counter | grep subscriber

Issue D: Is there a way to reset the user counter?


From the ESG Shell, enter the following command:
counter f

164 ESGA v7.6 201

Troubleshooting Guide

Slide 10:5

License Reset
You may need to perform an ESG subscription key reset after you have uninstalled a
fully licensed version of TRITON - Email Security, and then reinstalled the product.
An error message warns you that ESG is already licensed.

From the V-Series Appliance Manager


1. Log in to the Appliance Manager: https://Appliance_C_IP:9447/appmng/.
2. In the Administrator > Toolbox page, click Launch Utility in the Command
Line Utility section.
3. Select Websense Email Security Gateway in the Module drop-down list.
4. Select esg-license-reset in the Command drop-down list.
5. Click Run.
The Console output displays "Success" when the subscription key is successfully
reset.

From the ESG Shell


1. Log in to the ESG Shell as the root user.

ESGA v7.6 201 165

Troubleshooting Guide

2. Run the following command:


[root@v10000-esg ~]# reset license
The Console output displays "Success" when the subscription key is successfully
reset.

166 ESGA v7.6 201

Troubleshooting Guide

Slide 10:6

ESG Error
Issue A: Why Email Security could not be launched?
The screenshot indicates that TRITON - Email Security service either hasnt started or
met a starting issue.
Follow the steps to try to resolve this issue:
1. Log in to the machine where TRITON Unified Security Center is installed.
2. List all the services.
3. Restart the Websense TRITON - Email Security service.
4. Click Retry on TRITON - Email Security user interface.

Issue B: Unable to connect to the ESG appliance


The connection error page will be displayed when TRITON - Email Security cannot
connect to the ESG appliance.
Perform as follows to determine the cause of this issue:
In the Setting > Email Appliance page, check the unconnected reason by moving the
mouse over the warning icon.
Well talk about the way to solve this in the next slide.

ESGA v7.6 201 167

Troubleshooting Guide

Slide 10:7

Appliance Connection Error


The slide lists the possible reasons of the appliance connection error:

Appliance is not accessible:

Ping the IP addresses of the C, E, and P interfaces.


Restart the ESG modules if the console output returns with errors.

Check the statuses of all services in the appliance. If the network runs well,
the SSLProxy and config-daemon service issues would cause the connection
errors. Reset these services to fix the connection issue.

The TRITON - Email Security license does not match the appliance license:

The TRITON - Email Security version must be higher than the appliance:

Reset the appliance license on the appliance user interface, then set license to
the appliance.
Update the TRITON - Email Security or the appliance to the appropriate
version.

The TRITON - Email Security device ID does not match the appliance device ID:

168 ESGA v7.6 201

This error means the appliance is not the original appliance.


Change the IP address of the appliance and add it to the ESG appliance. Then
you can remove the original appliance from the ESG appliance.

Troubleshooting Guide

Slide 10:8

Log Server & Log Database Installation


ESG Log Server supports two installation modes:

Windows user authentication mode


In this mode, you must log in to the window with the existing domain user and can
access the SQL Server without password. The windows user must have the
permissions to access the following system resources:

ODBC resource (create/edit ODBC data source)

Service control manager (create/delete windows service)

Database user authentication mode


The database user must meet the following permission requirements:

ESG database owner account

The database owner must have the membership in one of the following
roles in the MS SQL database:
SQLAgentUser role
SQLAgentReader role
SQLAgentOperator role
The database owner must have the membership in db_datareader role
The database owner must be a member of the dbcreator fixed server role

Login account

ESGA v7.6 201 169

Troubleshooting Guide

170 ESGA v7.6 201

The installation account must have the sysadmin permissions, because


ESG stored procedures will access (start/stop operation) the SQL Server
Agent service.

Troubleshooting Guide

Slide 10:9

Log Server & Log Database Deployment


There are at least three deployment modes when you install the TRITON Management
Server (EIP Server), Log Server and Log Database:
1. EIP Server + Log Server in one machine (A), Log Database in another dedicated
machine (B);
2. EIP Server in one machine (A), Log server + Log Database in another machine
(B);
3. All in one machine (A).

We recommend using a dedicated SQL Server to run the ESG off-box products, saying
mode 1, thus you can get the best performance.
Especially for mode 1, you need to install the SQL Server component Tools on
machine (A) in advance. In this way, the Log Server can work in BCP mode and speed
up the log record insertion.

ESGA v7.6 201 171

Troubleshooting Guide

Slide 10:10

Log Server Errors


Issue A: What might cause the failure of the Log Server service start?

Failed to establish the connection with the Log Database.

The IP address/hostname of the Log Database may be changed.

Issue B: What might cause the Log Server failed in receiving ESG log
records
The ESG failed to establish the connection with the Log Server.

Check settings from the TRITON - Email Security: Settings > Report Settings >
Log Server.

Check if the firewall is blocking the connection request.

172 ESGA v7.6 201

Troubleshooting Guide

Slide 10:11

Log Database Errors


Issue A: What might cause the failure of the Log Database installation?

The Log Database directory must be an existing directory.

The Log Database directory must be writable on the remote SQL Server.

The Log Database user must have the permission to run the scripts.

Issue B: What might cause no data in ESG report?

No data is sent to the Log Server.

No ETL job is running on the Log Database.


Check job status:
a. Connect to the Log Database with the SQL Server Management Studio
b. Run the below command:
exec usp_esg_get_job_status

Issue C: What might cause the alert Log database extract, transform,
and load process is not functioning properly occurred?

The administrator modified the Log Database system time after installation.
ESGA v7.6 201 173

Troubleshooting Guide

Recreate the background ESG job with the following command:


exec usp_create_background_jobs

How can I check job performance?

Run the following command and check the value of timeusage (ms):
exec usp_esg_get_job_perf

174 ESGA v7.6 201

Troubleshooting Guide

Slide 10:12

Log Database ETL Failed

Check the error log table:


select * from wse_error_logs

Check the job history:


select * from msdb.dbo.sysjobhistory his,
msdb.dbo.sysjobs jobs

ESGA v7.6 201 175

Troubleshooting Guide

Check the SQL Server log in the SQL Server management studio:

176 ESGA v7.6 201

Troubleshooting Guide

Slide 10:13

Cluster - Secondary Not Sync


The slide lists the possible reasons of the second appliance not sync:

The cluster communication IP address cant connected between the primary and
the secondary nodes.

Set the correct cluster communication IP address for the primary or the
secondary nodes.

Check if the primary and the secondary can reach each other on TCP port
6671.

The appliances within a cluster are not the same V-Series platforms (V10000 G2
or V5000 G2).

Dont use different V-Series platforms.

Check if the V10000 platforms and ESG versions are the same for all cluster
nodes:

Open the file /usr/local/etc/feedback.xml and check the following fields to


make sure the platforms are the same:
<data name="ApplianceProvisioning">...</data>
<data name="AppliancePlatform">...</data>

Open the file /usr/local/etc/esg.version and check the field major_verion to


confirm the ESG versions are the same.

ESGA v7.6 201 177

Troubleshooting Guide

Check the file /tmp/md5sum.txt.


This file lists all the synchronized files between the primary and the secondary
nodes. The synchronized files are customized in /usr/local/etc/cluster_sync.conf.

178 ESGA v7.6 201

Troubleshooting Guide

Slide 10:14

Cluster - Primary or Secondary Crash


Issue A: When the primary ESG has crashed and cannot be recovered,
how to re-setup the cluster with the left nodes?
1. In the TRITON - Email Security > Setting > General > Email Appliances
page, release all the cluster nodes to standalone mode.
2. Re-setup the cluster.

Issue B: When the secondary ESG has crashed and cannot be recovered,
how to remove it from the cluster?
In the TRITON - Email Security > Setting > General > Email Appliances page,
remove the crashed node from the cluster.

ESGA v7.6 201 179

Troubleshooting Guide

Slide 10:15

Cluster - How to Track


Issue A: How to track the cluster negotiation result?

Check the cluster status

Log in to the ESG Shell as the root user.

Execute the command show cluster to show the current cluster status
Pay more attentions to the following fields:

180 ESGA v7.6 201

Work Mode indicates the ESG work mode


- standalone
- primary
- secondary
Work Cur State indicates the cluster negotiation status
- 0 negotiation failed
- 1 negotiation processing
- 2 configuration is out of sync
- 3 configuration is synchronized
- 4 lost network connection to peer
- 5 local version is higher than peer
- 6 local version is lower than peer
- 7 platform is not matched
Check this field for each secondary from the primary.

Troubleshooting Guide

Work Cur Mode indicates the cluster negotiation status.


The meaning of the status code is the same as above.
Check this field from the secondary.

Debug the cluster negotiation and configure the synchronized procedure

Log in to the ESG Shell as the root user.

Execute the following commands:


a. debug module heartbeat_daemon event all
b. debug module config_daemon event all
c. debug output cli
d. debug start

Check the system logs from TRITON - Email Security user interface.

Check the system alerts from TRITON - Email Security user interface.

Issue B: How to track the on-box logs?


1. Log in to the ESG Shell.
2. Run the following command:
[root@v10000-esg ~]# psql U postgres
3. Run the following command to connect to the ESG on-box database:
\c esgdb
4. Run the following command to list all the tables:
\d
5. Run the following command to show the audit log:
a. select * from audit_log
b. \g
6. Run the following command to show the system log:
a.

select * from system_log

b. \g

ESGA v7.6 201 181

Troubleshooting Guide

Slide 10:16

Hybrid
Issue A: Check if the sync service is running
1. Log in to the ESG Shell as the root user.
2. Run the command ps aux|grep sync_daemon
The Console output displays like the follows if the sync service is running:
root
2905 0.0 2.6 157692 102220 ?
Sl
2010
0:01 /usr/local/sbin/sync_daemon

Issue B: Export ESG sync service debug log


1. Log in to the ESG Shell as the root user.
2. Run the following commands:
a. bash-3.1# esgshell
b. esg1153-esg# debug module syncserver event all
c. esg1153-esg# debug output cli
d. esg1153-esg# debug start

182 ESGA v7.6 201

Troubleshooting Guide

Issue C: Check if the ESG can access the Hosted sync server
1. Log into the ESG Shell as the root user.
2. Run the command:
grep -i "HES_Sync_server" /usr/local/etc/hes_sync.conf
3. The Console output displays like below. This is the HES sync server name.
HES_Sync_server = t8-hsync-web.odd.blackspider.com
4. Telnet the sync server on port 443 like the below example:
telnet t8-hsync-web.odd.blackspider.com 443
Trying 172.16.167.146...
Connected to t8-hsync-web.odd.blackspider.com
(172.16.167.146).
Escape character is '^]'.
The Console output displays successful if the ESG can access the Hosted sync
server.
If not, configure the firewall settings to allow HTTPS traffic between the ESG and
HES servers.

Issue D: Allow the Hosted cluster to access ESG SMTP 25 port


During the hybrid configuration, the TRITON - Email Security shows the Hosted
cluster IP addresses. Make sure the firewall allows the email traffic from these IP
addresses to the ESG on port 25.

ESGA v7.6 201 183

Troubleshooting Guide

Slide 10:17

DLP Registration Failed


If the ESG failed in registration to the DSS Manager, check as the follows:
1. The ESG appliance needs a valid and unique FQDN. Run the following command
to view FQDN:
hostname -f
2. The DSS Manager should be reachable on port 5819, 5821, 8443, 9443 and 80.
3. The account information in the manual registration is valid and has the Manage
system module privilege.
4. If the registration still fails, run the register script manually to find the reason:
/usr/local/sbin/dlp_manager.sh register address
username password
address

DSS Manager IP address

Username/Password

Account information

184 ESGA v7.6 201

Troubleshooting Guide

Slide 10:18

DLP Policy Deployment Failed


If the deployment from the DSS Manager fails, check the follows:

If Eth0 of the ESG is reachable on port 5820, 8888, 8889 and 9080

If the Policy Engine (PE) service is running on the ESG by the PE service status

The warning messages during deployment

The following directory in the TRITON Management Server:


C:\Prograam Files(x86)\Websense\Data Security\tomcat\logs\dlp\dlp-all

ESGA v7.6 201 185

Troubleshooting Guide

Slide 10:19

DLP Policy Errors


I have added policies and rules in the DSS manager. Why they are not working?
If the added policies and rules arent working, check the following:
1. If the ESG is registered
If not, an alert displays on the TRITON - Email Security dashboard, saying that
the ESG is not registered.
2. If the DLP policy has been deployed from the DSS Manager
If not, an alert displays on the TRITON - Email Security, saying that the DLP load
configuration failed.
3. If the DLP policy on the ESG is in Enforce mode
4. If the rules action on the DSS Manager is Block All; or for the email DLP policy,
the action is Quarantine
5. If the rules of the DSS Manager is configured for the right direction: inbound/
Outbound/Internal
6. If the policies are still not working, unregister and re-register the ESG as the next
section described

186 ESGA v7.6 201

Troubleshooting Guide

The process to unregister and re-register ESG


1. Unregister ESG on the TRITON - Email Security.
2. Remove the ESG module on the TRITON - Date Security > System module.
3. Deploy on the DSS manager.
4. Register ESG on the TRITON - Email Security.
5. Deploy again on the DSS manager.

The message has been quarantined. Why the link to DSS incidence is not there in
ESG message log?
Mark the Audit incident option in the Action Plan of the rules.

Does the DLP scanning on ESG require DSS manager subscription?


Generally, the DLP scanning of ESG requires Data Loss Prevention feature on the
ESG subscription, but it does not require the DSS manager subscription.
While the questionable image feature does require the specific DSS subscription. If
the DSS subscription doesnt contain the questionable image feature, the feature
wouldnt be working.

If the DSS manager is down, is DLP policy still working on ESG?


The DLP policy would be working regardless of the availability of the DSS manager.
But if the DSS manager or the connection to the DSS manager is down, the incidents
and forensics would be cached locally on ESG.

How to debug the DLP scanning on ESG?


DLP scanning can be debugged with following command:
1. Log in to the ESG Shell as the root user.
2. Run the following commands to debug the DLP scanning:
a. debug module scan_engine event all
b. debug output cli
c. debug start
The DLP scanning result and rules triggered would be shown in xml format.
See the debug framework document for more details.

ESGA v7.6 201 187

Troubleshooting Guide

Slide 10:20

Quarantine Location
Quarantine Folders
All the quarantine relevant folders are located in the directory /var/spool/postfix/qd/.
They are either used to store quarantined messages, or to store information and
temporary files used by quarantine system.
The following table lists details of the quarantine folders.
qfolder

Refer to queue folders. All quarantined files are stored under this
folder. Each queue has a separate subfolder. For example, the spam
queue uses the qfolder/spam folder. If a queue is using a remote
storage, the subfolder will be a symbolic link with the same name. For
example, if the virus queue is using NFS storage, the qfolder/virus
will be a symbolic link to the mountpoint

qfile/qlink

All quarantine messages are stored here temporarily before they are
put into the destination queue folders. If there are many files under
this queue, probably the quarantine daemon stops processing
messages

blacklog

All quarantine requests from the filter module are stored here if the
quarantine daemon stops processing messages

188 ESGA v7.6 201

Troubleshooting Guide

defer

All logs of "delayed messages" are queued here before they are
processed and inserted into SQL server database

dbsync

If quarantine daemon fails to insert a quarantine log into SQL server


database (connection problem, execution timeout, execution failed),
the log will be saved in this folder

emls

Used to temporarily store quarantined or delayed messages in plain


text format (eml). The eml files are generated when you download or
view the messages through TRITON - Email Security user interface.
It will be then deleted, so supposedly there should be no files in this
folder

delay

If a queue is moving, for example, from the local storage to the remote
NFS storage, the incoming emails to this queue will be temporarily
saved in this folder because the destination queue folder is
unavailable. After the queue moving is done, these mails will be
flushed into the associated queue folder

rdelay

If a remote queue is temporarily unavailable, for example, the


network connection is down, the incoming emails to this queue will
be temporarily saved in this folder. After the queue recovers, these
mails will be flushed into the associated queue folder

failconf

Used to store the last configuration of a queue. If you see a file under
this folder, the queue is either moved, or failed to move

flusher, dlcache

Not used

Configuration File
The configuration file used by the quarantine daemon to connect to the Log Database
is /etc/freetds.conf.

ESGA v7.6 201 189

Troubleshooting Guide

Slide 10:21

Quarantine - No Messages in Blocked Messages


Ive sent a spam email, why cant I see quarantined logs in Blocked
Messages?
1. Log in to the ESG Shell with the root user.
2. Check /var/log/maillog to see if the email is captured by the scan engine. The log
should look like the following in normal situations:
/usr/local/sbin/filter[10435]: message id 2D55FA50002,
filter ASA-Filter, policy Outbound Default, rule AntiSpam, matched!
3. Make sure the quarantine action for spam and virus filters is turned on.
4. Check whether there are files under the /dbsync folder.

Yes.
It is likely to be a issue with the Log Database connection.
Run the below command to check whether the Log Database connects with
the quarantine daemon:
netstat anp | grep quarantine_daemon
a. The connection is well.
The SQL command for inserting quarantine logs may be failed. Start
debugging the quarantine daemon:
- Sent another email.

190 ESGA v7.6 201

Troubleshooting Guide

- View the SQL command that is sent to the SQL server.


- If the debug information shows that the SQL command failed, copy this
SQL string and execute it in the SQL Server Management Studio
manually.
b. No connection.
Check the Log Database configuration and the network connection.

No.
Log in to the SQL Server Management Studio to check whether the logs are
inserted into esglogdb76.dbo.esg_detail_quarantine_message and
esglogdb76.dbo.esg_detail_quarantine_recipient.
a. If yes, it is the TRITON - Email Security user interface problem.
Check the tomcat ogs on TRITON - Email Security for details.
b. If not, the quarantine daemon may fail to process messages.
Check whether there is message file under the /qfile/qlink folder;
Check whether there are core files under the /var/cores.

Why are the numbers of messages in queue list page and in message list
page different?
1. Log in to the ESG Shell as the root user.
2. Taking the spam queue as an example, run the following command to calculate the
total number of messages:
find /var/spool/postfix/qd/spam type f | wc -l
3. Compare the number with the queue list number and the message list number
If the number is the same as the queue list, but different with the message list,
some logs are probably not inserted into the Log Database successfully.
Check the dbsync folder to find these logs.

When I moved a queue from the local storage to NFS, TRITON - Email
Security user interface shows Data transfer ongoing for a very long
time. What happened?
If the ESG works in a cluster mode, perform as the follows:
1. Log in to the ESG Shell as the root user.
2. Run the command below to see the queue moving status of each cluster node:
show queuemovestatus
3. Debug the quarantine daemon to check whether the file copy is ongoing.
If you dont see any output of the file copy, cancel the data transfer from TRITON
- Email Security user interface and try again.

ESGA v7.6 201 191

Troubleshooting Guide

Slide 10:22

System Counters
How to show counter in ESG?
1. Log in to the ESG Shell as the root user.
2. Run the following command:
[root@localhost filter]# counter

What is the difference between ESG persistent counters and ESG nonpersistent counters?
We can ignore the persistent counters. They have the same values as the non-persistent
counters.

What does each counter mean?


Many counters are for the debugging purpose (dbg_* counters).

192 ESGA v7.6 201

Troubleshooting Guide

The below table lists the counters and the descriptions as the reference.
Counter Name

Description

received_msg

Messages number received externally. The messages include all


directions(inbound, outbound, internal, and open relay)

processed_msg

Messages which have been processed (policy has been applied on


the message)

Delivered_msg

Message which have been delivered

Dropped_msg

Message which have been dropped

Quarantined_msg

Message which have been quarantined.

In_queue_msg

Messages waiting to be processed or waiting to be delivered. A


large value of in_queue_msg means that the system is under stress.

In_deferred_queue
_msg

Messages in deferred queue

Queue_total/used/
freed_space

The 3 values indicate the disk space which can be used for message
processing

Inb_conn

Total smtp connections

Inb_reputation_rec

Connections dropped by reputation

Inb_rbl_rec

Connections dropped by rbl

Inb_rdns_rec

Connections dropped by rdns check

Inb_spf_rec

Connections dropped by spf

Av_hit

Number of virus-infected messages

Av_clean

Number of virus-clean messages

Av_err

Number of messages which have error in av-scanning

As_hit

Number of spam messages

As_clean

Number of not-spam messages

As_err

Number of messages which have error in as-scanning

Dlp_hit

Number of messages which have matched dlp policy

Dlp_clean

Number of messages which have not matched dlp policy

Dlp_err

Number of messages which have error in dlp-scanning

Filter_exception

Number of exception messages\

Encrypt

Number of messages which need encryption

Encrypt_success

Messages which succeed in encryption

Encrypt_fail

Messages which fail encryption

Decrypt

Number of messages which need decryption

Decrypt_success

Messages which succeed in decryption

Decrypt_fail

Messages which fail decryption

ESGA v7.6 201 193

Troubleshooting Guide

Counter Name

Description

Hes_dkim_verify

Messages from HES which need dkim verification

Hes_dkim_pass

Messages which have passed dkim verification

Relations within different counters


If a message with multiple recipients is applied with default policy and no exception
message happens, the below formula should be true:
Received_msg = processed_msg = delivered_msg + quarantined_msg +
filter_exception + bounced_msg + in_deferred_queue_msg
Dropped_msg = quarantined_msg

Why the above formula does not match?


The possible reasons could be:

The message has multiple recipients; different recipients have different policies;
e.g. some recipients are quarantined, and some recipients are delivered.
In this case, the counters will not match.

Exceptions
Some counters may be increased twice for a single mail.

Manual operations
For example, reprocess some messages from TRITON - Email Security > Main >
Message Management.

Messages are always coming into the ESG system.


Counters will not match until finishing all the messages process.

194 ESGA v7.6 201

Troubleshooting Guide

Appendix

ESG Shell Q&A


Debug tool do not show log. Why?

The debug daemon or modules to be debugged may not start.

Run the follow command:


svstat /service/debugd/

Run the below command to start it:


svc -du /service/ debugd /

Debug setting may be output to the wrong place.

Run the blow command to view the debug setting:


debug show

Some services dont function well.

Restart the ESG service to fix it:


/usr/local/sbin/esg_restart.sh

Several configuration values in the share memory are none or wrong.


Why?
Some services may not start.
1. Check whether the core files are created in the /var/cores/ directory.
2. Run the below command to start all services:
/usr/local/sbin/esg_restart.sh

The set/reset command do not function or report errors. Why?


The errors should be treated case by case. The following commands may help:

Run the set/reset command again.


The previous error maybe due to the related service busy.

Check the file /var/log/confd.log to find some clue.

Restart the ESG services.

System Diagnosis Q&A


Failed to generate the summary file. Why?
The summary file contains the configuration files and logs of all the modules,
therefore any errors from any module may cause the error.

ESGA v7.6 201 195

Troubleshooting Guide

The disk space and the network issues could be the most probable reasons.

Failed to open the configuration file. Why?


The file is encrypted with the decode key f~S#-/5A. Changes to the key may lead to
fail to decode the file.
If the ESG file in the whole package is damaged, check the hard disk usage of the ESG
module.

Health Daemon
Service didnt start

Service restart by health daemon

Description

Service which configured in /usr/local/etc/health.conf is not started.


Health daemon will start it by the configured command

Trigger condition

Service which configured in /usr/local/etc/health.conf is not started.


Health daemon will start it by the configured command

Message

Service %s down, start it by health_daemon

Alert

ALERT_TYPE_SERVICE_DOWN
The {%DyInfo} service is not available

Action

Start service by the configured command

Recovery

No need

Service restarted by administrator manually

Description

Service which configured in /usr/local/etc/health.conf is not started.


Health daemon will generate warning message to remind
administrator to start the service manually

Trigger condition

Service which configured in /usr/local/etc/health.conf is not started.


Health daemon will start it by the configured command

Message

Service %s down. Restart by itself

Alert

ALERT_TYPE_SERVICE_DOWN
The {%DyInfo} service is not available

Action

Administrator should start the service manually

Recovery

No need

196 ESGA v7.6 201

Troubleshooting Guide

Service start repeatedly


Description

Service restart and terminated abnormal repeatedly and exceeds a


threshold, health daemon will stop it

Trigger condition

Service restart and terminated exceeds 3 times

Message

Restart service:name=%s
pid=%d,mem=%f,max=%d,max_ratio=%f,cmd=%s service
memory exceed the configured limitation, restart it by the
configured command

Alert

No

Action

Health daemon stop the service. Administrator should check


system. The service can be restarted by command "svc -u
servicename" manually

Recovery

No need

Service memory leak


Description

Service memory keeps increasing and exceeds a configured


threshold (/usr/locat/etc/health.conf)

Trigger condition

If the "memory increase threshold" which configured by /usr/local/


etc/health.conf is not 0, the trigger condition is: when the memory
increase step is bigger the increase threshold and the memory is
bigger than the max limitation value.
If the "memory increase threshold" which configured by /usr/local/
etc/health.conf is 0, the trigger condition is: when the memory is
bigger than the max limitation value

Message

Kill service:%s pid=%d,mem_inc_thr=%d,


mem_last_val=%d,mem=%f,max=%d,max_ratio=%f cmd is [%s]
service memory exceed the configured limitation, killed by health
daemon

Alert

No

Action

Health daemon kill the service and generate warning messages in


the /var/log/messages

Recovery

No need

Service file handler leak


Description

Service generate too many pipe file handlers. It may be a file


handler leak. Health daemon will kill it and generate warning
messages

Trigger condition

Service generate too many pipe file handlers

Message

Process name [%s] pid [%u] may be fd leak, fd number[%u]. Please


check it
Service generate too many pipe file handlers. it may be a file
handler leak. Killed by health daemon

Alert

No

ESGA v7.6 201 197

Troubleshooting Guide

Action

Health daemon kill the service and generate warning messages in


the /var/log/messages

Recovery

No need

System disk space too small

/dev/xvda1 free space too small

Description

/dev/xvda1 free space is too little. Health daemon will set disk stress
flag. Mta will not be process email

Trigger condition

/dev/xvda1 free space is little then 4G

Message

/dev/xvda1 free space is little then 4G, set disk_stress

Alert

ALERT_TYPE_DISK_STRESS
Adequate amount of free disk space is not available

Action

Health daemon set the disk stress flag, mta will not process emails

Recovery

Release more disk space. Disk free space is bigger than 8G

/dev/xvda2 free space too small

Description

/dev/xvda2 free space is too little. Health daemon will set disk stress
flag. Mta will not be process email

Trigger condition

/dev/xvda2 free space is little then 4G

Message

/dev/xvda2 free space is little then 4G, set disk_stress

Alert

ALERT_TYPE_DISK_STRESS
Adequate amount of free disk space is not available

Action

Health daemon set the disk stress flag, mta will not process emails

Recovery

Release more disk space. Disk free space is bigger than 8G

/dev/xvda1 free inodes too small

Description

/dev/xvda1 free inodes is too small. Health daemon will set disk
stress flag. Mta will not be process email

Trigger condition

/dev/xvda1 free inodes is little then 1024 * 200

Message

/dev/xvda1 free inode is little then %d, set disk_stress

Alert

ALERT_TYPE_DISK_STRESS
Adequate amount of free disk space is not available

Action

Health daemon set the disk stress flag, mta will not process emails

Recovery

Release more disk space. Disk free space is bigger than 1024 * 300

198 ESGA v7.6 201

Troubleshooting Guide

/dev/xvda2 free inodes too small

Description

/dev/xvda2 free inodes is too small. Health daemon will set disk
stress flag. Mta will not be process email

Trigger condition

/dev/xvda2 free inodes is little then 1024 * 200

Message

/dev/xvda2 free inode is little then %d, set disk_stress

Alert

ALERT_TYPE_DISK_STRESS
Adequate amount of free disk space is not available

Action

Health daemon set the disk stress flag, mta will not process emails

Recovery

Release more disk space. Disk free space is bigger than 1024 * 300

ESGA v7.6 201 199

Troubleshooting Guide

Log stress monitor


Description

Log daemon can not send data to data base successfully. /log-defer
has files older than 3 days. Health daemon will set "log stress flag"
and generate warning messages in /var/log/messages. Mta will not
process emails

Trigger condition

/log-defer has files older than 3 days

Message

Log has files older than 3 days, set log stress

Alert

ALERT_TYPE_LOG_STRESS
Email Security log queues contain cache files older than 3 days

Action

Health daemon set the log stress flag, mta will not process emails

Recovery

Check the connection to data base. Clear all files (/log-defer) which
older than 3 days

Quarantine monitor
Description

Quarantine daemon can not send data to data base successfully.


Health daemon will set "log stress flag" and generate warning
messages in /var/log/messages. Mta will not process emails

Trigger condition

/var/spool/postfix/qd/dbsync has files older than 3 days and recent


successfully sending data time is older than 3 days
or /var/spool/postfix/qd/dbsync has files more than 100000

Message

DBsync files and sync time is older than 3 days, or files number is
bigger than %d, set quarantine stress

Alert

ALERT_TYPE_QUARANT_STRESS
Email Security quarantine queue contains cache files older than 3
days or too many files

Action

Health daemon set the quarantine stress flag, mta will not process
emails

Recovery

Check the connection to data base. Clear all files (/var/spool/


postfix/qd/dbsync)

Filter thread monitor


Description

Filter update thread or scanning thread take too long running time.
Health daemon will kill filter and generate warning message in /
var/log/messages

Trigger condition

Filter update thread or scanning thread running time is more than


20 minute

Message

Filter update running time too long. Current system time is [%lu],
filter update_start time is [%lu]
Filter update thread running time is longer than 20 minute. filter
service was killed by health daemon

Alert

No

200 ESGA v7.6 201

Troubleshooting Guide

Action

Health daemon kill filter and generate warning message in /var/log/


messages

Recovery

No need

Share memory dead lock


Description

Service share memory dead lock. Health daemon kill the service

Trigger condition

Same service and same function and same line hold the same share
memory lock for too long time

Message

Share memory [%s] is dead locked by function [%s] line [%u]


service hold share memory for too long time, it may be a dead lock.
Killed by health daemon

Alert

No

Action

Health daemon kill service and generate warning message

Recovery

No need

ESGA v7.6 201 201

Troubleshooting Guide

202 ESGA v7.6 201

Você também pode gostar