Server Cluster Best Practices Best practices for configuring and operating server clusters

Updated: January 21, 2005 Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

Best practices for configuring and operating server clusters

The following guidelines will help you effectively use a server cluster:

Secure your server cluster.

To prevent your server cluster from being adversely affected by denial of service attacks, data tampering, and other malicious attacks, it is highly recommended that you plan for and implement the security measures detailed in Best practices for securing server clusters.

Check that your server cluster hardware is listed in the Windows Catalog.
For Windows Server 2003, Enterprise Edition and Windows Server 2003, Datacenter Edition, Microsoft supports only complete server cluster systems chosen from the Windows Catalog. To see if your system or hardware components, including your cluster disks, are compatible, see Support resources. For a geographically dispersed cluster, both the hardware and software configuration must be certified and listed in the Windows Catalog. The network interface controllers (NICs) used in certified cluster configurations must be selected from the Windows Catalog. It is recommended that your cluster configuration consist of identical storage hardware on all cluster nodes to simplify configuration and eliminate potential compatibility problems.

Partition and format disks before adding the first node to your cluster.
Partition and format all disks on the cluster storage device before adding the first node to your cluster. You must format the disk that will be the quorum resource. All partitions on the cluster storage device must be formatted with NTFS (they can be either compressed or uncompressed), and all partitions on one disk are managed as one resource and move as a unit between nodes. Important

Cluster disks on the cluster storage device must be partitioned as master boot record (MBR) and not as GUID partition table (GPT) disks.

Correctly set up your server cluster's networks.

Follow the guidelines below to reduce network problems in your server cluster:

Use identical network adapters in all cluster nodes, that is, make sure each adapter is the same make, model, and firmware version.

Use at least two interconnects. Although a server cluster can function with only one interconnect, at least two interconnects are necessary to eliminate a single point of failure and are required for the verification of original equipment manufacturer (OEM) clusters.

Reserve one network exclusively for internal node-to-node communication (the private network). Do not use teaming network adapters on the private networks.

Set the order of the network adapter binding as follows: 1. 2. 3. External public network Internal private network (Heartbeat) [Remote Access Connections]

For more information, see Modify the protocol bindings order.

Manually set the speed and duplex mode for multiple speed adapters to the same values and settings. If the adapters are connected to a switch, ensure that the port settings of the switch match those of the adapters. For more information, see Change network adapter settings.

Use static IP addresses for each network adapter on each node. For private networks, define the TCP/IP properties for static IP addresses following the guidelines at Private network addressing options. That is, specify a class A, B, or C private address.

Do not configure a default gateway or DNS or WINS server on the private network adapters. Also, do not configure private network adapters to use name resolution servers on the public network; otherwise, a name resolution server on the public network might map a name to an IP address on the private network. If a client then received that IP address from the name resolution server, it may fail to reach the address because no route from the client to the private network address exists.

Configure WINS and/or DNS servers on the public network adapters. If Network Name resources are used on the public networks, set up the DNS servers to support dynamic updates; otherwise, the Network Name resources may not fail over correctly. For more information, see Configure TCP/IP settings.

Configure a default gateway on the public network adapters. If there are multiple public networks in the cluster, configure a default gateway on only one of these. For more information, see Configure TCP/IP settings.

Clearly identify each network by changing the default name. For example, you could change the name of the private network connection from the default Local Area Connection to Private Cluster Network.

Change the role of the private network from the default setting of All communications (mixed network) to Internal cluster communications only (private network) and verify that each public network is set to All communications (mixed network). For more information, see Change how the cluster uses a network.

Place the private network at the top of the Network Priority list for internal node-to-node communication in the cluster. For more information, see Change network priority for communication between nodes.

Do not install applications into the default Cluster Group. Do not delete or rename the default Cluster Group or remove any resources from that resource group.
The default Cluster Group contains the settings for the cluster and some typical resources that provide generic information and failover policies. This group is essential for connectivity to the cluster. It is therefore very important to keep application resources out of the default Cluster Group and so prevent clients from connecting to the Cluster Group's IP address and network name resources. If a resource for an application is added to this group and the resource fails, it may cause the cluster group to fail also, therefore reducing the overall availability of the entire cluster. It is highly recommended that you create separate resource groups for application resources. For more information, see Planning your groups and Checklist: Planning and creating a server cluster.

Back up your server cluster.

To be able to effectively restore your server cluster in the event of application data or quorum loss, or individual node or complete cluster failure, follow these steps when preparing backups: 1. 2. 3. Perform an Automated System Recovery (ASR) backup on each node in the cluster. Back up the cluster disks from each node. Back up each individual application (for example, Microsoft Exchange Server or Microsoft SQL Server) running on the nodes.

Note

By default, Backup Operators do not have the user rights necessary to create an Automated System Recovery (ASR) backup on a cluster node. However, Backup Operators can perform this procedure if that group is added to the security descriptor for the Cluster service. You can do that using Cluster Administrator or cluster.exe. For more information, see Give a user permissions to administer a cluster and Cluster.

For more information, see Backing up and restoring server clusters. For more information on backing up applications in a cluster, see the documentation for that application.

Maintain a backup of the RAID controller.

In a single quorum device server cluster, the RAID controller is a single point of failure. Always maintain a backup of the RAID controller configuration in case the RAID controller is replaced.

Do not use APM/ACPI Power saving features.

APM/ACPI Power saving features must not be enabled on server cluster members. A cluster member that turns off disk drives or enters "system standby" or "hibernate" mode can initiate a failure in the cluster. If multiple cluster nodes have power saving enabled, this can result in the entire cluster becoming unavailable. Cluster members must use any power scheme that sets the Turn off hard disks option to Never, for example, the Always On power scheme. For more information on choosing a power scheme (located under Power Options in Control Panel), see Choose a power scheme. For cluster nodes without Terminal Services installed, see Configure the Always On power scheme without Terminal Services installed. For cluster nodes with Terminal Services installed, see Configure the Always On power scheme with Terminal Services installed. Note

Installing Terminal Services on a system reduces the power management options available to the user. The System standby and System hibernates options are not available.

Give the Cluster service account full rights to administer computer objects if Kerberos authentication is enabled for virtual servers.
If you enable Kerberos authentication for a virtual server's Network Name resource, the Cluster service account does not need full access rights to the computer object associated with that Network Name resource. The Cluster service can use the default access rights given to members of the authenticated users group, but certain operations (for example, renaming the computer object) will be restricted. It is recommended that you work with your domain administrator to set up appropriate administration rights and permissions for the Cluster service account. For more information, see information about Kerberos authentication in Virtual servers.

Do not install scripts used by Generic Script resources on cluster disks.

It is recommended that you install script files used by Generic Script resources on local disks, not on cluster disks. Incorrectly written script files can cause the cluster to stop responding. Installing the script files on a local disk makes it easier to recover from this scenario. For guidelines on writing scripts for the Generic Script resource, see the Microsoft Platform Software Development Kit (SDK). For information on troubleshooting Generic Script resource issues, see article Q811685, "A Server Cluster with a Generic Script Resource Stops Responding" in the Microsoft Knowledge Base.

Best practices for securing server clusters

Updated: January 21, 2005 Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

Best practices for securing server clusters

Use the following guidelines to help keep your server cluster from being adversely affected by denial of service attacks, data tampering, and other malicious attacks:

Apply the latest software updates.

Apply the latest software updates, including security updates and service packs. Windows Server 2003 Service Pack 1 (SP1) contains a large number of improvements to security. We highly recommend that the version of Windows Server 2003 that you run is no earlier than Windows Server 2003 with SP1.

Obtain the latest information about security.

Review the Microsoft Web site from time to time for information about security. For example, review one or more of the following:

Information about Server Security on the Microsoft Web site (http://go.microsoft.com/fwlink/?LinkID=59992). Knowledge Base article 891597, "How to apply more restrictive security settings on a Windows Server 2003-based cluster server" on the Microsoft Support Web site (http://go.microsoft.com/fwlink/?LinkID=59993).

Restrict physical access to the server cluster and related infrastructure components to trusted personnel.
Only allow authorized personnel to have physical access to the server cluster nodes and related infrastructure components (for example, the network and storage components). For more information about standard best practices for securing servers, see Best practices for security.

Secure your cluster networks.

For public or mixed networks: Ensure that all servers (for example, infrastructure servers such as DNS or WINS servers) that are attached to the cluster subnet (or any other subnet) are secure and trusted. Also, use a firewall to protect your cluster from unauthorized client access. For private networks: Physically disjoint the private network(s) from other networks. Specifically, do not use a router, switch, or bridge to join a private cluster network to any other network. Do not include other network infrastructure or application servers on the private network subnet. If heartbeat messages that are carried by private or mixed networks are disrupted (for example, by unauthorized users flooding the cluster networks), the cluster's reaction to these false failures may affect not only groups with IP address resources, but also the health of the cluster nodes. Repeated failover of critical cluster resources (for example, resources associated with Microsoft Exchange Server) may cause clients to lose access to the cluster.

If you use Windows Firewall, use the Security Configuration Wizard to configure it.
The Security Configuration Wizard simplifies the process of configuring Windows Firewall for cluster nodes. For more information, see Using Windows Firewall with a server cluster.

Administer cluster nodes remotely only from trusted, secure computers.

If the remote computers are compromised, untrustworthy or malicious code can be started unknowingly by the cluster administration tools and adversely affect the cluster itself.

Make the Cluster service account a member of the local Administrators group on all nodes, but do not make it a member of the domain administrator group.
By giving the minimal possible administration rights and permissions to the Cluster service account, you avoid potential security issues if that account is compromised. For more information about the administration rights and permissions required by the Cluster service account, see Change the account under which the Cluster service runs.

Take steps to protect the Cluster service account, and do not use the Cluster service account to administer the cluster.
The Cluster service account has elevated administration rights and permissions. Limit the use of the account, restrict the number of people who know the account name and password, and from time to time, change the password. For more information about using another user account to administer a cluster, see Give a user permissions to administer a cluster. For detailed information about the rights and permissions required for the Cluster service account, see Change the account under which the Cluster service runs. For information about changing the password of the Cluster service account, which you can do without any downtime, see Change the Cluster service account password.

Use different accounts for the Cluster service and applications in the cluster.

This will limit exposure to the application and not the cluster account if an application account is compromised. In addition, if you use cluster.exe to change the password for the Cluster service account and one or more applications also use that account, your cluster applications may not function correctly. For more information, see Change the Cluster service account password.

Use different Cluster service accounts for multiple clusters.

This will limit exposure to one cluster only if a Cluster service account is compromised.

Remove the Cluster service account after evicting all the nodes in your cluster.
The cluster administration tools will not automatically delete the Cluster service account when all nodes have been evicted from the cluster. The Cluster service account has a high level of administration rights and permissions, and that can present potential security issues if it is compromised; it is highly recommended that you remove this account from the local Administrators group if you no longer have use for it. However, because the administrative rights and permissions for the Cluster service account are granted locally on each cluster node and not domain wide, the impact of a compromised account is limited to just the cluster nodes. For more information about deleting user accounts, see Delete a local user account.

For clusters in an Active Directory domain, enable Kerberos authentication for Network Name resources.
Kerberos authentication is much more secure than the alternative, NTLM authentication. Note that when you enable Kerberos authentication, you must add certain rights and permissions to the account that the Cluster service creates for the Network Name resource, and possibly to the Cluster service account itself. For more information, see Knowledge Base article 307532, "How to troubleshoot the Cluster service account when it modifies computer objects," on the Microsoft Support Web site (http://go.microsoft.com/fwlink/?LinkId=59994).

Audit security-related events in the cluster.

To ensure that only trusted personnel have user rights and permissions to administer the cluster and to track additions of unauthorized user accounts, audit changes to the local Administrators group. Note that, by default, when you create a server cluster or add nodes, the Cluster service account is added to the local Administrators group on each node. For more information about auditing security events, see Auditing Security Events.

Limit and audit access to shared data (for example, files and folders on cluster disks).
If you want to audit access to shared data, enable auditing on all cluster nodes. For more information, see Securing shared data in a cluster.

Limit client access to cluster resources.

Use Windows Server 2003 family security features to control client access to cluster resources as described in Limiting client access to cluster resources. Note that when you create user and group accounts through which you control access to cluster resources, you must use domain-level accounts, not local accounts, so that appropriate access is available regardless of which node currently owns a clustered resource.

Limit access to the quorum disk and ensure that the quorum disk always has sufficient free space.
To help reduce the risk of unauthorized reads and writes to the quorum disk, it is highly recommended that you give access to the quorum disk only to the Cluster service account and members of the local Administrators group. For the Cluster service to start and continue to write to the quorum log, the quorum disk must have sufficient free space. It is recommended that the quorum disk be at least 500 megabytes (MB) in size. For more information, see Checklist: Planning and creating a server cluster and Disk resource security.

Ensure that all applications running in the cluster are from a trusted source and that access to the cluster disks is restricted only to applications that are managed as a cluster resource.
For more information, see Disk resource security.

Secure script files called by Generic Script resources.

Use NTFS file-level security for execute permissions on script files and permissions for APIs called in those scripts. For more information, see Limiting client access to cluster resources.

Ensure that applications called by Generic Application resources are from a trusted source and that files, registry checkpoints, and other resources needed for those applications are in a secure location.
Applications launched as a Generic Application resource will run under the context of the Cluster service account, with its elevated administration rights and permissions. Therefore, ensure that such applications are from a trusted source. In addition, we recommend that in Cluster Administrator, when configuring the parameters for a Generic Application resource, you do not select Allow application to interact with desktop unless it is necessary. For more information, see Limiting client access to cluster resources.

Secure the cluster configuration log files created when remotely administering a cluster.

After using the New Server Cluster Wizard and the Add Nodes Wizard from a remote computer, the cluster configuration log file generated by those wizards is saved to the remote computer (at %systemroot%\system32\LogFiles\Cluster\ClCfgSrv.log). This log file contains important information about the cluster. Restrict access to that file to cluster administrators and the Cluster service account. For more information, see Set, view, change, or remove permissions on files and folders. Note If you do not have administrative rights and permissions on the node on which you are using the New Server Cluster Wizard or Add Nodes Wizard, the log file will be written to the local %Temp% directory.

Do not copy files to the local cluster directories in a majority node set cluster.
When using the majority node set model, each node maintains a copy of the quorum database in the cluster directories located at %systemroot%\Cluster\MNS.%ResourceGUID%$\%ResourceGUID%$\MSCS. If you put files in the directories below the \Cluster directory, and then delete the Majority Node Set resource, those files will be deleted by the Cluster service.

Do not change the default security settings on the HKEY_LOCAL_MACHINE system registry subtree.
By default, only members of the Administrators group in the Builtin folder and the local system account have Full Control of the HKEY_LOCAL_MACHINE system registry subtree. If this registry subtree is compromised, some resources (for example, the Generic Script resource) may fail to start. For more information about securing the system registry, see Maintain Registry Security. For more information about security in a server cluster, see Managing Security in a Cluster.