SAN Storage

CLARiiON Installation and Maintenance Back-End Loop Architecture
2005 EMC Corporation. All rights reserved.
FC Series
EXP Shunt
0
PRI
DAE
EXP
Shunt
0
PRI
DAE
EXP Shunt
0 SP
MID
DPE
CX Series
Shunt EXP
0
PRI
12
13
14
DAE2
Shunt EXP
0
PRI
12
13
14
DAE2
Shunt EXP
0 SP
PRI or MID
12
13
14
DAE2 (OS) or DPE
Port Bypass Device vs. Loop Switch

PBC
DAE2
Switch / Loop Switch
Switching Logic
DAE2P
Stiletto Enclosure - 4
CLARiiON Tools and Software
CLARiiON Tools and Software

Navisphere Manager
- GUI for configuration control and monitoring
Navisphere CLI
- Command line for configuration control and monitoring
CLARiiON Procedure Generator

- Utility for generating highly detailed installation and repair instructions
CAP2
- GUI utility for displaying CLARiiON Array Properties
SPLAT
- GUI utility for displaying CLARiiON event logs
FBI
- GUI utility for analysis of CLARiiON backend loops
Release Bundles
- Array based software, version determines functionality, field upgradeable
Recovery Images
- Array based packages used to recover/restore Storage Processor Boot Partitions
CLARiiON Installation & Maintenance Enhanced Install Procedure (EIP)
Enhanced Install Procedure (EIP)

The Enhanced Install Procedure (EIP) is incorporated into the CAP2
tool and replaces the earlier Healthcheck_install tool. It should be run at a very specific point in the installation of a CX-Series array. It performs the following tasks: Checks validity of supplied IP Addresses Checks connectivity to the IP Addresses Checks for privileged user access to the array Checks for any bound LUNs Resets and turns on statistics Saves and clears the event logs Cleans the dump directory on both Storage Processors Launches FBI (Fibre Backend Isolation) Prompts for power tests Runs zero_disk on all drives Executes and retrieves SPCollect on both Storage Processors Displays the results in CAP2
EIP Start
EIP Continued (2)
EIP Continued (3)
FBI Configuration
FBI Configuration (cont.)
Sizing
Initial sized screen
EIP Continued (3)
EIP Continued (4)
EIP Continued (5)
EIP Continued (6)
CLARiiON Installation and Maintenance CLARiiON Troubleshooting Tools
Objectives
Execute SPCollect and retrieve files via Navisphere CLI Execute SPCollect and retrieve files via Navisphere Manager (Release 19) Identify failing FRUs Execute CAP Version 2 Capture SPCollects via CAP Version 2 Execute the Enhanced Install Procedure Examine event logs using SPLAT Execute Host-based and Arrayside FBI
Tools
SPCollect CAP2
CLARiiON Array Properties version 2
SPLAT2
SP Log Analysis Tool version 2
FBI
Fibre Backend Isolation
spcollect Introduction
SP-based data collect script
Navisphere CLI Navisphere Manager
Revision 5 Invoked from management station
spcollect Basic Operation

Produces a data zip file, filename format is:
SPx_ChassisSerNum_SpSig_Date_Time_data.zip
Dump files not included C:\dumps folder on the SP Run spcollect on both Storage Processors
Running spcollect via Navisphere CLI
E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 spcollect -messner E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 spcollect -messner E:\Program Files\EMC\Navisphere CLI> E:\Program Files\EMC\Navisphere CLI>
Monitoring spcollect with CLI

E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 managefiles -list E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 managefiles -list Index Size in KB Last Modified Filename Index Size in KB Last Modified Filename 0 4 11/01/03 14:08:26 CIMOM_HangInfo0.txt 0 4 11/01/03 14:08:26 CIMOM_HangInfo0.txt 1 5 08/16/03 14:45:37 DeletedVolume.txt 1 5 08/16/03 14:45:37 DeletedVolume.txt 2 676 09/15/04 15:42:22 ktdump-040915-194222.txt 2 676 09/15/04 15:42:22 ktdump-040915-194222.txt 3 1002 09/15/04 16:28:59 ktdump-040915-202858.txt 3 1002 09/15/04 16:28:59 ktdump-040915-202858.txt 4 1 09/15/04 16:36:14 ktdump_log.txt 4 1 09/15/04 16:36:14 ktdump_log.txt 5 0 09/15/04 15:45:28 naviagent_Sep-15-04_19-45-15.log 5 0 09/15/04 15:45:28 naviagent_Sep-15-04_19-45-15.log 6 0 09/15/04 16:36:24 naviagent_Sep-15-04_20-36-10.log 6 0 09/15/04 16:36:24 naviagent_Sep-15-04_20-36-10.log 7 16 09/28/04 15:26:34 SPA_09-28-2004_19-26-34_runlog.txt 7 16 09/28/04 15:26:34 SPA_09-28-2004_19-26-34_runlog.txt 8 0 09/28/04 15:27:02 SPA_09-28-2004_19-26-34_stderr_log.txt 8 0 09/28/04 15:27:02 SPA_09-28-2004_19-26-34_stderr_log.txt 9 0 09/28/04 15:27:40 SPA_09-28-2004_19-26-34_stdout_log.txt 9 0 09/28/04 15:27:40 SPA_09-28-2004_19-26-34_stdout_log.txt 10 196 09/28/04 15:27:30 SPA_cfg_info.txt 10 196 09/28/04 15:27:30 SPA_cfg_info.txt 11 0 09/28/04 15:26:34 SPA_collect_name_rev.txt 11 0 09/28/04 15:26:34 SPA_collect_name_rev.txt 12 356 09/28/04 15:27:04 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_evt.zip 12 356 09/28/04 15:27:04 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_evt.zip 13 570 09/28/04 15:27:05 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_ktd.zip 13 570 09/28/04 15:27:05 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_ktd.zip 14 693 09/28/04 15:26:41 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_nav.zip 14 693 09/28/04 15:26:41 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_nav.zip 15 8 09/28/04 15:26:41 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_psm.zip 15 8 09/28/04 15:26:41 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_psm.zip 16 15 09/28/04 15:27:03 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_rtp.zip 16 15 09/28/04 15:27:03 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_rtp.zip 17 608 09/28/04 15:27:31 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_sus.zip 17 608 09/28/04 15:27:31 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_sus.zip
spcollect Completion
E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 managefiles -list E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 managefiles -list Index Size in KB Last Modified Filename Index Size in KB Last Modified Filename 0 4 11/01/03 14:08:26 CIMOM_HangInfo0.txt 0 4 11/01/03 14:08:26 CIMOM_HangInfo0.txt 1 5 08/16/03 14:45:37 DeletedVolume.txt 1 5 08/16/03 14:45:37 DeletedVolume.txt 2 676 09/15/04 15:42:22 ktdump-040915-194222.txt 2 676 09/15/04 15:42:22 ktdump-040915-194222.txt 3 1002 09/15/04 16:28:59 ktdump-040915-202858.txt 3 1002 09/15/04 16:28:59 ktdump-040915-202858.txt 4 1 09/15/04 16:36:14 ktdump_log.txt 4 1 09/15/04 16:36:14 ktdump_log.txt 5 0 09/15/04 15:45:28 naviagent_Sep-15-04_19-45-15.log 5 0 09/15/04 15:45:28 naviagent_Sep-15-04_19-45-15.log 6 0 09/15/04 16:36:24 naviagent_Sep-15-04_20-36-10.log 6 0 09/15/04 16:36:24 naviagent_Sep-15-04_20-36-10.log 7 2659 09/28/04 15:31:09 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_data.zip 7 2659 09/28/04 15:31:09 SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_data.zip
Transferring spcollect Data via CLI
E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 managefiles -retrieve -file E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 managefiles -retrieve -file SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_data.zip SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_data.zip Files selected to be retrieved are Files selected to be retrieved are SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_data.zip SPA__WRE00021500537_90a1f_09-28-2004_19-26-34_data.zip Do you want to continue (y/n)? y Do you want to continue (y/n)? y E:\Program Files\EMC\Navisphere CLI> E:\Program Files\EMC\Navisphere CLI>
spcollect Data Location
Running spcollect via Navisphere Manager
Transferring spcollect Data via Manager
Transferring spcollect Data via Manager
spcollect Data Contents
CLARiiON Installation and Maintenance Sniffer
Flare Verify
Basic operation (Verify) performed by Flare
Read 64K - each drive in LUN Check flare generated checksum is correct Check flare metadata ensures consistency of parity and data (Parity units only) XOR data and parity - ensures parity consistent with the data (Parity and mirror units only)
If any single error reported - attempts to correct Error correction can be one of the following.
Re-map bad sector and reconstruct data Correct a sector with bad checksum reconstruct sector Metadata and parity inconsistencies fixed
If multiple errors reported across a stripe

data and or parity may not be correctable message reported to Event Log
Two Levels of Verify

Sniff
Low priority check of an entire storage system Very low impact on performance
Background Verify
High priority check of a LUN, may be initiated automatically by system (non-vol verify) or manually by operator Once BV started, will run to completion and cannot be interrupted.
Sniffer Management
Navisphere CLI
- setsniffer, getsniffer
Navisphere Manager
- When binding LUN - LUN properties
Setsniffer Command Format

Release 16 and Prior
-setsniffer
lun sniff [-bv] [-bvtime priority] [-cr] [-snrate rate]
Release 19
-
setsniffer lun [-rg rg-number] [-all] sniff [-bv] [-bvtime priority] [-cr] [-snrate rate]
where: Lun rg-number Sniff -bv -bvtime priority -cr -snrate rate
Lun number Raid Group number 0 disables sniffer, 1 enables sniffer (no longer available for Release 19 arrays) Starts a full unit background verify Specifies the background verify priority Clears background verify reports Specifies the sniff rate
Note: This command must be executed from the owning SP

Getsniffer Command Format

Release 16 and Prior
- getsniffer lun [-alltot] [-curr] [-nonv] [-rec]
Release 19 - getsniffer lun [-rg rg-number] [-all] [-alltot] [-curr] [-nonv] [-rec]
where: Lun rg-number -alltot -curr -nonv -rec Lun number Raid Group number Displays historical totals Displays currently running Displays non-volatile historical totals Displays most recently completed
Note: This command may be executed from either SP

Modified Sniff Verify (Release 19)

Motivation
At the default sniff rate of one request (of 64kb) every 0.5 second, a 100GB disk takes about 9~10 days to finish one sniff verify pass. FC or ATA disks with larger capacities, Sniff Verify will take even longer to finish. Disks also seem to be becoming more susceptible to soft/hard media errors and a slow Sniff Verify doesnt help to catch disk errors in a timely fashion. These disk errors eventually might result in data corruption or loss. Modified Sniff Verify has been speed up, mainly to help detect media error on drives as early as possible. It also will no longer check data integrity and stripe consistency of the unit.

Performance
The performance goal of the Modified Sniff Verify is to keep its impact to host I/O at about the same degradation level as the current Sniff Verify with their respective default settings Default Sniff rate is changed from 64KB requests every 0.5 seconds to 512KB requests every second
Modified Sniff Drive Capacity 73 GB 181 GB 256 GB Rate = 512KB/sec 41 Hours 101 Hours 142 Hours
Current Sniff Rate = 64KB/0.5sec 158 Hours 393 Hours 556 Hours

Limitation
Sniff Verify will no long operate to check data integrity and stripe consistency of the unit. It will only issue disk verify operations to all the disks in a unit and have the disks read their data without returning them back and verifying their contents. The user can run a Background Verify on a bound unit at any time to inspect the units data integrity and coherency. Based on feedback from ongoing quality improvement teams in the DU/DL area, it was decided that the sniff verify functionality in R19 and subsequent releases will no longer be user adjustable. In normal usage, the user will not be able to disable sniffing and the rate will be controlled by the Flare driver. An undocumented setting will allow sniffing to be disabled in customer situations where performance is prioritized over data integrity.
Error Levels
Disk Recovered Error
820 (status 0x22) read successful due to internal drive heroics. 820 (status 0x3a) same success but for ATA drive.
RAID Correctable Error

820 with status 0x05 read command successful due to RAID
RAID Uncorrectable Error

0x957 (Uncorrectable Sector) 0x840 (Data Sector Invalidated)
Verify report
E:\Program Files\EMC\Navisphere CLI>navicli -h 192.168.1.23 getsniffer 1
VERIFY RESULTS FOR UNIT 1 VERIFY RESULTS FOR UNIT 1 Sniffing state: ENABLED Sniffing state: ENABLED Sniffing rate (100 ms/IO): 5 Sniffing rate (100 ms/IO): 5 Background verify time (hours): High Background verify time (hours): High Historical Total of all Non-Volatile Recovery Verifies (0 passes) Historical Total of all Non-Volatile Recovery Verifies (0 passes) ------------------------------------------------------------------------------------------------------------------------------------------Corrected Uncorrectable Corrected Uncorrectable Checksum errors 0 0 Checksum errors 0 0 Write Stamp errors 0 0 Write Stamp errors 0 0 Time Stamp errors 0 0 Time Stamp errors 0 0 Shed Stamp errors 0 0 Shed Stamp errors 0 0 Coherency errors 0 0 Coherency errors 0 0 Currently Running Full Unit Verify Currently Running Full Unit Verify ------------------------------------------------------------------------------------------------------------------------------------------Verify State: Sniff Running Verify State: Sniff Running Percent Complete: 10 Percent Complete: 10 Corrected Uncorrectable Corrected Uncorrectable Checksum errors 0 0 Checksum errors 0 0 Write Stamp errors 0 0 Write Stamp errors 0 0 Time Stamp errors 0 0 Time Stamp errors 0 0 Shed Stamp errors 0 0 Shed Stamp errors 0 0 Coherency errors 0 0 Coherency errors 0 0 Most Recently Completed Full Unit Verify Most Recently Completed Full Unit Verify ------------------------------------------------------------------------------------------------------------------------------------------Corrected Uncorrectable Corrected Uncorrectable Checksum error 0 0 Checksum error 0 0 Write Stamp errors 0 0 Write Stamp errors 0 0 Time Stamp errors 0 0 Time Stamp errors 0 0 Shed Stamp errors 0 0 Shed Stamp errors 0 0 Coherency errors 0 0 Coherency errors 0 0 Historical Total of All Full Unit Verifies (5 passes) Historical Total of All Full Unit Verifies (5 passes) ------------------------------------------------------------------------------------------------------------------------------------------Corrected Uncorrectable Corrected Uncorrectable Checksum errors 0 0 Checksum errors 0 0 Write Stamp errors 0 0 Write Stamp errors 0 0 Time Stamp errors 0 0 Time Stamp errors 0 0 Shed Stamp errors 0 0 Shed Stamp errors 0 0 Coherency errors 0 0 Coherency errors 0 0
Provide verify results for specific lun Information returned Four log sections Shows corrected and uncorrectable errors Types of errors
Verify Management with GUI
CLARiiON Troubleshooting CAP (CLARiiON Array Properties)
CAP Version 2 Introduction

Java based application
GUI Can be scripted
Data collection tool

Spcollect files Maxscript files Navicli getall files Online collection (local) Offline collection (remote)
Current version
2.15.16
CAP Version 2 Requirements

Windows NT 4.0 or later Java 1.4.2_5 or later
JRE 5 is NOT supported
128 MB of RAM
Supported Hardware
Storage Systems - FC4500, FC4700, CX Series Switches McData - DS16M, DS32M, DS64M
Brocade - DS8B, DS16B, DS16B2, DS32B2, ED12000
Hosts Requirements
- Navisphere Agent 6.2 or higher - Host must be registered with storage system
CAP Interface
Menu Bar
Main Window
Message Pane
File Menu
Edit Menu
View Menu
Tools Menu
Help Menu
Open Files
Open Files (2)
CAP Display
Tools Applications Integration
Capturing Array Data with CAP2
Capture Wizard (steps 1 and 2)
Capture Wizard (steps 3 and 4)
Capture Wizard (step 5)
Enhanced Install Procedure
EIP Wizard Configuration
EIP Wizard Execution
EIP Wizard - FBI
EIP Wizard Power Cycling
EIP Wizard Zerodisk
EIP Wizard Finish
CLARiiON Installation and Maintenance SPLAT (SP Log Analysis Tool)
SPLAT Introduction
GUI log file viewer
Launched from Start menu or from within CAP2
Requires
Java JRE 1.4.2 or later
Current Version 2.0
SPLAT Launch
SPLAT Interface
Menu Bar
Status Pane
Main Display
File Menu
Edit Menu
BookMark Menu
Mode Menu
Filter Menu
Colors Menu
Search Menu
Text input field
Selecting Filters
Displaying Filtered Data
Analyzing the Display
Display With Analysis
Analyze an Entry
Entry With Analysis
CLARiiON Installation and Maintenance CLARalert
Trapping events with Event Monitor

Receive automatic notification in the event of array or component failures
Email Pager SNMP Other
Configured Through Navisphere Manager, and run from a Windows Monitoring Station on the network One Monitoring Station can monitor multiple arrays, each with different filter settings.
Event Monitor Architecture

CX Series The Agent running on the monitoring station needs to be able to reach the Agents on the monitored arrays (or hosts, for legacy systems) through the TCP/IP network CX Series
CX Series
Ethernet TCP/IP Network
Legacy Array
FC Monitoring Station Legacy Host

CLARalert Methods
Centralized monitor Email-Home for Windows
Requires modem
Centralized monitor Email-Home with Webex Remote Access Distributed monitor Email-Home with Remote Access
Requires modem
Centralized monitor Email-Home for Solaris Centralized monitor Dial-Home for Windows
Requires modem
CLARalert Centralized Monitor Email-Home with Webex Keys to Success
Ensure that all the storage system, Windows workstation, and Host Agent environment and system requirements are met. Follow this entire procedure. Do NOT deviate from the installation procedures. Ensure that the proper Java Runtime Environment is installed on the Windows workstation. Ensure that there is a usable SMTP gateway mail server in the customers network environment. Ensure that the customers management workstation has internet access.
Installation from the CLARalert Remote Service CD
EMCRemote Introduction
Windows based client/server application
Server is SP resident Client is PC resident
Two modes of access

Dial-up LAN
Client becomes terminal on host (server)
EMCRemote Requirements
Operating Systems
EMCRemote Host/Client supports the following operating systems:
Version 4.02 Windows 98 Windows NT Workstation with SP 6A Windows NT Server with SP 6A Windows 2000 Professional with SP 2, 3, 4 Windows 2000 Server with SP 3, 4 Windows 2000 Advanced Server with SP 3, 4
Version 5
Windows XP Professional with SP1 Windows 2003 Server Enterprise Edition
Installing EMCRemote (TPM)
Installing EMCRemote (TPM) - continued
EMCRemote Installation
EMCRemote Installation
EMCRemote Execution
EMCRemote Configuration Wizard
EMCRemote Configuration Wizard (2)
EMCRemote Address Book Wizard
EMCRemote Address Book Wizard (2)
EMCRemote Address Book Wizard (3)
EMCRemote Call Manager
Installing Navisphere CLI - continued
Installing Navisphere Agent
Installing Navisphere Agent - continued
Configuring Navisphere Agent
Configuring Navisphere Agent
Portal Configuration
Portal Configuration - continued
Portal Configuration - completed
Selecting Monitored Systems
Importing the Monitoring Template
Editing the Monitoring Template
Editing the Monitoring Template - continued
Applying the Monitoring Template
Verifying The Webex Connectivity
Test Email-Home
C:\Program Files\EMC\Navisphere CLI\navicli h <SP IP> inserttesevent C:\Program Files\EMC\Navisphere CLI\navicli h <SP IP> inserttesevent
Note: Enter the storage system in clarify before executing this command
Viewing Message File
Viewing Message File - continued
Viewing Response Log
Closing Slide

SAN Storage

Enviado por

Dados do documento

Descrição original:

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

SAN Storage

Enviado por

Direitos autorais:

Formatos disponíveis

CLARiiON Installation and Maintenance Back-End Loop Architecture

2005 EMC Corporation. All rights reserved.

2005 EMC Corporation. All rights reserved.

DAE2 (OS) or DPE

2005 EMC Corporation. All rights reserved.

Port Bypass Device vs. Loop Switch

Switch / Loop Switch

2005 EMC Corporation. All rights reserved.

CLARiiON Tools and Software

2005 EMC Corporation. All rights reserved.

CLARiiON Tools and Software

CLARiiON Procedure Generator

2005 EMC Corporation. All rights reserved.

CLARiiON Installation & Maintenance Enhanced Install Procedure (EIP)

2005 EMC Corporation. All rights reserved.

Enhanced Install Procedure (EIP)

2005 EMC Corporation. All rights reserved.

EIP Continued (2)

2005 EMC Corporation. All rights reserved.

EIP Continued (3)

2005 EMC Corporation. All rights reserved.

2005 EMC Corporation. All rights reserved.

FBI Configuration (cont.)

2005 EMC Corporation. All rights reserved.

2005 EMC Corporation. All rights reserved.

Initial sized screen

2005 EMC Corporation. All rights reserved.

EIP Continued (3)

2005 EMC Corporation. All rights reserved.

EIP Continued (4)

2005 EMC Corporation. All rights reserved.

EIP Continued (5)

2005 EMC Corporation. All rights reserved.

EIP Continued (6)

2005 EMC Corporation. All rights reserved.

CLARiiON Installation and Maintenance CLARiiON Troubleshooting Tools

2005 EMC Corporation. All rights reserved.

2005 EMC Corporation. All rights reserved.

Revision 5 Invoked from management station

2005 EMC Corporation. All rights reserved.

spcollect Basic Operation

2005 EMC Corporation. All rights reserved.

Running spcollect via Navisphere CLI

2005 EMC Corporation. All rights reserved.

Monitoring spcollect with CLI

2005 EMC Corporation. All rights reserved.

2005 EMC Corporation. All rights reserved.

Transferring spcollect Data via CLI

2005 EMC Corporation. All rights reserved.

spcollect Data Location

2005 EMC Corporation. All rights reserved.

Running spcollect via Navisphere Manager

2005 EMC Corporation. All rights reserved.

Transferring spcollect Data via Manager

2005 EMC Corporation. All rights reserved.

Transferring spcollect Data via Manager

2005 EMC Corporation. All rights reserved.

spcollect Data Contents

2005 EMC Corporation. All rights reserved.

CLARiiON Installation and Maintenance Sniffer

2005 EMC Corporation. All rights reserved.

If multiple errors reported across a stripe

Two Levels of Verify