Sun Confidential: Need to Know

X4-2B Services TOI Architecture and features

Revision 0.2
Wei Lu

NDA Notification
This presentation contains information related to projects that are currently
in planning and/or development stages. This information represents the current
intentions of Oracle, Inc. However, all aspects of these projects, including, but not
limited to funding, availability, shipping dates, configurations, capacities, performance,
and all other characteristics are subject to change
and/or cancellation without notice.

This material is confidential to Oracle and should be disclosed to non-employees only

under the terms of an executed confidential-disclosure agreement.
Distribution of this material to Oracle Field Representatives should be done through
the GSO CDA process. Under no circumstances should these roadmaps be given
directly to a Oracle Field Representative.
The information contained herein is considered privileged, confidential
and protected from disclosure. If you are not the intended recipient
or their employee/agent responsible for its delivery, you are requested
to refrain from any duplication or distribution of this message or its
contents, in respect of the rights of the designated recipient and
sender, and to delete it from your computer.

2012 Oracle Corporation. - Proprietary and Confidential

Note

The following is intended to outline our general product

direction. It is intended for information purposes only, and
may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality,
and should not be relied upon in making purchasing
decisions. The development, release, and timing of any
features or functionality described for Oracles products
remains at the sole discretion of Oracle.

2012 Oracle Corporation. - Proprietary and Confidential

Agenda

Mensa Overview
Intel Romley Platform
Mensa Architecture
Mensa Components
Mensa Remote Management
Appendix

2012 Oracle Corporation. - Proprietary and Confidential

Mensa Overview

X4-2Bserver module (Mensa+) for

the Sun Blade 6000 chassis:
A90-B-N (aka C10+)

A90-D (aka C10 M2)

Sun Blade 6000 chassis can hold
up to ten server modules

2012 Oracle Corporation. - Proprietary and Confidential

Oracle X4-2B Blade Server

Compute
2x Intel Ivy Bridge-EP CPUs
1x PCH southbridge
24x DDR3 RDIMMs

Up to 768 GB of memory

I/O
2x PCIe 2.0 EM slots (8x lane per EM)
2x PCIe 2.0 FEM slots (8x Lane per FEM)
2x GbE ports (Powerville)
1x REM Slot (8x lane/slot)
4x USB 2.0 (two front, two internal)
2x USB 2.0 dongle
1x VGA
AST2300 BMC with remote KVMS
1x Serial Port

2-Socket, 24DIMM EnterpriseClass System Blade

2012 Oracle Corporation. - Proprietary and Confidential

4x hot swap 2.5 SAS/SSD drive bays

4x SAS ports (REM)

Mensa Board Layout

2012 Oracle Corporation. - Proprietary and Confidential

Mensa vs Mensa+

Mensa
Mensa+
Sandy Bridge-EP CPU
Ivy Bridge-EP CPU
Socket R
Socket R
Max N of cores - 8
Max N of cores - 12
24 DDR3 DIMMs, 384GB Max 24 DDR3 DIMMs, 768GB Max

2012 Oracle Corporation. - Proprietary and Confidential

Intel Product Name Decoder

Before Launch Date
Ivy Bridge-EP
Socket R

After Launch Date

Intel(R) Xeon(R)
Intel(R) Xeon(R)
Intel(R) Xeon(R)
Intel(R) Xeon(R)
Intel(R) Xeon(R)

Patsburg - J

Powerville

Romley-EP Platform

E5-2697 v2 12-core 2.7GHz 130W

E5-2690 v2 10-core 3.0GHz 130W
E5-2650 v2 8-core 2.6 GHz 95W
E5-2630 v2 6-core 2.6GHz 60W
E5-2609 v2 4-core 2.5GHz 60W

Intel(R) C600 series chipset

I350-AM2
Intel(R) Xeon(R) processor E5-2600 v2 product family

2012 Oracle Corporation. - Proprietary and Confidential

10

Mensa+ Architecture

2012 Oracle Corporation. - Proprietary and Confidential

11

Mensa Block Diagram 2 Socket Configuration

2012 Oracle Corporation. - Proprietary and Confidential

12

Mensa Front View

1 Locate Button and LEDWhite
2 Ready to Remove LEDBlue
3 Fault- Service Action Required LEDAmber

4 PowerOK LEDGreen
5 On/Standby Power button
6 NMI Reset button (Service only)
7 RFID tag (with serial number)

8 USB ports (2)

9 Universal Connector Port (UCP)
10 Hard Disk Drive (HDD0) *
11 Hard Disk Drive (HDD1) *

12 Hard Disk Drive (HDD3) *

13 Hard Disk Drive (HDD2) *
14 Ejector levers
* HDDs are optional
2012 Oracle Corporation. - Proprietary and Confidential

13

Mensa Rear View

1 Power connector
2 I/O Connector
3 Rear cover (remove before
inserting into chassis)

4 USB flash drives 2, 3

2012 Oracle Corporation. - Proprietary and Confidential

14

Mensa - Top View

CPU1

Powerville

FEM
Slots

REM
CPU0

Disk
Bays
USB
(Internal)

REM
SAS

FPGA

2012 Oracle Corporation. - Proprietary and Confidential

PCH

15

Mensa+ Components

2012 Oracle Corporation. - Proprietary and Confidential

16

Mensa Components

2012 Oracle Corporation. - Proprietary and Confidential

1 HDD/SSD
2 REM
3 CPU
4,6 Heatsink
5 FEM
7 USB stick
8 DIMM
9 Battery
10 Enclosure

17

CRU vs. FRU, Hot vs. Cold

Hot service CRUs:
Drives
EMs

Cold service CRUs

DIMMs or fillers
REM
FEMs
System Battery
SAS Cable
Internal USB sticks

FRUs
Motherboard
Processor and heat sink

2012 Oracle Corporation. - Proprietary and Confidential

18

Memory Architecture

Each processor has 4 DDR3 memory channels (A,B,C,D)

Each DDR3 memory channel supports up to 3 DIMMs for
a total of 12 DIMMs per processor
X4-2B supports 1 DIMM per channel, 2 DIMMs per
channel, and 3 DIMMs per channel across both sockets.

2012 Oracle Corporation. - Proprietary and Confidential

19

Mensa Memory Bank Layout

2012 Oracle Corporation. - Proprietary and Confidential

20

Supported DIMMs

DDR3 LVDIMMs (1.35V)

8 GB DDR3-1600 DIMM
16 GB DDR3-1600 DIMM
DIMM slot filler panels
32 GB DDR3-1600 LRDIMM (Post RR)
(1DPC:1600 2DPC:1600 3DPC:1066)

2012 Oracle Corporation. - Proprietary and Confidential

21

Memory Population

2012 Oracle Corporation. - Proprietary and Confidential

22

Memory Fillers (No Need in X4-2B)

2012 Oracle Corporation. - Proprietary and Confidential

23

REMs

Niwot REM (LSI) (SG-SAS6-R-REM-Z)

SAS Eight Port (2 X4) 6Gb RAID
PCI-E Gen2 HBA, Coyote
Replacement (RAID 0, 1, 10, 5, 6)
Erie REM (LSI) (SG-SAS6-REM-Z)
SAS Eight Port (2 X4) 6Gb RAID
PCI-E Gen2 HBA.

2012 Oracle Corporation. - Proprietary and Confidential

24

FEMs
Dual 10Gb Ethernet, PCIe 2.0 (Intel Niantic - X4871A-Z)
Required to support OPUS
Passthru FEM: (aka Hunter - X4263A-N)
Nalia FEM (7100283- ATO, 7100633 - PTO)
For SOL_NEM

2012 Oracle Corporation. - Proprietary and Confidential

25

PEMs

PEMs
Combo GbE/FC
Emulex 2-port 8Gb FC, 2-port GbE
Qlogic 2-port 8Gb FC, 2-port GbE

10GbE/1GbE
Quad GbE Copper / Fiber PCIe 2.0 (Intel, Powerville)
Intel 2-port SFP+ 10GbE NIC
Intel Niantic Copper: Dual port 10GbE Twinville

SAS2
Erie-EM (No RAID. Uses IT fw)

IB PCI-E
QDR IB-HCA PCI-E Gen2 Mellanox ConnectX-2
Based

FC/FcoE (Ganymede)
Emulex 2 Port 16Gb FC or 10Gb FcoE
Qlogic 2 Port 16Gb FC or 10Gb FcoE

2012 Oracle Corporation. - Proprietary and Confidential

26

NEMs

Sun Blade 6000 10-Port GbE Pass-Thru NEM (X4250A-N)

Sun Blade 6000 Ethernet Switched NEM 24p 10GE (OPUS - X2073A-N)
Sun Blade 6000 Virtualized 40GbE NEM (SOL_NEM - 7100090)

2012 Oracle Corporation. - Proprietary and Confidential

27

Internal Storage - HDDs

Mensa supports up to 4
Internal HDDs
300GB/600GB/1.2TB 10000
rpm 2.5-inch SAS2 HDDs
with Marlin bracket
400GB 2.5-inch SATA Solid
State Disks (SSDs) with
Marlin bracket
HDD Filler should be used
in case when HDD is not
installed

2012 Oracle Corporation. - Proprietary and Confidential

28

Internal Drives Mixing Rules

Mixing of hard disk technologies (e.g., SAS HDD and

SATA SSD) is allowed within the 4 internal drive slots
as long as they are in separate RAID volumes, or if
treated as JBOD.
Within the RAID volume, the drives must be identical
(capacity and type)
For example, a RAID volume with two 300GB/10K
SAS HDDs is supported, but a RAID volume with a
300GB/10K SAS HDD and a 600 GB/10K SAS HDDs
is not supported.

2012 Oracle Corporation. - Proprietary and Confidential

29

Cabling (Not support)

2012 Oracle Corporation. - Proprietary and Confidential

30

USB

2 Front USB2 slots

2 Internal USB2 slots
Supplied USB Drive
USB Thumbdrive (USB 2.0
compliant) 4GB, MLC, USB-2,
SBC, RoHS-6 Compliant

2012 Oracle Corporation. - Proprietary and Confidential

31

UCP Dongle

1 Video cable
2 Serial console
3 USB (x2)

2012 Oracle Corporation. - Proprietary and Confidential

32

Mensa Front Panel LEDs & BUTTONS

N Name

Color

1 Locate LED/button

White

2 Ready to Remove LED

Blue

3 Service Action Required LED

Amber

4 Power/OK LED

Green

5 Power button

None

6 NMI button

None

2012 Oracle Corporation. - Proprietary and Confidential

33

Fault Tracing

Fault Remind button helps

identify or locate a faulty
component even after the
blade has been removed
from chassis
Amber fault LED of these
components will be lit by
Fault Remind button
Processors
DIMMs

2012 Oracle Corporation. - Proprietary and Confidential

34

Blade cooling

No fans are used on the Mensa blade, cooling is provided by

the C10 chassis.
Airflow is front to back, in at the front panel, through the DIMM
memory arrays and CPUs, and out the back
The following blade components have heatsinks:
Processors
Patsburg
Powerville

2012 Oracle Corporation. - Proprietary and Confidential

35

Mensa Remote Management

2012 Oracle Corporation. - Proprietary and Confidential

36

Service Processor

Based upon the Aspeed AST-2300 integrated processor

BMC provides industry standard IPMI feature set
Remote KVMS over IP
Serial port
Ethernet access to SP through a dedicated C10 10/100BaseT management port,
and optionally through one of the host GbE ports through NCSI interface
Front side KVM with Dongle
Support for in-system upgrade (BIOS, CPLD, HPC, etc.)
256MB DDR3 Memory
32MB Flash

2012 Oracle Corporation. - Proprietary and Confidential

37

Oracle ILOM Key Functions

Management Interfaces

CLI, IPMI

Firmware Updates
Remote Host Management
Inventory and Component Management
System Monitoring and Alert/Fault Management
Thermal and Fan Control
User Account Management
Power Consumption Management

2012 Oracle Corporation. - Proprietary and Confidential

38

Appendix

2012 Oracle Corporation. - Proprietary and Confidential

39

Mensa
SMBUS
BLOCK

2012 Oracle Corporation. - Proprietary and Confidential

40

Memory Population Rules

Rule1: Always populate the channels as follows
* Fill up all Blue Sockets
* Next, fill up all White sockets
* Finally, fill up the Black sockets

This is known as the "fill farthest" approach to DIMM install. Always

populate the DIMMs furthest from the processor (blue sockets) first.
Example: A processor with 4 DIMMs. Install 4 DIMMs in Blue sockets
D0, D3, D6 and D9.
Example: A processor with 8 DIMMs. Install 4 DIMMs in Blue sockets
D0, D3, D6 and D9, and 4 more DIMMs in the WHITE sockets D1,
D4, D7 and D10.
Example: A processor with 12 DIMMs. Install DIMMs in all the sockets

2012 Oracle Corporation. - Proprietary and Confidential

41

Memory Population Rules, 2

Rule 2: Memory symmetry across processors is required.
Processor #1 memory must match processor #0 memory, in
placement, type, size, capacity, frequency and voltage.
Rule 3: Each processor can support single DIMM, two DIMMs,
three DIMMs or four DIMMs per color socket set

Rule 4: Within every set of 4 DIMMs (e.g., blue socket set, white
socket set, black socket set), mixing/matching of different size,
memory speed and voltage is NOT ALLOWED
Memories in D0, D3, D6 and D9 must be all the same.
Next, memories in D1, D4, D7 and D10 must be all the same, etc.
While doing this, Rule 2 must be maintained.

2012 Oracle Corporation. - Proprietary and Confidential

42

Memory Population Rules, 3

Rule 5: Mixing and matching of different size, voltage and speed
across different 4 DIMM sets is allowed.
For example, memories in D0, D3, D6, D9 (blue sockets)
must have the same size. But, they do not have to match memory
size/voltage/speed in D1, D4, D7, D10 (white sockets).
Note: When mixing speed across different 4 DIMM sets, all
memory will be tuned to the slower speed.
Rule 6: Mixing of different DIMM technology is not supported.
The blade must be all RDIMMs.
Rule 7: Each processor can support a maximum of:
* 12 single-rank (SR) or dual-rank (DR) DIMMs; or
* 12 Load Reduced (LR) DIMMs;

2012 Oracle Corporation. - Proprietary and Confidential

43

Memory Population Rules, 4

Rule 8: For maximum performance, apply the following rules:
* DIMMs can run in one of the 2 speeds: 1600 and
1067MHz.
DIMM speed rules are: (SR=single rank; DR=dual rank;
LR=Loading reduced)
* Memory bus speed also depends on processor SKU.
1. 1DPC or 2DPC (SR/DR/LR) = 1600 MHz
2. 3DPC (SR/DR/LR) = 1067 MHz
Rule 9: Mixing different size DIMMs:
Farthest socket on a channel gets highest density DIMM,
second socket gets the next highest density, etc

2012 Oracle Corporation. - Proprietary and Confidential

44

Sun Confidential: Need to Know