Unit 6

Storage and Other I/O Topics Lecture 01 Storage

6.1 Introduction

Introduction

I/O devices can be characterized by

Behavior: input, output, storage, network Partner: human or machine Data rate: bytes/sec, transfers/sec

Storage I/O bus connections I/O Management

Chapter 6 Storage and Other I/O Topics 2

Outline

Storage
Disk Flash

Interconnecting Components and I/O bus File System & Web Benchmarks I/O System Design
Chapter 6 Storage and Other I/O Topics 3

I/O System Characteristics

Dependability is important

Particularly for storage devices Latency (response time) Throughput (bandwidth) Desktops & embedded systems

Performance measures

Mainly interested in response time & diversity of devices

Servers

Mainly interested in throughput & expandability of devices

Chapter 6 Storage and Other I/O Topics 4

Dependability

Service accomplishment Service delivered as specified

Fault: failure of a component

May or may not lead to system failure Fault-tolerant system

working

Restoration repair

Failure

Repair rate m Service interruption Deviation from specified service

Failure rate l

failed Finite State Machine

Dependability Measures

Reliability R(t): Probability to the first failure in a given time period MTTF: Mean Time To Failure = R(t)dt
0

Service interruption: mean time to repair (MTTR) Mean time between failures

MTBF = MTTF + MTTR

Availability = MTTF / (MTTF + MTTR) Improving Availability

Increase MTTF: fault avoidance, tolerance, forecasting Reduce MTTR: improved tools and processes for diagnosis and repair

6.3 Disk Storage

Disk Storage
Nonvolatile, rotating magnetic storage

Each sector records

Sector

Sector ID Data (512 bytes, 4096 bytes proposed) Error correcting code (ECC)

Used to hide defects and recording errors Fault tolerance technique

Synchronization fields and gaps

Disk Sectors and Access

Access time to a sector involves

Queuing delay if other accesses are pending Seek: move the heads to select tracks Disk rotational latency to select sector Data transfer time to read the sector Controller overhead

Sectors

Disk

Tracks

Platter Reading arms and heads

Platters

Track Chapter 6 Storage and Other I/O Topics 8

Disk Performance Example

Given

512B sector, 15,000rpm, 4ms average seek time, 100MB/s transfer rate, 0.2ms controller overhead, idle disk (no waiting queue) seek time = 4ms avg rotational latency = / (15,000/60) = 2ms data transfer time 512 / 100MB/s = 0.005ms controller delay = 0.2ms Total = 6.2ms

Average read time

If actual average seek time, considering locality, is 1ms

Average read time = 3.2ms

Disk Performance Issues

Manufacturers quote average seek time

Based on all possible seeks Locality and OS scheduling lead to smaller actual average seek times: there are many patented algorithms of reducing seek time!

Disk controller hides specific details of physical sectors on disk

Present logical (standard) sector interface to host SCSI, ATA, SATA Pre-fetch sectors in anticipation of access Avoid seek and rotational delay

Disk drives include caches

6.4 Flash Storage

Flash Storage

Nonvolatile semiconductor storage

100 1000 faster than disk Smaller, lower power, more robust But more $/GB (between disk and DRAM)

Chapter 6 Storage and Other I/O Topics 11

Flash Types (Solid State Drive)

NOR flash: bit cell like a NOR gate

Random read/write access Used for instruction memory in embedded systems Denser (bits/area), but block-at-a-time access Cheaper per GB Used for USB keys, media storage, Not suitable for direct RAM or disk replacement Wear leveling: remap data to less used blocks

NAND flash: bit cell like a NAND gate

Flash bits wears out after 1000s of accesses

Chapter 6 Storage and Other I/O Topics 12

NOR vs. NAND Flash

Source: Toshiba America

Outline

Storage Interconnecting Components and I/O bus IO Management File System & Web Benchmarks I/O System Design

Chapter 6 Storage and Other I/O Topics 14

Interconnecting Components

Need interconnections between

CPU, memory, I/O controllers, devices

Bus: shared communication channel

Parallel set of wires for data, address, controls Wires are shared in different time slots One component can send at one time; All components will receive the data (broadcasting). If the destination address is different, discard data. Collision will occur if two components send at the same time; Then, resend. Coordination among control and data lines necessary Can become a bottleneck
Wire length, number of components connected

Performance limited by physical factors

Bus Types

Synchronous Buses: Short, high speed, clocked, taking an action in each clock Processor-memory bus Graphic bus

Asynchronous Buses: Longer, slower, and allowing multiple devices, using handshaking

I/O buses Backplane Bus connecting bridges to IO Busses

Bus Types and Organization

Processor-Memory Bus
Backplane Bus
Control lines Data lines

CPU

SRAM

High-speed bus bridge

Bus bridge I/O Bus I/O Bus

Synchronous Bus

DRAM

Graphic Card ...

Asynchronous Bus

...

Bus Types and Organization Example

ProcessMemory Bus
FSB

L2 Cache

Synchronous Bus

Disk CD Keyboard

Asynchronous Bus

Mouse

Bus Slots Connecting to Device

Bus Signals and Synchronization

Data lines

Carry address and data Multiplexed (shared) or separate Indicate data type, synchronize transactions Uses a bus clock Use for fast devices Uses request/acknowledge control lines for handshaking Use for slow speed devices

Control lines

Synchronous Bus Protocol

Asynchronous Bus Protocol

Synchronous Separate Address and Data Lines (Write)

Address

Data Write

Clock

Synchronous Separate Address and Data Lines (Read)

Address

Data Read DataReady

Clock

Synchronous Communication Protocol Analogy

Teaching a large class Clock Instructor Students

week1
week2 week3 week4 week5 week6 week7 week8 week9 week10

Prepare assignment
Assignment 1 release Prepare assignment Assignment 2 release Prepare assignment Assignment 3 release Prepare assignment Assignment 4 release Prepare assignment Assignment 5 release Do assignment 1 Assignment 1 submission Do assignment 2 Assignment 2 submission Do assignment 3 Assignment 3 submission Do assignment 4 Assignment 4 submission Do assignment 5

week11

Prepare assignment

Assignment 5 submission

Asynchronous Communication Protocol Analogy

Individual Learning Week1 Instructor Prepare assignment 1 Assignment 1 release Student

week2

Prepare assignment 2 Assignment 2 release

Do assignment 1 I am nearly completed Assignment 1 submitted

week7

Prepare assignment 3 Assignment 3 release

Do assignment 2 I am nearly completed Assignment 2 submitted

Do assignment 3 I am nearly completed Assignment 3 submitted Do assignment 4 I am nearly completed

week 9

Prepare assignment 4 Assignment 4 release

I/O Bus Examples

Firewire USB 2.0 PCI Express Serial ATA Serial Attached SCSI

Intended use External

Devices per channel Data width Peak bandwidth Hot pluggable Max length 63 4 50MB/s or 100MB/s Yes 4.5m

External
127 2 0.2MB/s, 1.5MB/s, or 60MB/s Yes 5m

Internal
1 2/lane

Internal
1 4

External
4 4 300MB/s

250MB/s/lane 300MB/s 1, 2, 4, 8, 16, 32 Depends 0.5m Yes 1m

Yes 8m

Standard

IEEE 1394

USB Implementers Forum

PCI-SIG

SATA-IO

INCITS TC T10

Chapter 6 Storage and Other I/O Topics 25

Bus Arbitration
Multiple master devices are connected to a single bus, which can start to send a message at the same time. Arbitration is needed to decide which device is going to use the bus. Bus arbitration schemes A single master (processor): All devices send requests to the processor, which decides which device will use the bus Daisy chain arbitration Centralized parallel arbitration Distributed arbitration by collision detection: Ethernet

Daisy Chain Arbitration

Advantages: simple to implement high utilization of bus (no conflict overhead) extendable Disadvantages: unfairness: low priority devices may never get a chance priority is position-related unreliable (a broken device can block other devices)

Centralized Parallel Arbitration

Advantages: a broken device can not block other devices can design flexible priorities high utilization of bus (no conflict overhead) disadvantages: not easy to extend once the design is completed vulnerable for single-point-of-failure (arbiter)

Distributed Arbitration

Advantages: less vulnerable for single-point-of-failure fairness among the devices (contention protocols) extendable disadvantages: lower utilization of bus (conflict overhead)

Distributed Bus Arbitration/LAN Protocols

ALOHA broadcast protocol (Contention Protocol)

(1) A station transmits whenever it has data,

(2) Check whether the feedback = sent message?

(3) If not (collision), wait a random time, transmit again

IEEE 802.3 and Ethernet Protocol

1. listen to the channel, if the channel is free, then it transmits, else wait until its free, 2. earlier collision detection and abortion 3. by collision, wait a random time, goto (1)
The random time is defined by binary exponential backoff: 1. by 1st collision, wait 0 or 1 slot (= 2t, the round-trip time);

2. by 2nd collision, wait 0, 1, 2, or 3 slots;

3. by 3rd collision, wait 0, 1, 2, 3, 4, 5, 6, or 7 slots; 4. by 4th collision, wait 0 , 1, 2, 3, ..., or 15 slots;

5.
6. by nth collision, wait 0 , 1, 2, 3, ..., or 2n-1 slots;

IEEE 802.11 Wireless Ethernet Protocol

Can be configured into two modes

ad hoc mode

infrastructure mode

Different radios and bandwidths

802.11a:

radios transmit at 5 GHz and send data up to 54

Mbps, with distances of about 60 feet

802.11b radios transmit at 2.4 GHz and send data up to 11 Mbps, with distances of about 300 feet. More: 802.11c, 802.11d, 802.11e, 802.11f, 802.11g ...

IEEE 802.11 Ad Hoc Mode Devices communicate with each other directly

IEEE 802.11 Infrastructure Mode

Devices communicate with each other via base stations, similar to cellular phone networks

station

wired network

station

Outline

Storage Interconnecting Components and I/O bus I/O Management I/O Performance

Chapter 6 Storage and Other I/O Topics 35

6.6 Interfacing I/O Devices

I/O Management

I/O is mediated by the OS

Multiple programs share I/O resources

Need protection and scheduling Same mechanism as exceptions OS provides abstractions to programs Using library functions to access I/O, e.g., system calls, which are OS functions, in MARS simulator

I/O causes asynchronous interrupts

I/O programming is fiddly (small, but awkward)

36

I/O Registers and Commands

I/O devices are managed by I/O controller hardware

Transfers data to/from device Synchronizes operations with software Cause device to do something Indicate what the device is doing and occurrence of errors

I/O controller
Status register

Command registers

Status registers

CPU data

busy ready error

Data register Command reg

Data registers

Write: transfer data to a device Read: transfer data from a device

Device

I/O Register Mapping

Memory mapped I/O (MIPS)

I/O registers are addressed in same space as memory (considered as memory locations) Address decoder distinguishes between them OS uses address translation mechanism to make them only accessible in kernel mode, not available in the user mode. Drivers are running in kernel mode! No additional I/O instructions are required Separate instructions to access I/O registers Can only be executed in kernel mode Example: x86
38

I/O instructions (Intel)

Typical x86 PC I/O System

Other I/O Devices

Interface Devices to CPU

Polling with loop-waiting: while (not busy) do send data Polling with periodically checking the busy bit and possible multitasking Interrupt-driven: More efficient if requests are not frequent; less efficient if requests are frequent, I/O processor: delegating I/O from CPU
I/O controller
Status register

I/O controller
Ready for next data
Status register

periodically CPU data

busy ready error

Data register

CPU data

busy ready error

Data register

Polling

Command reg

Interruptdriven

Command reg

Device

Device

Example: Polling
CPU wants to transfer 1024 words to a device (e.g., printer)
// CPU process counter = 1024 L0 load the status register of the device test the busy bit if busy = true goto L0 // or call Sleep(500) in multitasking OS else store a word into the data register of the device set data-ready = true counter = counter - 1 CPU I/O controller if counter = 0 exit else goto L0
Status register

// Device process L1 set busy bit to false L2 test data-ready bit if data-ready = false goto L2 set busy bit to true print the data byte0, byte1, byte2, byte3 set data-ready = false goto L1

busy ready error

data
Data register Command reg

Printer

Device Control Example

Which device should we use polling? Which device should we use interrupts? What about a computer science doctors office hours and a medical doctors office hours?
Sonar range sensor Touch sensor

42

I/O Data Transfer with DMA

Polling and interrupt-driven I/O

CPU transfers data between memory and I/O data registers Time consuming for high-speed devices OS provides starting address in memory and package size I/O controller transfers to/from memory autonomously Controller interrupts on completion or error
43

Direct Memory Access (DMA)

DMA and I/O Processor

DMA: Direct Memory Access, a specialized processor that transfers data between memory & device, e.g., printer or disk. CPU sends DMA address and # of bytes. After completion, DMA sends done.

I/O processor (I/O controller or channel controller): is

more intelligent than DMA. It handles multiple inputs in a queue and multiple outputs to a number of devices. An I/O processor normally doesnt convert data, only transfer.

DMA and I/O Processor

CPU Mem

CPU

Mem
address address address address address
dst/src dst/src dst/src dst/src dst/src

address size DMA data

size size size size size

DMA
Device Device

data
I/O Processor Device . . . Device