tanford Pervasive Parallelism

Lab (PPL)

Alex Aiken, Bill Dally, Pat Hanrahan,

John Hennessy, Mark Horowitz,
Christos Kozyrakis, Kunle Olukotun,
Mendel Rosenblum
April 2007
 End of Uniprocessor Performance

10000 3X
From Hennessy and Patterson, Computer
Architecture: A Quantitative Approach, 4th ??%/year
edition, October, 2006
1000
Performance (vs. VAX-11/780)

52%/year

100

The free lunch is over!

10
25%/year Now in the multicore era

1
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006
The Multicore Era

Jesse Fang , Intel Corp

 Predicting The
End of Uniprocessor Performance

10000
Sun Niagara 1

Afara Websystems ??%/year

1000
Performance (vs. VAX-11/780)

52%/year

100 Stanford Hydra CMP Project

Superior performance and

10
25%/year performance/Watt using
multiple simple cores

1
1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006
The Multicore Programming Problem

• Now average software developers will have to write

parallel programs to maintain performance growth
• Parallel programming is hard
Š First, write the sequential program
Š Find independent tasks
Š Map tasks to threads
Š Define & implement synchronization protocol
Š Race conditions & deadlock avoidance
Š Predictable & scalable performance
• Parallel programming gap: Growing divide between the
capabilities of today’s programmers, programming
languages, models, and tools and the challenges of
future parallel architectures and applications
 Pervasive Parallelism

• Enable use of parallelism beyond traditional scientific

computing
DOE
DOE ASC
ASC
Seismic and climate modeling

NIH
NIH NCBC
NCBC
Geophysics
Geophysics
PPL
PPL
AI/ML
AI/ML
Graphics
Graphics Vision
Vision
Games
Games
Web,
Web, Mining
Mining Mobile
Mobile
Streaming
Streaming DB
DB HCI
HCI
Rich virtual worlds Speech understanding
Huge datasets

• High-level domain specific languages

Š Hide parallelism from programmers (Matlab, SQL, Map-reduce, …)
 Pervasive Parallelism Lab

• Goal: Parallel Programming environment for 2012

Š Parallelism for the masses: make parallel programming
accessible to the average programmer
Š Parallel algorithms, development environments, and runtime
systems that scale to 1000s of hardware threads

• PPL is a combination of
Š Leading Stanford researchers in applications, languages,
systems software and computer architecture
Š Leading companies in computer systems, software and
applications
Š Compelling vision for creating and managing pervasive
parallelism

• kunle@stanford.edu
 Components for Bridging the
Parallel Programming Gap

Š New courses in parallel programming

Education Š Move parallelism into mainstream CS curriculum

Š Business, scientific, gaming, desktop, embedded

Parallel
Š Strong links to domain experts inside and outside
Applications of Stanford

Programming Š High-level concurrency abstractions

paradigms Š Domain specific languages

Š Hardware support for new paradigms

Architecture Š Real system prototypes for complex application
development