Google Spanner

What is it ? How does it work ? Future Scale Architecture Terms

www.semtech-solutions.co.nz

info@semtech-solutions.co.nz

Google Spanner What is it ?

The world's largest distributed database Internally used by Google Has a true time API to avoid latency problems Supports Google's Advertising business It is fault tolerant to large scale outages Offers very high availability and latency

Aiming for 99% and 50 ms

www.semtech-solutions.co.nz

info@semtech-solutions.co.nz

Google Spanner How does it work ?

Has a true time API

Atomic clocks GPS Clocks Locally determine accurate time No need for global time sync

One single global name space Data stored globally via directory namespace Uses a Paxos algorithm

www.semtech-solutions.co.nz

info@semtech-solutions.co.nz

Google Spanner Scale

How big is it , what are they aiming for ?

Aiming for 107 machines 1013 directories 1018 bytes of storage 1000's of storage locations 109 clients

Current data centres up to 100 ms apart

www.semtech-solutions.co.nz

info@semtech-solutions.co.nz

Google Spanner Architecture

A zonemaster has 100's of spanservers Zonemasters assign data to spanservers which serve clients Location proxies help clients locate spanservers Universe master displays zone status information Placement driver automates data zone movement

www.semtech-solutions.co.nz

info@semtech-solutions.co.nz

Google Spanner Architecture

www.semtech-solutions.co.nz

info@semtech-solutions.co.nz

Google Spanner Architecture

Each spanserver manages 100's of tablets Each spanserver has a paxos machine Paxos machine supports replication Lead replica has lock table Lead replica has transaction manager For transactions over multiple paxis groups

2 phase commit used For control of transactions

Coordinator leader & Coordinator slaves used

info@semtech-solutions.co.nz

www.semtech-solutions.co.nz

Google Spanner Terms

NewSQL

A modern RDBMS that scales like NoSQL but offers OLTP ACID guarantees Google's storage system built on GFS Google RDBMS for the Adwords system An algorithm for determining concensus in a network of unreliable processors

BigTable

Google F1

Paxos

www.semtech-solutions.co.nz

info@semtech-solutions.co.nz

Google Spanner Terms

RDBMS

Relational Database Management System

A highly optimized database for large storage volumes, it offers a less constrained consistency model than traditional rdbms's ACID (Atomicity, Consistency, Isolation, Durability) is a set of properties that guarantee that database transactions are processed reliably

NoSQL

ACID

OLTP

Online transaction processing

info@semtech-solutions.co.nz

www.semtech-solutions.co.nz

Google Spanner Terms

Time Synchronisation

The coordination of events to operate a system in unison Ensuring global users have a consistent view of data Clocks based upon atomic physics principles Clocks that use GPS to determine time from multiple satellite atomic clocks

Global Consistency

Atomic clock

GPS clock

www.semtech-solutions.co.nz

info@semtech-solutions.co.nz

Contact Us

Feel free to contact us at

www.semtech-solutions.co.nz info@semtech-solutions.co.nz

We offer IT project consultancy We are happy to hear about your problems You can just pay for those hours that you need To solve your problems