Inside PostgreSQL SharedMemory

Outline
1. File storage format
2. Shared memory creation
3. Shared buffers
4. Row value access
5. Locking
6. Other structures

FileSystemData

Postgres

/data

Pages

/base /16385

/24692
8k

Postgres

Postgres

8k

8k

8k

Data Pages

Postgres

/data

/base /16385

/24692
8k

8k

8k

8k

Postgres

Postgres
Page Header

Item

Item

Item

8K
Tuple
Tuple

Tuple

Special

FileSystemBlockTuple
Postgres

/data

/base /16385

/24692
8k

8k

8k

8k

Postgres
Page Header

Item

Item

Item

Postgres
8K
Tuple
Tuple

Tuple

Special

Tuple

Here am going to highlight a drop down which am going to eloborate

more on tuple. Am not only showing page but will also show what is there
in tuple in my next slide

FileSystemTuple
Martin

int4in(9241)
Tuple

textout()
Header

Value

OID - object id of tuple (optional)

xmin - creation transaction id
xmax - destruction transaction id
cmin - creation command id
cmax - destruction command id
ctid - tuple id (page / item)
natts - number of attributes
infomask - tuple flags
hoff - length of tuple header
bits - bit map representing NULLs

Value

Value

Value

Value

Value

It is a process which is used to start/stop process for connecting database. Also starts a OS Kernal
called fork(), fork() makes copy of postmaster and creates two new sessions postgres.

SharedMemoryCreation
k()

postmaster

This is the
program which
runs

OS Technique

for

postgres

postgres

SESSION1

SESSION2

Program (Text)

Program (Text)

Program (Text)

Data

Data

Data

Shared Memory

Shared Memory

Shared Memory

Stack

Stack

Stack

Any changes made

to share memory will
automatically seen by
other process

All programs/
Data/SM/
Stack
are separate

SharedMemory

PROC

Lightweight Locks

XLOG Buffers

Proc Array

Lock Hashes

CLOG Buffers

LOCK

Subtrans Buffers

PROCLOCK

Two Phase Structs

Auto Vacuum
Btree Vacuum

Multi XACT Buffers

Statistics

Background Writer

Synchronized Scan

Buffer Descriptors
Shared Buffers

Semaphores

Shared Invalidation

The way SM works is that basically have SB, SB is copied in SM

If any changes in pages is directly copied into SB. If multiple users are changing the locking comes into picture.

SharedBuffers

Buffer Descriptors

Pin Count prevent page replacement

LWLock for page changes

8k

8k

8k
Shared Buffers

read()

Page Header

Item

Item

Item

write()
Postgres

/data /base /16385 /24692

8K
8k 8k 8k 8k

Tuple
Tuple

Tuple

Special

Postgres

Postgres

14

HeapTuples

8k

8k

8k
Shared Buffers

Page Header

Item

Item

Item

8K
Tuple
Tuple

Tuple

Special

Martin

int4in(9241)

HeapTuple

Tuple

textout()
Header

Value

Value

Value

Value

Value

Postgres

Value

C pointer
OID
xmin
xmax
cmin
cmax
ctid
natts

object id of tuple (optional)

creation transaction id
destruction transaction id
creation command id
destruction command id
tuple id (page / item)
number of attributes

infomask

tuple flags

hoff

length of tuple header

bits

bit map representing NULLs

Any users can make modifications, but how do you do that. How to prevent them performing at the same time and accessing sahred memory.
The way is done with locking

Test And SetLock

Can Succeed Or Fail
If you use single lock as below it failes because, if two persons
looking to access te data the second user will fail

This person got the lock

succeded

0/1

Two different people looking to obtain a lock.

But here he did not get the lock and failed.

Success

Failure

Was 0 on exchange

Was 1 on exchange
Lock already taken

It is more sophisticated lock called spinlock. If it fails comes back and

tries once again because of sleep.

SpinLock
AlwaysSucceeds

0/1

Sleep of increasing duration

Success

Failure

Was 0 on exchange

Was 1 on exchange
Lock already taken

Spinlocks are designed for short-lived locking operations, like access to

control structures. They are not be used to protect code that makes
kernel calls or other heavy operations.

Light WeightLocks
Sleep On Lock

PROC

Lightweight Locks

XLOG Buffers

Proc Array

Lock Hashes

CLOG Buffers

LOCK

Subtrans Buffers

PROCLOCK

Two Phase Structs

Auto Vacuum
Btree Vacuum

Multi XACT Buffers

Statistics

Background Writer

Synchronized Scan

Shared Invalidation

Buffer Descriptors
Shared Buffers

Semaphores

Light weight locks attempt to acquire the lock, and go to sleep on a

semaphore if the lock request fails. Spinlocks control access to the light
weight lock control structure.

WE have finally database locks. This would be for select/update.

Database Object Locks

PROC

PROCLOCK

LOCK
Lock Hashes

Every processor has got proc structure and contains session ID and
info of transaction.

Proc

PROC
empty

used

Proc Array

used

empty

used

empty

Other SharedMemoryStructures

PROC

Lightweight Locks

XLOG Buffers

Proc Array

Lock Hashes

CLOG Buffers

LOCK

Subtrans Buffers

PROCLOCK

Two Phase Structs

Auto Vacuum
Btree Vacuum

Multi XACT Buffers

Statistics

Background Writer

Synchronized Scan

Buffer Descriptors
Shared Buffers

Semaphores

Shared Invalidation