Solaris 8 Memory Architecture

Page 1 of 4

sun.
http://developers.sun.com/solaris/articles/sol8memory.html

May 04, 2010

Article

Solaris 8 Memory Architecture

August 2000
The Solaris 8 Operating Environment incorporates an improved memory architecture that improves performance for
desktop systems and servers. Better yet, it does not require any tuning, it is easier to monitor, and it provides better
memory usage reporting to help you decide if you need more memory.
Use this table to locate subjects in this article:
Where Has All My Memory
Gone?

Explains why there is little free memory reported on systems running a non-Solaris 8
operating environment such as Solaris 7.

The Least Recently Used

Paging Algorithm

Describes the criteria used to determine how primary memory and the swap device
is used by the system.

Priority Paging

Describes how to tune the virtual memory algorithm and improve performance for
certain systems.

The Solaris 8 Cyclical Page

Cache

Shows why tuning virtual memory on systems running Solaris 8 is unnecessary.

Where Has All My Memory Gone?

A common question pre-Solaris 8 users ask is "Where has all my memory gone"? The vmstat command, used to report
virtual memory statistics, often reports that free memory (measured in Kbytes in the free column) is zero or close to zero
on a pre-Solaris 8 system that has been up and running for a while.
Most likely, memory is being used to cache file system data, since the virtual memory system is shared by applications,
data, the kernel, and file system data. By default, any free memory is used to cache data read from or written to the file
system (including NFS). The size of the file system cache is dynamic -- it grows or shrinks depending on free memory.
The idea of this memory allocation scheme is to simultaneously enhance file system performance and optimize the use of
an important system resource -- virtual memory. The two computing tasks of running applications and reading and writing
data compete equally for system memory.
Generally, sharing a pool of memory is not an issue on small memory systems with low compute power, but with today's
powerful desktop systems and servers, the file system cache can overwhelm the memory pool and make application
performance suffer. Another drawback is that file system performance is tied to how quickly the virtual memory system
can free memory.
Even worse, it is difficult to measure memory usage amongst the consumers of memory on the system. The vmstat
command is often the first tool users run to examine virtual memory usage, but pre-Solaris 8 versions do a poor job of
indicating why a system is paging (running an algorithm that moves data out from physical memory to disk, and back into
physical memory from disk).
So, the question becomes: is it because the system is caching file system data, or is it because memory is a bottleneck
and the system is struggling to keep up?
Back to Top

The Least Recently Used Paging Algorithm

http://developers.sun.com/jsp_utils/PrintPage.jsp?url=http%3A%2F%2Fdevelopers.sun.com... 5/4/2010

Solaris 8 Memory Architecture

Page 2 of 4

To better understand how the virtual memory system works, let's examine the least recently used paging algorithm
(LRU). The LRU paging algorithm is controlled by the amount of available primary memory. If free primary memory falls
below an established watermark, the system attempts to reclaim memory by scanning pages and looking for pages that
have not been accessed recently. Pages that meet this criteria are moved to the free list, meaning they can be paged out
to the swap device, thus freeing the page in primary memory.
For example, here are the watermarks used by the LRU paging algorithm for a system with 128 Mbytes of primary
memory:
When Free Primary
Memory Reaches ...

The Watermark
Is ...

And the LRU Algorithm ...

2 MB

lotfree
(will run up to
120 MHz)

runs the page scanner. The page scanner scans pages in memory that
have not been used recently and moves them to the free list.

1 MB

desfree

starts conserving memory used by the kernel

512 KB

minfree

stops allocating memory for user programs and file system read
operations

Back to Top

Priority Paging
To alleviate the pressure on virtual memory induced by heavy file system activity, priority paging was introduced for the
Solaris 2.6 and Solaris 7 operating environments.
Priority paging introduces a new watermark in the LRU paging algorithm, cachefree:
When Free Primary
Memory Reaches ...

The Watermark
Is ...

And the LRU Algorithm ...

2X lotsfree

cachefree
(will run up to
120 MHz)

only frees pages with file system data that have not been referenced
recently. Executables and shared libraries are untouched

2 MB

lotsfree

runs the page scanner. The page scanner scans pages in memory that
have not been used recently and moves them to the free list

1 MB

desfree

starts conserving memory used by the kernel

512 KB

minfree

stops allocating memory for user programs and file system read
operations

Priority paging helps performance on desktop systems with more than 64 Mbytes of primary memory, especially when a
windowing system such as the Common Desktop Environment (CDE) is used.
It is also a boon for servers running online transaction processing (OLTP) applications. OLTP applications will see better
response time, as will applications that use batch processing, such as high performance computing (HPC) environments.
Also, servers running mixed applications (for example, NFS and database systems) will benefit from priority paging.
By default, priority paging is not enabled on systems running the Solaris 7 operating environment. To enable priority
paging, set the following system variable in the /etc/system file and reboot:
set priority_paging=1
Priority paging has been back-ported for systems running the Solaris 2.6 operating environment (you must first install
kernel patch 105181-09 or higher) and the Solaris 2.5.1 operating environment (you must install kernel patch 103640-25
or higher). Use the same system variable specified previously and reboot to enable priority paging on these systems.
Back to Top

The Solaris 8 Cyclical Page Cache

http://developers.sun.com/jsp_utils/PrintPage.jsp?url=http%3A%2F%2Fdevelopers.sun.com... 5/4/2010

Solaris 8 Memory Architecture

Page 3 of 4

Priority paging is not available in the Solaris 8 operating environment, for good reason. Solaris 8 introduces a new file
system caching architecture, the cyclical page cache, that eliminates most of the problems with virtual memory resulting
from file system cache activity.
Caution - Do not set the priority_paging system variable on systems running Solaris 8. Also, be sure to remove
this system variable from the /etc/system file on systems to be upgraded to Solaris 8.
The new file system caching architecture in Solaris 8 introduces a new free list, the file system free list. The file system
free list is dedicated to caching file system data only -- other objects in virtual memory such as applications, uninitialized
application data, the kernel, shared libraries, and so on are managed on a separate free list.
The result is a much cleaner architecture where the file system cache no longer competes for virtual memory resources,
since it effectively pages against itself. Heavy file system activity does not force applications out of primary memory -- it
now throws only other file system pages out.
Because of these changes, the tools used to report virtual memory statistics report significantly different values that you
should take into account when monitoring memory usage on your system.
The most obvious changes are:
Higher Page Reclaims
This is considered normal during heavy file system activity.
Higher Free Memory Values
The amount of free memory will be higher, since the free memory count now includes a large component of the file
system cache.
Zero Scan Rates
Scan rates will be almost zero, unless there is a shortage of system wide available memory. Scanning is no longer
used to replace the free list during normal file system activity.
The pre-Solaris 8 problem of not being able to differentiate between a true memory shortage or simply heavy file system
activity has been eliminated. The scan rate (sr) column in the vmstat command is now a true indication of a memory
bottleneck, and any non-zero values appearing in the sr column mean you don't have enough primary memory to keep
up with the demand on the system.
Also, the vmstat command has been updated to report virtual memory usage statistics based on the cyclical page
cache. Use the vmstat -p command to report paging activity details for applications (executable), data (anonymous),
and file system activity, as shown in the following example:

% vmstat -p 3
memory
swap free
473160 40080
416920 45168
416920 45168

re
1
0
0

mf
2
1
0

page
fr de
1
0
0
0
0
0

sr
0
0
0

executable
epi epo epf
0
0
0
0
0
0
0
0
0

anonymous
api apo apf
0
0
0
0
0
0
0
0
0

filesystem
fpi fpo fpf
11
0
0
0
0
0
0
0
0

Tuning virtual memory on systems running Solaris 8 is unnecessary -- the virtual memory system has been optimized for
the best performance straight out of the box, providing you with a compelling reason to upgrade your computing
environment (especially servers) to Solaris 8.

Related Links
A Practical Guide to Adopting the Solaris 8 Operating Environment
A white paper for Solaris or UNIX system administrators who are curious about the new Solaris 8 features.
Solaris Product Line: Solaris 8 Binaries
This page has information about the Free Solaris Binary License Program.
August 2000
Back to Top

http://developers.sun.com/jsp_utils/PrintPage.jsp?url=http%3A%2F%2Fdevelopers.sun.com... 5/4/2010

Solaris 8 Memory Architecture

Page 4 of 4

Rate and Review

Tell us what you think of the content of this page.
Excellent

Good

Fair

Poor

Comments:

Your email address (no reply is possible without an address):

Sun Privacy Policy
Note: We are not able to respond to all submitted comments.
Submit
copyright Sun Microsystems, Inc

http://developers.sun.com/jsp_utils/PrintPage.jsp?url=http%3A%2F%2Fdevelopers.sun.com... 5/4/2010