The vmstat command (short for virtual memory statistics) is a built-in monitoring utility in Linux. Users can observe system activity virtually in real-time by specifying a sampling period. The command helps you identify performance bottlenecks and diagnose problems in your system. The command is used to obtain information about following
- memory,
- system processes,
- paging,
- interrupts,
- block I/O,
- disk, and
- CPU scheduling
How to install vmstat?
$ sudo apt install sysstat [On Debian, Ubuntu and Mint]
$ sudo yum install sysstat [On RHEL/CentOS/Fedora and Rocky Linux/AlmaLinux]
$ sudo emerge -a app-admin/sysstat [On Gentoo Linux]
$ sudo pacman -S sysstat [On Arch Linux]
$ sudo zypper install sysstat [On OpenSUSE]
$ vmstat -a
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free inact active si so bi bo in cs us sy id wa st
0 0 0 128240 818236 797612 0 0 4 18 127 244 0 0 100 0 0
$ vmstat -s
2020092 K total memory
280684 K used memory
797668 K active memory
818204 K inactive memory
128240 K free memory
44440 K buffer memory
1566728 K swap cache
0 K total swap
0 K used swap
0 K free swap
33428 non-nice user cpu ticks
3750 nice user cpu ticks
10322 system cpu ticks
17519407 idle cpu ticks
2294 IO-wait cpu ticks
0 IRQ cpu ticks
146 softirq cpu ticks
4287 stolen cpu ticks
752241 pages paged in
3243317 pages paged out
0 pages swapped in
0 pages swapped out
22338795 interrupts
42924056 CPU context switches
1665378296 boot time
40142 forks$ vmstat -h
Usage:
vmstat [options] [delay [count]]
Options:
-a, --active active/inactive memory
-f, --forks number of forks since boot
-m, --slabs slabinfo
-n, --one-header do not redisplay header
-s, --stats event counter statistics
-d, --disk disk statistics
-D, --disk-sum summarize disk statistics
-p, --partition <dev> partition specific statistics
-S, --unit <char> define display unit
-w, --wide wide output
-t, --timestamp show timestamp
-h, --help display this help and exit
-V, --version output version information and exit
For more details see vmstat(8).
vmstat commands example
Example: vmstat 1 5
1 -> the values will be re-measured and reported every second
5 -> the values will be reported five times and then the program will stopExecute vmstat ‘X’ seconds and (‘N’number of times). With this command, vmstat execute every two seconds and stop automatically after executing six intervals.
$ vmstat 2 6
Vmstat with timestamps. vmstat command with -t parameter shows timestamps with every line printed as shown below.
$ vmstat -t 1 5
Statistics of Various Counter, vmstat command and -s switch displays summary of various event counters and memory statistics.
$ vmstat -s
Disks Statistics, vmstat with -d option display all disks statistics.
$ vmstat -d
Display Statistics in Megabytes, The vmstat displays in Megabytes with parameters -S and M(Uppercase & megabytes). By default vmstat displays statistics in kilobytes.
$ vmstat -S M 1 5
Display number of forks since last boot, This displays all the fork system calls made by the system since the last boot. This displays all fork, vfork, and clone system call counts.
$ vmstat -f
Display slab info, Use option -m, to display the slab info as shown below.
$ vmstat -m
Understanding vmstat output
There are four sections in the output:
1. The first section of the output deals with the amount of used memory:
- Total physical memory.
- Currently used memory.
- Active memory.
- Inactive memory.
- Free memory.
- Buffer memory.
- Cache memory.
- Swap memory information.
2. The second section displays CPU statistics:
- Non-nice CPU ticks – the number of times the CPU was used for high priority processes.
- Nice CPU ticks – the number of times the CPU was used for lower priority processes.
- System CPU ticks – the number of times the CPU was used for kernel processes.
- Idle CPU ticks – the number of times the CPU was idle.
- IO-wait – the number of times the CPU was used for input/output management.
- IRQ – the number of times the CPU received interrupt requests.
- softirq – the number of times the CPU received software interrupts.
- stolen CPU ticks – the number of times a virtual machine stole CPU time.
3. The third section shows values related to memory paging:
- Total pages paged in from virtual memory.
- Total pages paged out to virtual memory.
- Total pages read from swap memory.
- Total pages written to swap memory.
4. The last section describes various event counters:
- The number of interrupts since boot.
- The number of context switches performed.
- The timestamp for the last boot time.
- The total number of forks.
Decoding vmstat output
Examples
CPU User Load Example
A standard audio file will be encode as an MP3 file by means of the lame encoder[1] in this example. This process is quite CPU intensive and also demonstrates the execution of vmstat in parallel with a user CPU time of 97%:
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
6 1 0 302380 16356 271852 0 0 561 568 80 590 43 7 43 7 0
1 0 0 300892 16364 273256 0 0 0 52 79 274 97 3 0 0 0
2 0 0 299544 16364 274532 0 0 0 0 78 372 97 3 0 0 0
1 0 0 298292 16368 275780 0 0 0 0 53 255 97 3 0 0 0
1 0 0 296820 16368 277192 0 0 0 0 77 377 97 3 0 0 0
[user@RHEL ~]$
CPU System Load Example
In this example, a file will be filled with random content using dd.
[user@fedora9 ~]$ dd if=/dev/urandom of=500MBfile bs=1M count=500
For this, /dev/urandom[2] will supply random numbers, which will be generated by the kernel. This will lead to an increased load on the CPU (sy – system time). At the same time, the vmstat executing in parallel will indicate that between 93% and 97% of the CPU time is being used for the execution of kernel code (for the generation of random numbers, in this case).
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 0 402944 54000 161912 745324 5 14 54 59 221 867 13 3 82 2 0
1 0 402944 53232 161916 748396 0 0 0 0 30 213 3 97 0 0 0
1 0 402944 49752 161920 751452 0 0 0 0 28 290 4 96 0 0 0
1 0 402944 45804 161924 755564 0 0 0 0 29 188 2 98 0 0 0
1 0 402944 42568 161936 758608 0 0 0 17456 272 509 7 93 0 0 0
[user@RHEL ~]$
The time for executing system calls[3][4][5] will be counted as system time (sy).
RAM Bottleneck (swapping) Example
In this example, many applications will be opened (including VirtualBox with a Windows guest system, among others). Almost all of the working memory will be used. Then, one more application (OpenOffice) will be started. The Linux kernel will then swap out several memory pages to the swap file on the hard disk, in order to get more RAM for OpenOffice. Swapping the memory pages to the swap file will be seen in the so (swap out – memory swapped to disk) column as vmstat executes in parallel.
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 1 244208 10312 1552 62636 4 23 98 249 44 304 28 3 68 1 0
0 2 244920 6852 1844 67284 0 544 5248 544 236 1655 4 6 0 90 0
1 2 256556 7468 1892 69356 0 3404 6048 3448 290 2604 5 12 0 83 0
0 2 263832 8416 1952 71028 0 3788 2792 3788 140 2926 12 14 0 74 0
0 3 274492 7704 1964 73064 0 4444 2812 5840 295 4201 8 22 0 69 0
[user@RHEL ~]$
High IO Read Load Example
A large file (such as an ISO file) will be read and written to /dev/null using dd.
[user@RHEL ~]$ dd if=bigfile.iso of=/dev/null bs=1M
Executed in parallel, vmstat will show the increased IO read load (the bi value).
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 1 465872 36132 82588 1018364 7 17 70 127 214 838 12 3 82 3 0
0 1 465872 33796 82620 1021820 0 0 34592 0 357 781 6 10 0 84 0
0 1 465872 36100 82656 1019660 0 0 34340 0 358 723 5 9 0 86 0
0 1 465872 35744 82688 1020416 0 0 33312 0 345 892 8 11 0 81 0
0 1 465872 35716 82572 1020948 0 0 34592 0 358 738 7 8 0 85 0
[user@RHEL ~]$
High IO Write Load Example
In contrast with the previous example, dd will read from /dev/zero and write a file. The oflag=dsync will cause the data to be written immediately to the disk (and not merely stored in the page cache).
[user@RHEL ~]$ dd if=/dev/zero of=500MBfile bs=1M count=500 oflag=dsync
Executed in parallel, vmstat will show the increased IO write load (the bo value).
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 1 0 35628 14700 1239164 0 0 1740 652 117 601 11 4 66 20 0
0 1 0 34852 14896 1239788 0 0 0 23096 300 573 3 16 0 81 0
0 1 0 32780 15080 1241304 0 0 4 21000 344 526 1 13 0 86 0
0 1 0 36512 15244 1237256 0 0 0 19952 276 394 1 12 0 87 0
0 1 0 35688 15412 1237180 0 0 0 18904 285 465 1 13 0 86 0
[user@RHEL ~]$
CPU Waiting for IO Example
In the following example, an updatedb process is already running. The updatedb utility is part of mlocate. It examines the entire file system and accordingly creates the database for the locate command (by means of which file searches can be performed very quickly). Because updatedb reads all of the file names from the entire file system, the CPU must wait to get data from the IO system (the hard disk). For that reason, vmstat running in parallel will display large values for wa (waiting for IO):
[user@RHEL ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 1 403256 602848 17836 400356 5 15 50 50 207 861 13 3 83 1 0
1 0 403256 601568 18892 400496 0 0 1048 364 337 1903 5 7 0 88 0
0 1 403256 600816 19640 400568 0 0 748 0 259 1142 6 4 0 90 0
0 1 403256 600300 20116 400800 0 0 476 0 196 630 8 5 0 87 0
0 1 403256 599328 20792 400792 0 0 676 0 278 1401 7 5 0 88 0
[user@RHEL ~]$
Additional vmstat Options
vmstat –help
[user@RHEL ~]$ vmstat --help
usage: vmstat [-V] [-n] [delay [count]]
-V prints version.
-n causes the headers not to be reprinted regularly.
-a print inactive/active page stats.
-d prints disk statistics
-D prints disk table
-p prints disk partition statistics
-s prints vm table
-m prints slabinfo
-S unit size
delay is the delay between updates in seconds.
unit size k:1000 K:1024 m:1000000 M:1048576 (default is K)
count is the number of updates.
vmstat
[user@fedora9 ~]$ vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
2 0 14960 38016 6584 1069284 0 1 506 81 727 1373 12 4 81 3 0
[user@fedora9 ~]$
vmstat -V
[user@fedora9 ~]$ vmstat -V
procps version 3.2.7
[user@fedora9 ~]$
vmstat -a
[user@fedora9 ~]$ vmstat -a
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free inact active si so bi bo in cs us sy id wa st
3 0 14960 38024 988284 461704 0 1 506 81 726 1372 12 4 81 3 0
[user@fedora9 ~]$
vmstat -d
[user@fedora9 ~]$ vmstat -d
disk- ------------reads------------ ------------writes----------- -----IO------
total merged sectors ms total merged sectors ms cur sec
ram0 0 0 0 0 0 0 0 0 0 0
ram1 0 0 0 0 0 0 0 0 0 0
ram2 0 0 0 0 0 0 0 0 0 0
ram3 0 0 0 0 0 0 0 0 0 0
ram4 0 0 0 0 0 0 0 0 0 0
ram5 0 0 0 0 0 0 0 0 0 0
ram6 0 0 0 0 0 0 0 0 0 0
ram7 0 0 0 0 0 0 0 0 0 0
ram8 0 0 0 0 0 0 0 0 0 0
ram9 0 0 0 0 0 0 0 0 0 0
ram10 0 0 0 0 0 0 0 0 0 0
ram11 0 0 0 0 0 0 0 0 0 0
ram12 0 0 0 0 0 0 0 0 0 0
ram13 0 0 0 0 0 0 0 0 0 0
ram14 0 0 0 0 0 0 0 0 0 0
ram15 0 0 0 0 0 0 0 0 0 0
sda 136909 31536 13893867 1197609 58190 219323 2233264 7688807 0 677
sda1 35703 6048 1326394 511477 6728 16136 182984 419232 0 222
sda2 85 1489 2935 653 141 3603 29952 5254 0 1
sda3 101111 23961 12564154 685330 51321 199584 2020328 7264321 0 512
sr0 0 0 0 0 0 0 0 0 0 0
fd0 0 0 0 0 0 0 0 0 0 0
[user@fedora9 ~]
vmstat -D
[user@fedora9 ~]$ vmstat -D
22 disks
0 partitions
273820 total reads
63034 merged reads
27787446 read sectors
2395193 milli reading
116450 writes
438666 merged writes
4467248 written sectors
15377932 milli writing
0 inprogress IO
1412 milli spent IO
vmstat -p
vmstat -p will not work under Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=485246. The following report comes from an Ubuntu 9.10 system.
user@RHEL:~$ vmstat -p /dev/sda9
sda9 reads read sectors writes requested writes
23420 411365 24464 530801
vmstat -s
[user@fedora9 ~]$ vmstat -s
1553972 total memory
1516180 used memory
461892 active memory
988304 inactive memory
37792 free memory
6644 buffer memory
1069388 swap cache
1052248 total swap
14960 used swap
1037288 free swap
161467 non-nice user cpu ticks
7586 nice user cpu ticks
46310 system cpu ticks
1108919 idle cpu ticks
46832 IO-wait cpu ticks
2694 IRQ cpu ticks
2452 softirq cpu ticks
0 stolen cpu ticks
6947021 pages paged in
1116896 pages paged out
183 pages swapped in
3744 pages swapped out
9985406 interrupts
18852586 CPU context switches
1239004323 boot time
15072 forks
[user@fedora9 ~]$
vmstat -m
[user@fedora9 ~]$ vmstat -m
Cache Num Total Size Pages
fuse_request 11 11 368 11
fuse_inode 9 9 448 9
rpc_inode_cache 8 8 512 8
nf_conntrack_expect 0 0 168 24
nf_conntrack 26 80 248 16
dm_uevent 0 0 2464 3
UDPv6 22 22 704 11
TCPv6 6 6 1344 6
kmalloc_dma-512 8 8 512 8
sgpool-128 12 12 2048 4
scsi_io_context 0 0 104 39
ext3_inode_cache 6822 8360 496 8
ext3_xattr 85 85 48 85
journal_handle 170 170 24 170
journal_head 76 219 56 73
revoke_record 256 256 16 256
flow_cache 0 0 80 51
bsg_cmd 0 0 288 14
mqueue_inode_cache 7 7 576 7
isofs_inode_cache 0 0 376 10
hugetlbfs_inode_cache 11 11 344 11
dquot 0 0 128 32
shmem_inode_cache 1058 1071 448 9
xfrm_dst_cache 0 0 320 12
UDP 19 21 576 7
TCP 17 24 1216 6
blkdev_queue 21 21 1080 7
biovec-256 2 2 3072 2
biovec-128 5 5 1536 5
biovec-64 7 10 768 5
sock_inode_cache 619 650 384 10
file_lock_cache 39 39 104 39
Acpi-Operand 2935 2958 40 102
Acpi-Namespace 1700 1700 24 170
Cache Num Total Size Pages
taskstats 25 26 312 13
proc_inode_cache 233 242 360 11
sigqueue 28 28 144 28
radix_tree_node 7888 8606 296 13
bdev_cache 24 24 512 8
inode_cache 370 462 344 11
dentry 6592 15390 136 30
names_cache 2 2 4096 2
avc_node 73 73 56 73
selinux_inode_security 9888 10030 48 85
idr_layer_cache 627 644 144 28
buffer_head 2308 2688 64 64
mm_struct 659 693 448 9
vm_area_struct 11110 11592 88 46
files_cache 115 130 384 10
sighand_cache 141 150 1344 6
task_struct 246 248 3696 2
anon_vma 4778 5120 16 256
kmalloc-4096 95 112 4096 8
kmalloc-2048 272 304 2048 16
kmalloc-1024 518 524 1024 4
kmalloc-512 764 888 512 8
kmalloc-256 198 208 256 16
kmalloc-128 629 832 128 32
kmalloc-64 4322 5568 64 64
kmalloc-32 1554 1664 32 128
kmalloc-16 2644 3584 16 256
kmalloc-8 3561 3584 8 512
kmalloc-192 6349 6930 192 21
kmalloc-96 885 1176 96 42
[user@fedora9 ~]$ 
- DevOps Certified Professionals (DCP)
- Site Reliability Engineering Certified Professionals (SRECP)
- Master in DevOps Engineering (MDE)
- DevSecOps Certified Professionals (DSOCP)
URL - https://www.devopsschool.com/certification/
My Linkedin - https://www.linkedin.com/in/rajeshkumarin
- Best AI tools for Software Engineers - November 4, 2024
- Installing Jupyter: Get up and running on your computer - November 2, 2024
- An Introduction of SymOps by SymOps.com - October 30, 2024
