How Linux’s Ring Buffer Works
In Linux and Unix-like computers, booting and startup are two distinct phases of the sequence of events that take place when the computer is powered on.
The boot processes (BIOS or UEFI, MBR, and GRUB) take the initialization of the system to the point where the kernel is loaded into memory and connected to the initial ramdisk (initrd or initramfs), and systemd is started.
The startup processes then pick up the baton and complete the initialization of the operating system. In the very early stages of initialization, logging daemons such as syslogd or rsyslogd are not yet up and running. To avoid losing notable error messages and warnings from this phase of initialization, the kernel contains a ring buffer that it uses as a message store.
A ring buffer is a memory space reserved for messages. It is simple in design, and of a fixed size. When it is full, newer messages overwrite the oldest messages. Conceptually it can be thought of as a “circular buffer.”
The kernel ring buffer stores information such as the initialization messages of device drivers, messages from hardware, and messages from kernel modules. Because it contains these low-level startup messages, the ring buffer is a good place to start an investigation into hardware errors or other startup issues.
But don’t go empty-handed. Take dmesg with you.
The dmesg Command
The dmesg command allows you to review the messages that are stored in the ring buffer. By default, you need to use sudo to use dmesg.
All of the messages in the ring buffer are displayed in the terminal window.
That was a deluge. Obviously, what we need to do is pipe it through less:
Now we can scroll through the messages looking for items of interest.
You can use the search function within less to locate and highlight items and terms you’re interested in. Start the search function by pressing the forward slash key “/” in less.
RELATED: How to Use the less Command on Linux
Removing the Need for sudo
If you want to avoid having to use sudo each time you use dmesg, you can use this command. But, be aware: it lets anyone with a user account your computer use dmesg without having to use sudo.
Forcing Color Output
By default, dmesg will probably be configured to produce colored output. If it isn’t, you can tell dmesg to colorize its output using the -L (color) option.
To force dmesg to always default to a colorized display use this command:
Human Timestamps
By default, dmesg use a timestamp notation of seconds and nanoseconds since the kernel started. To have this rendered in a more human-friendly format, use the -H (human) option.
This causes two things to happen.
The output is automatically displayed in less. The timestamps show a timestamp with the date and time, with a minute resolution. The messages that occurred in each minute are labeled with the seconds and nanoseconds from the start of that minute.
Human Readable Timestamps
If you don’t require nanosecond accuracy, but you do want timestamps that are easier to read than the defaults, use the -T (human readable) option. (It’s a little confusing. -H is the “human” option, -T is the “human readable” option.)
The timestamps are rendered as standard dates and times, but the resolution is lowered to a minute.
Everything that happened within a single minute has the same timestamp. If all you’re bothered about is the sequence of events, this is good enough. Also, note that you’re dumped back at the command prompt. This option doesn’t automatically invoke less.
Watching Live Events
To see messages as they arrive in the kernel ring buffer, use the –follow (wait for messages) option. That sentence might seem a little strange. If the ring buffer is used to store messages from events that take place during the startup sequence, how can live messages arrive in the ring buffer once the computer is up and running?
Anything that causes a change in the hardware connected to your computer will cause messages to be sent to the kernel ring buffer. Update or add a kernel module, and you’ll see ring buffer messages about those changes. If you plug in a USB drive or connect or disconnect a Bluetooth device, you’ll see messages in the dmesg output. Even virtual hardware will cause new messages to appear in the ring buffer. Fire up a virtual machine, and you’ll see new information arriving in the ring buffer.
Note that you are not returned to the command prompt. When new messages appear they are displayed by dmesg at the bottom of the terminal window.
Even mounting a CD-ROM disk is seen as a change, because you’ve grafted the contents of the CD-ROM disk onto the directory tree.
To exit from the real-time feed, hit Ctrl+C.
Retrieve the Last Ten Messages
Use the tail command to retrieve the last ten kernel ring buffer messages. Of course, you can retrieve any number of messages. Ten is just our example.
The last ten messages are retrieved and listed in the terminal window.
Searching For Specific Terms
Pipe the output from dmesg through grep to search for particular strings or patterns. Here we’re using the -i (ignore case) option so that the case of matching strings is disregarded. our results will include “usb” and “USB” and any other combination of lowercase and uppercase.
The highlighted search results are in uppercase and lowercase.
We can isolate the messages that contain references to the first SCSI hard disk on the system sda. (Actually, sda is also used nowadays for the first SATA hard drive, and for USB drives.)
All of the messages that mention sda are retrieved and listed in the terminal window.
To make grep search for multiple terms at once, use the -E (extend regular expression) option. You must provide the search terms inside a quoted string with pipe “|” delimiters between the search terms:
Any message that mentions any of the search terms is listed in the terminal window.
Using Log Levels
Every message logged to the kernel ring buffer has a level attached to it. The level represents the importance of the information in the message. The levels are:
emerg: System is unusable. alert: Action must be taken immediately. crit: Critical conditions. err: Error conditions. warn: Warning conditions. notice: Normal but significant condition. info: Informational. debug: Debug-level messages.
We can make dmesg extract messages that match a particular level by using the -l (level) option and passing the name of the level as a command-line parameter. To see only “informational” level messages, use this command:
All of the messages that are listed are informational messages. They don’t contain errors or warnings, just useful notifications.
Combine two or more log levels in one command to retrieve messages of several log levels:
The output from dmesg is a blend of messages of each log level:
The Facility Categories
The dmesg messages are grouped into categories called “facilities.” The list of facilities is:
kern: Kernel messages. user: User-level messages. mail: Mail system. daemon: System daemons. auth: Security/authorization messages. syslog: Internal syslogd messages. lpr: Line printer subsystem. news: Network news subsystem.
We can ask dmesg to filter its output to only show messages in a specific facility. To do so, we must use the -f (facility) option:
dmesg lists all of the messages relating to daemons in the terminal window.
As we did with the levels, we can ask dmesg to list messages from more than one facility at once:
The output is a mix of syslog and daemon log messages.
Combining Facility and Level
The -x (decode) option makes dmesg show the facility and level as human-readable prefixes to each line.
The facility and level can be seen at the start of each line:
The first highlighted section is a message from the “kernel” facility with a level of “notice.” The second highlighted section is a message from the “kernel” facility with a level of “info.”
That’s Great, But Why?
In a nutshell, fault finding.
If you are having issues with a piece of hardware not being recognized or not behaving properly, dmesg may throw some light on the issue.
Use dmesg to review messages from the highest level down through each lower level, looking for any errors or warnings that mention the hardware item, or may have a bearing on the issue. Use dmesg to search for any mention of the appropriate facility to see whether they contain any useful information. Pipe dmesg through grep and look for related strings or identifiers such as product manufacturer or model numbers. Pipe dmesg through grep and look for generic terms like “gpu” or “storage”, or terms such as “failure”, “failed” or “unable”. Use the –follow option and watch dmesg messages in real-time.
Happy hunting.
