- Unix Commands Reference
- Unix Commands - Home
crash Command in Linux
The crash command in Linux is a powerful tool used for analyzing the state of a Linux system, either during its operation or after a kernel crash has occurred. It is an invaluable resource for system administrators and developers dealing with system failures, providing a means to diagnose and understand the causes of a crash.
The crash command is a testament to the robustness of Linux systems, providing a comprehensive set of tools for post-mortem analysis. It empowers users to dissect and understand the intricacies of system crashes, making it an essential tool for maintaining system stability and performance.
Table of Contents
Here is a comprehensive guide to the options available with the crash command −
- Understanding crash Command in Linux
- Install crash Command in Linux
- How to Use crash Command in Linux?
- Examples of crash Command in Linux
- Alternatives of crash Command in Linux
Understanding crash Command in Linux
The command set of crash includes a variety of tools for kernel core analysis −
- Kernel stack backtraces for all processes
- Source code disassembly
- Formatted displays of kernel structures and variables
- Virtual memory data examination
- Dumps of linked lists
Additionally, crash provides commands that delve into specific kernel subsystems, enhancing its capability to perform detailed analysis.
Prerequisite: Install crash Command in Linux
The method to install the crash command depends on your Linux distribution.
For most modern Linux distributions, crash is available as a pre-built package. You can install it using the appropriate package manager for your system −
Ubuntu/Debian −
sudo apt install crash
Fedora/CentOS/RHEL −
sudo yum install crash
Arch Linux −
sudo pacman -S crash
Other distributions −
Use the package manager specific to your distribution (e.g., zypper for openSUSE, dnf for Fedora-based systems).
Building from Source (If Pre-Built Package Not Available) −
If the crash package is not available for your distribution, you can build it from source. This requires additional dependencies like gdb and kernel headers.
Download the source code −
git clone https://github.com/crash-utility/crash.git
Install dependencies −
Ensure you have gdb, kernel headers, and other necessary libraries installed. Configure and build −
cd crash ./configure make sudo make install
Once installed, you can verify the installation by running −
crash --version
How to Use crash Command in Linux?
The crash command in Linux is a powerful tool used for analyzing the state of a Linux system during a crash or after a kernel crash has occurred. It is an invaluable resource for system administrators and developers when it comes to diagnosing system failures, understanding the root cause of a crash, and preventing future incidents.
| Options | Descriptions |
|---|---|
| NAMELIST | The NAMELIST option specifies the path to an uncompressed kernel image file, which is necessary for the crash utility to analyze the system. This file should be compiled with the "-g" option to include debugging information. |
| MEMORY-IMAGE[@ADDRESS] | This option is used to specify a kernel core dump file created by facilities like netdump, diskdump, LKCD, kdump, xendump, kvmdump, or VMware. If this argument is not provided, crash will be invoked on the live system. |
| -h or --help | Displays general help about crash commands. |
| -h [option] | Explains a specific crash command (if provided as option). |
| -h input | Shows details about command-line input options. |
| -h all | Displays all available help messages. |
| -h output | Explains command-line output options. |
| vmlinux and dumpfile | These are arguments (not options) that specify the kernel image and crash dump file, respectively. |
| -i file | Executes commands from a file before entering the interactive mode. |
| -L | Attempts to lock all memory used by crash. |
| -d num | Sets the debugging level (higher number means more debug info). |
| -s | Starts crash silently, skipping version and initialization info. |
| -S | Uses /boot/System.map as the symbol information file. |
| -e vi or -e emacs | Sets the command-line editing mode (default is vi). |
| -f | Forces using a compressed kernel image file. |
| -k | The symbol information file is a special LKCD deginfo file. |
| -g [namelist | Checks if a kernel image or Xen symbol file contains debug data. |
| -t | Displays the system crash timestamp and exits. |
| -c tty-device | Redirects debug messages to a specific terminal device. |
| -p page-size | Sets the page size if it cannot be determined automatically. |
| -o filename | Creates a new vmcore file from a raw RAM dump. |
| -m option=value | Passes architecture-specific options (mostly for developers). |
| -x | Loads extension modules automatically. |
| --* options (many!) | These are more advanced options for specific situations, like analyzing Xen hypervisors, setting memory behavior, and overriding default settings. |
Users can place commands in a $HOME/.bashrc file or a .crashrc file in the current directory. These commands are executed during the initialization of crash, with the $HOME/.crashrc commands running first.
Examples of crash Command in Linux
Now, let’s go through some examples to better understand the working of the crash command in Linux −
- Displaying Kernel Variables
- Inspecting Linked Lists
- Exploring Process Information
- Inspecting Crash Dump
- Examining Crash Reason
- Analyzing Stack Traces
- Setting Breakpoints
- Stepping Through Code
- Inspecting Registers
- Displaying Memory Contents
- Searching for Data
- Disassembling Code
Displaying Kernel Variables
Prints the number of CPUs available in the system as stored in the kernel variable sysctl_nr_cpus
crash > print sysctl_nr_cpus
Inspecting Linked Lists
Prints a list of all active tasks in the system, providing details about each task's state and related information.
crash> p task_list
Exploring Process Information
Displays detailed information about the currently executing process in the kernel.
crash> p current
Inspecting Crash Dump
Loads the crash analysis tool, then opens the core dump file located at /var/crash/core for examination and debugging.
crash> crash /var/crash/core
Note − Users can place commands in a $HOME/.crashrc file or a .crashrc file in the current directory. These commands are executed during initialization, enhancing the utility's flexibility and automation.
Examining Crash Reason
Displays information about the kernel crash, including crash reason, registers, stack traces, and other relevant details to help diagnose the issue.
crash> show crash_info
Analyzing Stack Traces
Displays the current execution stack trace, showing the sequence of function calls leading up to the current point in the kernel's execution.
crash> bt
Setting Breakpoints
Sets a breakpoint at the do_fork function in the kernel. When the program execution reaches this point, the debugger will pause, allowing you to inspect variables, registers, and memory. This is useful for debugging kernel-related issues related to process creation.
crash> break do_fork
Stepping Through Code
The crash> step command executes the next instruction in the current function, allowing you to examine the kernel's execution flow step-by-step. This is useful for debugging and understanding the kernel's behavior.
crash> step
Inspecting Registers
Displays the current values of all general-purpose CPU registers. Useful for understanding the program's execution state and debugging issues.
crash> info registers
Note − This MEMORY-IMAGE[@ADDRESS] argument specifies a kernel core dump file created by facilities like netdump, diskdump, LKCD, kdump, xendump, kvm dump, or VMware. If this argument is not entered, the crash session will be invoked on the live system.
Displaying Memory Contents
Examines 100 8-byte (64-bit) memory locations starting from the hexadecimal address 0xc0000000, displaying the contents in hexadecimal format.
crash> x/100xg 0xc0000000
Searching for Data
This command searches the entire address space of the kernel for the specified string "magic_string". It's used to locate specific data structures or memory regions within the kernel's memory.
crash> search "magic_string"
Disassembling Code
Disassembles the code starting at the entry point of the kernel. This command is useful for understanding the initial execution flow of the kernel and potentially identifying issues during the boot process.
crash> disassemble /entry
Here's an in-depth look at some of the options available with the crash command:
The crash utility's extensive command set and options make it a versatile tool for a wide range of diagnostic tasks. Whether you're dealing with a system crash, debugging kernel modules, or simply monitoring system performance, the crash command provides a detailed window into the inner workings of the Linux kernel.
For more detailed information and a complete list of commands and options, you can refer to the Linux manual page.
For a complete list of options and a more detailed explanation, you can refer to the crash manual page. It is always recommended to consult the official documentation for the most accurate and up-to-date information.
Remember, using such powerful tools requires a good understanding of the system and should be done with caution to avoid unintended consequences.
Alternatives of crash Command in Linux
crash relies on kernel symbol information (usually found in vmlinux) to interpret data structures. It provides a command-line interface for exploring the kernel state. It's built on top of gdb, offering powerful debugging capabilities. It can analyze core dumps generated by kernel crashes.
While crash is a powerful tool for kernel analysis, there are other options available, depending on the specific task at hand.
General-Purpose Debuggers
- GDB (GNU Debugger) − This is the foundation upon which crash is built. It offers more granular control and flexibility but requires a deeper understanding of debugging concepts.
- LLDB (Low Level Debugger) − A modern debugger with a focus on usability and performance. It can be used for both user-space and kernel-space debugging.
Kernel-Specific Tools
- Kdump − This creates a kernel core dump on a crash, allowing for post-mortem analysis. It's often used in conjunction with crash or other tools.
- Perf − Primarily for performance analysis, but can also provide insights into kernel behavior and issues.
- Ftrace − A dynamic tracing tool that can be used to capture kernel events and analyze system behavior.
- SystemTap − A scripting language for live system exploration, profiling, and troubleshooting.
Other Tools
- Kexec − Can be used to reboot the system into a kernel debugger without restarting the hardware.
- KGDB − Enables remote debugging of a running kernel.
Conclusion
Understanding and effectively utilizing the crash command can significantly reduce system downtime and improve recovery processes, making it an essential skill for anyone responsible for maintaining Linux systems. Remember, with great power comes great responsibility—use the crash command wisely to ensure system stability and security.
The crash command is a powerful tool for investigating the state of a Linux system, either while it's running or after a kernel crash has produced a core dump. It's essentially a user-friendly interface built on top of the gdb debugger, making it easier to analyze complex kernel data structures.