crash Command

AIX Version 4.3 Commands Reference, Volume 1

crash Command

Purpose

Displays system images for examining a dump.

Syntax

crash [-a ] [ -i IncludeFile ] [ SystemImageFile [ KernelFile ] ]

Description

The crash command is an interactive utility for examining an operating system image, or the running kernel. The crash command facility interprets and formats control structures in the system and certain miscellaneous functions for examining a dump.

The SystemImageFile parameter specifies the file that contains the system image. The default for the SystemImageFile parameter is the /dev/mem file. The KernelFile parameter contains the kernel symbol definitions. The default for the KernelFile parameter is the /unix file.

You can run the crash command with no arguments to examine an active system. If you specify a system image file, the crash command assumes it is a system dump file and sets the default thread to the thread running at the time of the crash.

Notes:

When using the crash command a kernel file must be available.

Stack tracing of the current process on a running system does not work.

When crash is run on a running system, you will see a message about the possibility that crash may cause a system crash and/or data corruption, and there will be error log entries noting each time crash is started and stopped. These are done because of the risk posed by certain crash subcommands. Those subcommands are shown here with the messages, labeled Attention. If you limit yourself to subcommands without these messages, you can avoid problems.

The crash command recognizes the following aliases in subcommand format specifications:

Format	Aliases	Format	Aliases
byte	b	instruction	I
character	char, c	longdec	ld, D
decimal	dec, e	longoct	lo, O
directory	direct, dir, d	octal	oct, o
hexadecimal	hexadec, hex, h, x	write	w
i-node	ino, i

Note:The instruction format disassembles instructions at a specified address. The crash command attempts to disassemble instructions to a PowerPC or POWER architecture instruction, depending upon the machine architecture on which the system dump was taken. If the instruction is not in the appropriate architecture, the crash command attempts to disassemble it to an instruction from the other architecture. This behavior can be modified with the set idarch subcommand. For more information, see "Interpreting an Assembler Listing" in AIX Assembler Language Reference.

Symbols displayed by crash

Several crash subcommands (such as trace, ds, ....) display symbol names and offsets corresponding to numeric addresses. All symbols from KernelFile display with the symbol name, a + sign, and the offset, for example:

.thread_terminate+3e0
vmmerrlog + 0x00000018

Text symbols from kernel extensions are shown in the form .[KernelExtensionName:FunctionName]+Offset, but only if the traceback tables for the kernel extension can be found in SystemImageFile. Offset is the hexadecimal offset (in bytes) from the beginning of the function. In the following example, the address being shown is 0x44c bytes past the beginning of the svc_run routine in the nfs_ext kernel extension.

.[nfs.ext:svc_run]+44c

Data symbols from kernel extensions cannot be displayed by crash, so the addresses of data items within kernel extensions display in the form .[KernelExtensionName]+Offset, where Offset is the offset of the address from the beginning of the kernel extension. This is also used for text symbols when traceback tables are not available in SystemImageFile. In the following example, the address being shown is 0x4c bytes past the beginning of the nfs.ext kernel extension:

.[nfs.ext] + 0x0000004c

Addresses in crash

Many of the commands in crash take addresses or kernel symbol names as parameters. Addresses are always specified in hexadecimal, and can usually be specified in one of the following forms:

addr	An 8 digit hexadecimal number, taken to be as an effective address within the context of the current process and thread, or (in some cases) the context of the thread specified on a previous cm command. addr can be prefixed with the characters `0x`.
segid:offset	segid is the segment ID for a virtual memory segment. The maximum size is 6 hex digits. offset is the offset (in bytes) from the beginning of that segment. The maximum size is 7 hex digits.
r:realaddr	r is the literal character "r". realaddr is a real memory address. This form can only be used when running crash against a system dump, and it only will display dump data areas that were dumped by real address instead of virtual address. readaddr can be up to 12 hexadecimal digits. To enhance readability, you may include underscores ("_") anywhere within these values.

Examples:

18340050
2314:55300
r:14_3370_0560 (same as r:1433700560)

Command-line Editing

The crash command provides command line editing features similar to those provided by the Korn shell. vi mode provides vi-like editing features, while emacs mode gives you controls similar to emacs. You can turn these features on by using the crash subcommand set edit. So, to turn on vi-style command-line editing, you would type the subcommand set edit vi.

Output Redirection

The crash command provides a subset of Korn shell input/output redirection. Specifically, the following operators are provided:

| (pipe symbol)

Pipes all output of the command before the symbol to the input of the command after the symbol. Both standard output and error output are affected, which is different than standard shell behavior.

> filename

Writes the output of the command before the > to filename. Both standard and error output are written to the file.

>> filename

Adds the output of the command before the >> to the end of filename. Both standard and error output are written to the file.

Subcommands

The crash command recognizes several subcommands. The crash command presents a > (greater-than sign) prompt when it is ready to interpret subcommands entered from the workstation. The general subcommand format for the crash command is:

Subcommand [Flags] [StructuresToBeDisplayed]

When allowed, the Flag parameters modify the format of the data displayed. If you do not specify which structure elements you want to examine, all valid entries are displayed. In general, those subcommands that perform I/O operations with addresses assume hexadecimal notation.

Since the crash command only deals with kernel threads, the word thread when used alone will be used to mean kernel thread in the crash documentation that follows. The default thread for several subcommands is the current thread (the thread currently running). On a multiprocessor system, you can use the cpu subcommand to change the current processor: the default thread becomes the running thread on the selected processor.

The parameters ProcessTableEntry and ThreadTableEntry are used in many subcommands to indicate a process or thread respectively. These parameters are simply numbers for table entry indexes which can be displayed using the proc and thread subcommands.

Most of the subcommands recognized by the crash command have aliases (abbreviated forms that give the same result). The crash command recognizes the following subcommands:

alter [[-c] [-s] [-w] [-l] [L \| ll] addr val
	Recognized by the poke subcommand alias. Put the value val at given address addr. Attention:You should use this command with extreme caution. Errors in the use of this command may cause system failures. -c Indicates that val is a char.(1 byte) This is the default. -s Indicates that val is a short. (2 bytes) -w Indicates that val is a word. (4 bytes) -l Indicates that val is a long. (4 bytes) -L \| ll Indicates that val is a long long. (8 bytes)
alter [[-c] [-s] [-w] [-l]] [L \| ll] addr origval=val
	Recognized by the poke subcommand alias. Put value at given address only if origval is already stored at that address. Attention:You should use this command with extreme caution. Errors in the use of this command may cause system failures. Note: The flags indicate the val is a char, short, word, long, or L \| ll for storing 64-bit information.
alter [[-c] [-s] [-w] [-l]] [L \| ll] addr
	Recognized by the poke subcommand alias. Start altering memory at the given address, interactively, Attention:You should use this command with extreme caution. Errors in the use of this command may cause system failures. Note: The flags indicate the val is a char, short, word, long, or L \| ll for storing 64-bit information.
buf [BufferHeaderNumber] . . .
	Recognized by the bufhdr and hdr subcommand aliases. Displays the system buffer headers. The buff.# file is created where the # variable file extension is the BufferHeaderNumber. Binary data is written to the buff.# file.
buffer [Format] [BufferHeaderNumber] . . .
	Recognized by the b subcommand alias. Displays the data in a system buffer according to the Format parameter. When specifying a buffer header number, the buffer associated with that buffer header is displayed. If you do not provide a Format parameter, the previous Format is used. Valid parameters are the decimal, octal, hex, character, byte, directory, i-node, and write formats. The write format creates a file in the current directory containing the buffer data.
calc expr	Recognized by the @ subcommand alias. Expression can be any expression supported by the bc command, although mixed case is supported. Values are in hex.
callout	Recognized by the c, call, calls, time, timeout,tout, and trb subcommand aliases. Displays all entries on the active trblist (rough equivalent of the cutoutable).
cm [vmmon \| vmmoff] [ThreadTableEntry SegmentNumber]
	Changes the segment map of the crash command internal pointers for any process thread segment not paged out if you specify the process-thread-slot number and segment number. This allows the od subcommand to display data relative to the beginning of the segment desired for threads other than the current thread. Specification of vmmon or vmmoff allows selection of whether effective addresses in the range 0x70000000 through 0xa0000000 are to be interpreted by the od subcommand as kernel or VMM data references. When entering the cm subcommand without any parameters, it resets the map of internal pointers.
conv num	Recognized by the % subcommand alias. If num is hex, convert to decimal, otherwise convert to hex. Bases are guessed by leading 0x for hex or base# for bases other than decimal.
conv [-] [bdxo] num	Recognized by the % subcommand alias. Convert num to specified base binary, decimal, hex, or octal. Bases are guessed by leading 0x for hex or base# for bases other than decimal.
conv [-] [bdxo] [bdxo] num
	Recognized by the % subcommand alias. Convert num from base specified by the first flag to the base specified by the second flag, binary, decimal, hex, or octal.
cpu [ProcessorNumber]
	If no argument is given, the cpu subcommand displays the number of the currently selected processor. Initially, the selected processor is the processor that caused the system crash (when running crash against a dump), or processor 0 (when running against a running system). If the ProcessorNumber argument is given, the cpu subcommand selects the specified processor as the current processor. By extension, this selects the current thread (the running thread on the selected processor). Processor numbering starts from zero.
dblock Address	Recognized by the dblk subcommand alias. The dblock subcommand displays the allocated streams data block headers. The Address parameter is required. If the Address is not supplied, this subcommand prints an error message stating that the address is required. See /usr/include/sys/stream.h header file for the datab structure definition. The `freep` and `db_size` descriptions are not included in /usr/include/sys/stream.h. These structure members and their descriptions are: `freep` Address of the free pointer. `db_size` Size of the data block. There is no checking performed on the address passed in as the required parameter. The dblock subcommand will accept any address. It is up to the user to be sure that a valid address is specified. To determine a valid address, you will need to run the mblock subcommand. From the output of the mblock subcommand, select a non-zero data block address from under the column heading, DATABLOCK.
decode instr	Decode the given instruction word. instr is specified as a hex value.
devsw	Show device switch table.
devsw major	Show the device switch structure for device with the given major number.
dlock[ThreadIdentifier \| -p [ProcessorNumber] ]
	Displays deadlock analysis information about all types of locks (simple, complex, and lockl). The dlock subcommand searches for deadlocks from a given start point. If ThreadIdentifier is given, the corresponding thread is the start point. If -p is given without a ProcessorNumber, the start point is the running thread on the current processor. If -p ProcessorNumber is given, the running kernel thread on the specified processor is the start point. If no arguments are given, dlock searches for deadlocks among all threads on all processors. The first output line gives information about the starting thread, including the lock which is blocking the kernel thread, and a stack trace showing the function calls which led to the blocking lock request. Each subsequent line shows the lock held by the blocked thread from the previous line, and identifies the thread or interrupt handler which is blocked by those locks. If the information required for a full analysis is not available (paged out), an abbreviated display is shown; in this case, examine the stack trace to locate the locking operations which are causing the deadlock. The display stops when a lock is encountered for a second time, or no blocking lock is found for the current thread.
dmodsw	Displays the streams drivers switch table. The information printed is contained in an internal structure. The members of this internal structure and their descriptions are: `address` Address of dmodsw. `d_next` Pointer to the next driver in the list. `d_prev` Pointer to the previous driver in the list. `d_name` Name of the driver. `d_flags` Flags specified at configuration time. `d_sqh` Pointer to synch queue for driver-level synchronization. `d_str` Pointer to streamtab associated with the driver. `d_sq_level` Synchronization level specified at configuration time. `d_refcnt` Number of open or pushed count. `d_major` Major number of a driver. The `flags` structure member, if set, is based on one of the following values:

#define	Value	Description
F_MODSW_OLD_OPEN	0x1	Supports old-style (V.3)open/close parameters
F_MODSW_QSAFETY	0x2	Module requires safe timeout/bufcall callbacks
F_MODSW_MPSAFE	0x4	Non MP Safe drivers need funneling

The synchronization level codes are described in the /usr/include/sys/strconf.h header file.

ds [DataAddress] . . .	Finds the data symbols closest to the given addresses.
du [ThreadTableEntry]	Displays a combined hex and ASCII dump of the specified thread's uthread structure and of the user structure of the process which owns the thread. If the data is not available (paged out), a message is displayed. The default is the current thread.
dump	Displays the name of each component for which there is data present. After you select a component name from the displayed list, the crash program loads and runs the associated formatting routine contained in the/usr/lib/ras/dmprtns directory. If there is more than one data area for the selected component, the formatting routine displays a list of the data areas and allows you to select one. The crash command then displays the selected data area. You may enter the quit subcommand to return to the previously displayed list and make another selection or enter the quit subcommand a second time to leave the dump subcommand loop.
errpt [ count]	Displays messages in the error log. Count is the number of messages to print that have already been read by the errdemon process. (The default is 3 messages.) errpt always prints all messages that have not yet been read by the errdemon process.
file [FileTableEntry] . . .	Recognized by the files and f subcommand aliases. Displays the file table. Unless specific file entries are requested, only those with a nonzero reference count are displayed.
find [-u] [-s] [-p slot] [-c context] [-a alignment] pattern
	Recognized by the x subcommand alias. Search user-space for a given pattern. The default is to search the GPR save areas in the mstsave areas which are both on the Current Save Area Chain (CSA) and in each uthread area for every thread. -u Search all process private segments, (Stack, Uarea,...) -s Search all process private segment from the current stack pointer. -c context Number of bytes of context to print on a match. -a alignment Byte alignment for pattern. The default is 4. -p slot Search only specified process. The default is to search all processes. Attention: Using this command on a running system may cause system crashes. Rules for pattern pattern is a search pattern of any arbitrary length that contains either a hexadecimal number or a string. To specify a hexadecimal pattern, just type the hex digits. 'Don't care' digits can be represented with the character `x`. To specify a string pattern, enclose the pattern in double quotes. 'Don't care' characters can be represented with the sequence `\x` Examples: > find -k 02x4 00110a28: 02140008 \|....\| 00110af0: 02640004 \|.d..\| 00110c80: 02e40004 \|....\| 003f0ed8: 02242ff8 \|.$/.\| ... > find -k "b\xt" 00012534: 6269745f \|bit_\| 00012618: 6269745f \|bit_\| 0001264c: 6269745f \|bit_\| 00021cb0: 62797465 \|byte\| 00021d60: 62797465 \|byte\| ... > find -k "i_ena" 0 250000 001ceaa8: 695f656e 61626c65 \|i_enable\|
find -k [-c context] [-a alignment] pattern [start[end]]
	Recognized by the x subcommand alias. Search the kernel segments. The default range is the whole of each kernel segment.

-c context	Number of bytes of context to print on a match.
-a alignment	Byte alignment for pattern. The default is 4.
find -b branch_addr [start_addr[end_addr]]
	Recognized by the x subcommand alias. Search for a branch to the given address. The default range is the whole of each kernel segment.
find -m [-a addr] [-t type] [-c] [-i] [start[end]]
	Recognized by the x subcommand alias. Search the things that look like mbufs. The default search range is the network memory heap.

-a	Search for mbufs that point to this cluster address.
-t type	Only search for this type of mbuf.
-c	Only search for clusters.
-i	Ignore length sanity checks.
find -v [-f] wordval [start[end]]
	Recognized by the x subcommand alias. Search for the first word not matching the given value. The default is to search the kernel segments.

-f	Force scan to continue when a region not in the dump is scanned.
find -U seg_id	Recognized by the x subcommand alias. Search for processes whose segment registers contain the given segment ID.
fmodsw	Displays the streams modules switch table. The information printed is contained in an internal structure. The members of this internal structure and their definitions are:

`address`	Address of fmodsw.
`d_next`	Pointer to the next module in the list.
`d_prev`	Pointer to the previous module in the list.
`d_name`	Name of the module.
`d_flags`	Flags specified at configuration time.
`d_sqh`	Pointer to synch queue for module-level synchronization.
`d_str`	Pointer to streamtab associated with the module.
`d_sq_level`	Synchronization level specified at configuration time.
`d_refcnt`	Number of open or pushed count.
`d_major`	-1.

The flags structure member, if set, is based on one of the following values:

#define	Value	Description
F_MODSW_OLD_OPEN	0x1	Supports old-style (V.3)open/close parameters
F_MODSW_QSAFETY	0x2	Module requires safe timeout/bufcall callbacks
F_MODSW_MPSAFE	0x4	Non MP Safe drivers need funneling

The synchronization level codes are described in the /usr/include/sys/strconf.h header file.

fs [ThreadTableEntry]
	Traces a kernel stack for the thread specified by the thread slot number. Displays the called subroutines with a hex dump of the stack frame for the subroutine that contains the parameters passed to the subroutine. The default thread is the thread currently running. This subcommand will not work on the current thread or a running system because it uses stack tracing; however, it does work on a dump image.
help	Recognized by the ? subcommand alias. Print a list of a commands with short descriptions.
help cmd...	Recognized by the ? subcommand alias. Print a long description of the specified command.
help /regex...	Recognized by the ? subcommand alias. Print those long descriptions that match the specified extended, case-insensitive regular expression.
hide symbol...	Hide the specified symbol from the crash commands that convert addresses to symbols and offsets. The main reason for this ability is to hide symbols that may show up in the middle of a function. This occurs in assembly routines. See the unhide subcommand.
hide	Show all hidden symbols. See the unhide subcommand.
id[...] addr* [units]
id[...] symname* [units]	Instruction Decode. Equivalent to od with the instruction format. The units are measured in number of instructions.
inode [-] [ <MAJ> <MIN> <INUMB> ] . . .
	Recognized by the ino and i subcommand aliases. Displays the i-node table and the i-node data block addresses. A specific i-node can be displayed by specifying the major and minor device number of the device where the i-node resides and the i-node number. The i-node will only be displayed if it is currently on the system hash list.
kfp[FramePointer] . . .	Recognized by the fp and rl subcommand aliases. If the kfp subcommand is entered without parameters, it displays to the screen the last kernel frame pointer address that was set using kfp. If a frame pointer address is provided, then it sets the kernel frame pointer to the new address. This subcommand is used conjunction with the -r flag on the trace subcommand.
knlist[Symbol] . . .	Displays the addresses of all the symbol names given. If the symbol is not found, a no-match message is displayed to the screen. This subcommand reads the kernel export list for either the running system or a dump image.
le [ModuleAddress \| Keyword]
	Displays the kernel load list entries. If you specify an address in a Kernel Extension, the corresponding load list entry is displayed. If you specify a name or part of a name of a kernel extension, then the load list entries matching the name are displayed. Attempting to display a paged out loader entry produces an error message.
link addr num [next_offset [end_val]
	Recognized by the ll subcommand alias. Follows linked list starting at addr. Print num words for each entry. next_offset is the offset in words of the next pointer, the default is 0. end_val is the value of the next pointer that terminates the list, the default is 0.
linkblk	Recognized by the lblk subcommand alias. The linkblk subcommand displays the streams linkblk table. See /usr/include/sys/stream.h header file for the linkblk structure definition. If there are no linkblk structures found on the system, the linkblk subcommand prints a message stating that no structures are found.
lock	Recognized by the locks subcommand alias. Print status on global kernel locks as well as threads waiting on events and locks.
lock [-clsq] addr\|symbol...
	Recognized by the locks subcommand alias. Print lock at addr, or address specified by symbol. The default format is that of a Simple_lock.
-c	Print as a Complex_lock
-l	Print as a lock_t
-s	Print as a Simple_lock. This is the default.
-q	Suppress instrumentation data.

Status Bits:
I INTERLOCK

W WAITING

WW WANT_WRITE (Complex_lock)

RD READ_MODE(Complex_lock)

L LOCKBIT (Simple_lock)

S INSTR_ON (Simple_lock)

mblock Address Recognized by the mblk subcommand alias. The mblock subcommand displays the allocated streams message block headers. The Address parameter is required. If Address is not supplied, the subcommand prints an error message stating that the address is required. See /usr/include/sys/stream.h header file for the msgb structure definitions.
There is very little checking performed on the address passed in as the required parameter. The mblock subcommand accepts any address that falls on a 128-byte boundary. It is up to the user to be sure that a valid address is specified.

To determine a valid address, run the queue subcommand. From the output of the queue subcommand, you will need to select a non-zero address in the head of the message queue, under the column heading HEAD, for either a read queue or a write queue.

mbuf Address Displays system mbuf structures at the specified address.

mst [-f] [Address] . . . Displays the mstsave portion of the uthread structure at the addresses specified (see the uthread.h and mstsave.h header files in /usr/include/sys). If you do not specify an address, it displays all of the mstsave entries on the CSA chain except the first. If you specify the -f flag the first mstsave area on the CSA chain displays.

ndb Displays network kernel data structures either for a running system or a system dump. The ndb subcommand, short for network debugger, supports the following options:
? Provides first-level help information.

help Provides additional help information.

tcb [Addr] Shows TCBs. The default is HEAD TCB.

udb [Addr] Shows UDBs. The default is HEAD UDB.

socketsAddr Shows sockets at the given address.

mbuf [Addr] Shows the mbuf at the specified address.

ifnet [Addr] Shows the ifnet structures at the specified address.

quit Stops the running option.

xit Exits the ndb submenu.

netm Displays the most recent net_malloc_police record.
Note: Requires that the Memory Overlay Detection System (MODS) be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.

netm -a Displays all records, starting with the most recent.
Note: Requires that the MODS be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.

netm addr Displays records whose address or caller fields match the given address.
Note: Requires that the MODS be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.

netstat Equivalent to the command line version of the netstat command.

nm[Symbol] . . . Displays symbol value and type as found in KernelFile.

od [*...] [SymbolName | Address] [Count] [Format]

Recognized by the rd subcommand alias. Dumps the Count parameter number of data values starting at the symbol value, or address given according to the Format parameter. Possible formats are the ascii, octal, longoct,decimal, longdec, character, hex, byte and instruction formats. The ASCII format is similar to the hex format but has an ASCII sidebar. The instruction format disassembles the number of instructions specified. The word "instruction" is abbreviated as the uppercase letter "I". The id subcommand is an alias for od...I. You can dereference a symbol or address by prefixing it with one or more asterisks.

ppd [ProcessorNumber *]

Displays per-processor data area (PPDA) structures for the specified processor. If no processor is specified, the current processor selected by the cpu subcommand is used. If the asterisk argument is given, the PPDA of every enabled processor is displayed.

prall Equivalent to crash -a from the command line.

print [Type] Address Recognized by the pr, str, or struct subcommand aliases. Does dbx-style printing of structures. The -i option must be given on the command line to use this feature. Type is the name of the structure to be displayed.

print -d type Recognized by the pr, str, or struct subcommand aliases. Sets the default type for subsequent print commands to type.

print -l offset|name [-e end_val] [type] address

Recognized by the pr, str, or struct subcommand aliases. Displays a linked list, starting at address using offset or structure member name as the location of the next pointer. Stop when next value of equals end_val. The default end_val is 0.

proc [-] [-r] [ProcessTableEntry] . . .
Recognized by the ps and p subcommand aliases. Displays the process table, including the thread count (the number of threads in the process) and state of each process. (See the /usr/include/sys/proc.h file for this structure definition.) The -r flag displays only runnable processes. The - (minus) flag displays a longer listing of the process table.

qrun Displays the list of scheduled streams queues. If there are no queues found for scheduling, the qrun subcommand prints a message stating that there are no queues scheduled for service.

queue [Address] Recognized by the que subcommand alias. The queue subcommand displays the STREAMS queue. If the optional parameter, Address, is not supplied, crash will display information for all write queues available. Refer to the /usr/include/sys/stream.h header file for the queue structure definition.
If you wish to see the information stored for a read queue, issue the queue subcommand with the read queue address specified as the Address parameter.

When you issue the queue subcommand with the Address parameter, the column headings do not distinguish between the read queue and the write queue. One queue address will be displayed under the column heading, QUEUE and the other queue in the pair will be displayed under the column heading, OTHERQ. The write queue will have a numerically higher address than the read queue.

quit Recognized by the q subcommand alias. Exits from the crash command.

search[-sn] name Search the symbols table for name.
-s Prints symbols matching name in the nm format. Also prints the symbol table entry for the last symbol found.

-n Prevents the search from examining kernel extensions..

search[-n] addr Search for the symbol with the largest value less than or equal to addr.

-n Prevents the search from examining kernel extensions..

segst64 [-p pslot | -t tslot] [-l limit [-s segflag[:value][, segflag[:value]]...] [-n [start_esid [end_esid]]

Recognized by the adspace, as, and sr subcommand aliases. Displays segstate information for a 64-bit process. The segstate for the current process displays unless the -p or -t flags are specified. All of the segstate entries display unless limited by the -l flag or the starting esid, start_esid and possible ending esid, end_esid. Specifying the -s flag limits the display to only those segstate entries matching the given segflags, matching pattern types, as well as their corresponding values. The -l flag limits the display to a maximum number of entries. The -n flag also prints the segnodes for the displayed data. Segnode entries are not included in the count when limiting the data with -l.i

-p pslot Specifies the process slot number.

-t tslot Specifies the thread slot number.

-s segflag:[value] Limites the display to the segstate entries matching that segflag and value.

-l limit Specifies the number of entries to print.

-n Prints the uadnodes for the displayed data.

select Recognized by the sel subcommand alias. Displays all select control blocks.

select p proc_slot Recognized by the sel subcommand alias. Displays select control blocks for process in specified slot.
Note: The p flag is not prefixed with a -.

select dev_id unique_id

Recognized by the sel subcommand alias. Displays select control blocks matching the specified device and unique IDs

set Display crash variables and values.

set allhex [no] Causes crash to use only hex values for both input and output as oppossed to a mixture of hex and decimal. Specify no to turn this option off.

set edit [emacs|gmacs|none|vi]

Sets command line editing mode.

set fpregs [yes|no|auto]

Specify whether or not floating-point registers should be displayed. If auto is used, the fpeu variable in the mstsave area determines when to display the registers.

set idarch [ppc|pwr|auto]

Set instruction decode architecture. auto detects the architecture from the system.

set logfile [filename]

Set logfile to given name, or turn off logging if no name is given.

set loglevel [0|1|2] Set logging granularity to:
0

coarse-only commands will be logged.

1

medium-commands and output to terminal will be logged.

2

fine-commands and all outputs will be logged, including redirected.

set prtype [type] Set the default print type. This is equivalent to print -d type.

set quiet [no] Suppress error messages concerning missing or swapped out threads and processes. Specify no to turn off this option.

socket[-] . . . Recognized by the sock subcommand alias. Displays the system socket structures. If the - (minus sign) flag is used, the socket buffers will also be displayed.

sr64 [-p pslot | -t tslot] [-l limit [-n [start_esid [end_esid]]

Recognized by the segst and seg subcommand aliases. Displays the effective segment IDs (esid) and their corresponding segvals for a 64-bit process. If you do not specify the -p or -t flags, sr64 uses the current process. Otherwise, it uses pslot as the process slot number for the desired process, or tslot as the thread slot number of a thread contained within the desired process. It lists all entries in the adspace unless a starting esid, start_esid and possible ending esid, end_esid is given. Also, it stops listing if the number of entries specified by the -l flag have printed. Since adspace_t holds 16 entries, each line consists of an esid, its corresponding value, and the 3 subsequent values following it in the adspace_t. The -n flag also prints the uadnodes for the displayed data. uadnode entries are not included in the count when limiting the data with -l.

-p pslot Specifies the process slot number.

-t tslot Specifies the thread slot number.

-l limit Specifies the number of entries to print.

-n Prints the uadnodes for the displayed data.

stack[ThreadTableEntry ] . . .

Recognized by the stk, s, kernel, and k subcommand aliases. Displays a dump of the kernel stack of the thread identified by ThreadTableEntry. If you do not specify an entry, information about the last running kernel thread is displayed. You cannot trace the stack of the current kernel thread on a running system.

stat Displays statistics found in the dump. These statistics include the panic message (if a panic occurred), time of crash, system name, and whether the MODS (here called xmalloc debug) is enabled.

status [ProcessorNumber] Displays a description of the kernel thread scheduled on the designated processor. If no processor is specified, the status subcommand displays information for all processors. The information displayed includes the processor number, thread identifier, thread table slot, process identifier, process table slot, and process name.

stream Displays the stream head table. The information printed is contained in an internal structure. The members of this internal structure are:

address Address of stream head

wq Address of streams write queue

dev Associated device number of the stream

read_error Read error on the stream

write_error Write error on the stream

flags Stream head flag values

push_cnt Number of modules pushed on the stream

wroff Write offset to prepend M_DATA

ioc_id ID of outstanding M_IOCTL request

pollq List of active polls

sigsq List of active M_SETSIGs

The flags structure member, if set, is based on combinations of the following values:

#define	Value	Description
F_STH_READ_ERROR	0x0001	M_ERROR with read error received, fail all read calls
F_STH_WRITE_ERROR	0x0002	M_ERROR with write error received, fail all writes
F_STH_HANGUP	0x0004	M_HANGUP received, no more data
F_STH_NDELON	0x0008	do TTY semantics for ONDELAY handling
F_STH_ISATTY	0x0010	this stream acts as a terminal
#define	Value	Description
F_STH_MREADON	0x0020	generate M_READ messages
F_STH_TOSTOP	0x0040	disallow background writes (for job control)
F_STH_PIPE	0x0080	stream is one end of a pipe or fifo
F_STH_WPIPE	0x0100	stream is the "write" side of a pipe
F_STH_FIFO	0x0200	stream is a fifo
F_STH_LINKED	0x040	stream has one or more lower streams linked
F_STH_CTTY	0x0800	stream controlling tty
F_STH_CLOSED	0x4000	stream has been closed, and should be freed
F_STH_CLOSING	0x8000	actively on the way down

symptom[-e]	Displays the symptom string for a dump. It is not valid on a running system. The optional -e option will create an error log entry containing the symptom string, and is normally only used by the system and not entered manually. The symptom string can be used to identify duplicate problems.
tcb [ThreadTableEntry] . . .	Displays the mstsave portion of the user structures of the named threads (see the user.h and mstsave.h header files). If you do not specify an entry, information about the last running thread displays. This subcommand replaces the pcb subcommand.
thread [-] [-r] [-p ProcessTableEntry \| -a Address \| ThreadTableEntry]
	Recognized by the th subcommand alias. Displays the contents of the thread table. The - (minus) flag displays a longer listing of the thread table. The -r flag displays only runnable threads. The -p flag displays only those threads which belong to the process identified by ProcessTableEntry. The -a flag displays the thread structure at Address. If ThreadTableEntry is given, only the corresponding thread is displayed.
trace [-r \| -m [-f]] [-k \| -s] [-r] [ThreadTableEntry ] . . .
	Recognized by the t subcommand alias. Displays a kernel stack trace of the thread identified by ThreadTableEntry. If you do not specify a thread table entry, information about the current thread is displayed. When using the -k flag, the stack frame addresses indicate the stack frame containing the link register value pointing to the function that is displayed. The -m flag causes trace to display the traceback associated with each mstsave area on the Current Save Area Chain (CSA), except the first. To see a traceback from the first mstsave area, specify the -f flag. When either the -m or -k flags are used, trace may also show the LR (link register) and top stack frame pointer. These are not part of the stack trace and are therefore marked with an asterisk. The -r flag will cause trace to use the kernel frame pointer set up by the kfp subcommand as its starting address instead of the frame pointer found in the SystemImageFile. The trace subcommand will stop and an error will be reported if an invalid frame pointer is encountered. The -s flag displays saved register information for each stack frame. With no flags, trace prints address information.
ts[TextAddress] . . .	Finds the text symbols closest to the given addresses.
tty[ d ] [ l] [ e ] [ Name \| Major [ Minor ] ]
	Displays the tty structures. If no parameters are specified, a short list of all open terminals is displayed. Selected terminals can be displayed by specifying the terminal name, such as tty1, or a major number with optional minor number. The flags modify the displayed information: the d flag displays driver information; the l flag displays line discipline information; and the e flag displays information for every module or driver present in the stream for the selected lines.
unhide symbol...	Unhide the specified symbol. See the hide subcommand.
unhide	Unhide all hidden symbols. See the hide subcommand.
user [-s] [ThreadTableEntry ] . . .
	Recognized by the uarea, u_area, and u subcommand aliases. Displays the uthread structure and the associated user structure of the thread identified by ThreadTableEntry. (See the/usr/include/sys/user.h file for the user structure definition.) If you do not specify the entry, the information about the last running thread displays. The -s flag limits the output to segment register information.
var	Recognized by the tunables, tunable, tune, and v subcommand aliases. Displays the tunable system parameters.
vfs [-] [VfsSlotNumber]	Recognized by the mount, mnt, and m subcommand aliases. The vfs uses the specified VfsSlotNumber to display an entry in the vfs table. Use the - flag to display the v-nodes associated with the vfs. The default displays the entire vfs table. See the sys/vfs.h header file for structure definitions. See examples 8 and 9 for samples of the vfs subcommand output.
vnode [VNodeAddress]	Displays data at the specified v-node address as a v-node. The address must be specified in hexadecimal notation. The default is to display all v-node structures. See the sys/vfs.h header file for structure definitions. See example 10 for a sample of the vnode subcommand output.
which symbol \| addr	Recognized by the wf subcommand alias. Displays the name of the kernel source file containing symbol or addr.
xmalloc	Recognized by the xm and malloc subcommand aliases. Prints information concerning the allocation and usage of kernel memory (the pinned_heap and the kernel_heap).
xmalloc [addr]	Recognized by the xm and malloc subcommand aliases. Prints xmalloc information about addr. If addr is not specified crash attempts to find the addresses involved in the system crash caused by the MODS.
xmalloc -s [addr]	Recognized by the xm and malloc subcommand aliases. Prints debug xmalloc allocation records associated with addr. Note: The -s flag requires that the memory overlay detection system (MODS) has been turned on.
xmalloc -h [addr]	Recognized by the xm and malloc subcommand aliases. Prints MODS xmalloc free list records associated with addr. Note: The -h flag requires that the memory overlay detection system (MODS) has been turned on.
xmalloc [-l] -f	Recognized by the xm and malloc subcommand aliases. Prints allocation records on free list from earliest-freed to latest-freed. The -l flag prints a long listing. Note: The -f flag requires that the memory overlay detection system (MODS) has been turned on.
xmalloc [-l] -a	Recognized by the xm and malloc subcommand aliases. Prints the allocation record table. The -l flag prints a long listing. Note: The -a flag requires that the memory overlay detection system (MODS) has been turned on.
xmalloc [-l] -p pageno	Recognized by the xm and malloc subcommand aliases. Prints page descriptor information for page pageno. The -l flag prints additional information.
xmalloc -d [addr]	Recognized by the xm and malloc subcommand aliases. Prints debug xmalloc allocation record hash chain associated with the record hash value for addr.
xmalloc -v	Recognized by the xm and malloc subcommand aliases. Verify allocation trailers of allocated records, and free fill patterns of freed records. Note: The -v flag requires that the memory overlay detection system (MODS) has been turned on.
!	Runs shell commands.
?	Displays summary of crash commands.

Flags

-a	Prints an assortment of data structures without user interaction to standard output.
-i	Reads this include file and to allow the structures declared in this file to be used with the print subcommand. You can supply several options on the command line with this flag.

Examples

To invoke the crash command with the default system image and kernel image files, enter:
```
crash
```
The crash command returns a > prompt and waits for you to enter a subcommand.
To invoke the crash command with the system image file and the UNIX kernel file, enter:
```
crash sysimage /unix
```
The crash program returns a > prompt and waits for you to enter a subcommand.

The following examples show examples of the usage of crash subcommands:
To write buffer 0 to the buff.0 file in binary format, enter:
```
buffer write 0
```
The crash command returns a > prompt and waits for you to enter a subcommand.
To set the segment map to thread slot number 1 and segment 2, then display ten words starting from address 2ff3b400, using segment 2 of thread slot number 1, enter:
```
>cm 1 2 
t1,2>od 2ff3b400 10 
```
To display the user structure from the unix kernel sys/user.h, enter:
```
crash -i sys/user.h <print user *u 
```
To display deadlocks anywhere on the system, enter:
```
>dlock
```

To display a deadlock involving a given thread 00d3f, enter:

>dlock 00d3f

The output is similar to:


Deadlock from tid 00d3f.  This tid waits for the first line lock, 
owned by Owner-Id that waits for the next line lock, and so on...  
     LOCK NAME    |   ADDRESS  |  OWNER-ID | WAITING FUNCTION 
           lockC1 | 0x001f79e0 |  Tid 113d | .lock_write_ppc 
                   called from :  .times + 0000020c 
Dump data incomplete.Only 0 bytes found out of 4.  
                   called from :  .file + 0000000b 
            lockC2 | 0x001f79e8 | Tid d3f | .lock_write_ppc 
                     called from :  .times + 000001c8 
Dump data incomplete.Only 0 bytes found out of 4.  
                   called from :  .file + 0000000b

To display the third entry in the vfs table, enter:

> vfs 3

The output is similar to:


VFS ADDRESS TYPE OBJECT STUB NUM FLAGS PATHS 
  3 1a62494  jfs 1a6d47c 1a6d650 5 D /dev/hd1 mounted over /u 
        flags:  C=disconnected D=device I=remote P=removable 
                R=readonly     S=shutdown U=unmounted Y=dummy

To display the v-node table data associated with the third entry in the vfs table, enter:

> vfs - 3

The output is similar to:


VFS ADDRESS TYPE  OBJECT    STUB NUM FLAGS PATHS 
  3 1a62494  jfs 1a6d47c 1a6d650   5 D   /dev/hd1 mounted over /u 
ADDRESS VFS MVFS VNTYPE FSTYPE COUNT ISLOT    INODE FLAGS 
1a6e0ac   3    -   vreg    jfs     1    -   18f82c0 
1a6e218   3    -   vreg    jfs     1    -   18f8770 
1a6e24c   3    -   vreg    jfs     1    -   18f8590 
1a6e17c   3    -   vdir    jfs     3    -   18f7f00 
1a6dea4   3    -   vreg    jfs     2    -   18f65b0 
1a6dfa8   3    -   vdir    jfs     5    -   18f6100
1a6d47c   3    -   vdir    jfs     1    -   18ea580 vfs_root

To display the v-node data for the address 1a6e078, enter


> vnode 1a6e078 
ADDRESS VFS MVFS VNTYPE FSTYPE COUNT ISLOT    DATAPTR FLAGS  
1a6e078   0    -   vreg    jfs     4     -    18f6790 
 Total VNODES printed 1

Files

/usr/sbin/crash	Contains the crash command.
/dev/mem	Default system image file
/unix	Default kernel file
*/usr/include/sys/.h**	Header files for table and structure information.

Related Information

The ksh command, mount command, pstat command, ps command, stty command.

Memory Overlay Detection System (MODS) in the AIX Version 4.3 Kernel Extensions and Device Support Programming Concepts.

I	INTERLOCK
W	WAITING
WW	WANT_WRITE (Complex_lock)
RD	READ_MODE(Complex_lock)
L	LOCKBIT (Simple_lock)
S	INSTR_ON (Simple_lock)

mblock Address	Recognized by the mblk subcommand alias. The mblock subcommand displays the allocated streams message block headers. The Address parameter is required. If Address is not supplied, the subcommand prints an error message stating that the address is required. See /usr/include/sys/stream.h header file for the msgb structure definitions. There is very little checking performed on the address passed in as the required parameter. The mblock subcommand accepts any address that falls on a 128-byte boundary. It is up to the user to be sure that a valid address is specified. To determine a valid address, run the queue subcommand. From the output of the queue subcommand, you will need to select a non-zero address in the head of the message queue, under the column heading HEAD, for either a read queue or a write queue.
mbuf Address	Displays system mbuf structures at the specified address.
mst [-f] [Address] . . .	Displays the mstsave portion of the uthread structure at the addresses specified (see the uthread.h and mstsave.h header files in /usr/include/sys). If you do not specify an address, it displays all of the mstsave entries on the CSA chain except the first. If you specify the -f flag the first mstsave area on the CSA chain displays.
ndb	Displays network kernel data structures either for a running system or a system dump. The ndb subcommand, short for network debugger, supports the following options:
?	Provides first-level help information.
help	Provides additional help information.
tcb [Addr]	Shows TCBs. The default is HEAD TCB.
udb [Addr]	Shows UDBs. The default is HEAD UDB.
socketsAddr	Shows sockets at the given address.
mbuf [Addr]	Shows the mbuf at the specified address.
ifnet [Addr]	Shows the ifnet structures at the specified address.
quit	Stops the running option.
xit	Exits the ndb submenu.
netm	Displays the most recent net_malloc_police record. Note: Requires that the Memory Overlay Detection System (MODS) be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.
netm -a	Displays all records, starting with the most recent. Note: Requires that the MODS be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.
netm addr	Displays records whose address or caller fields match the given address. Note: Requires that the MODS be enabled or that the network option net_malloc_police be turned on. For additional information on the net_malloc_police option, see the no command.
netstat	Equivalent to the command line version of the netstat command.
nm[Symbol] . . .	Displays symbol value and type as found in KernelFile.
od [...] [SymbolName* \| Address] [Count] [Format]
	Recognized by the rd subcommand alias. Dumps the Count parameter number of data values starting at the symbol value, or address given according to the Format parameter. Possible formats are the ascii, octal, longoct,decimal, longdec, character, hex, byte and instruction formats. The ASCII format is similar to the hex format but has an ASCII sidebar. The instruction format disassembles the number of instructions specified. The word "instruction" is abbreviated as the uppercase letter "I". The id subcommand is an alias for od...I. You can dereference a symbol or address by prefixing it with one or more asterisks.
ppd [ProcessorNumber *]
	Displays per-processor data area (PPDA) structures for the specified processor. If no processor is specified, the current processor selected by the cpu subcommand is used. If the asterisk argument is given, the PPDA of every enabled processor is displayed.
prall	Equivalent to crash -a from the command line.
print [Type] Address	Recognized by the pr, str, or struct subcommand aliases. Does dbx-style printing of structures. The -i option must be given on the command line to use this feature. Type is the name of the structure to be displayed.
print -d type	Recognized by the pr, str, or struct subcommand aliases. Sets the default type for subsequent print commands to type.
print -l offset\|name [-e end_val] [type] address
	Recognized by the pr, str, or struct subcommand aliases. Displays a linked list, starting at address using offset or structure member name as the location of the next pointer. Stop when next value of equals end_val. The default end_val is 0.
proc [-] [-r] [ProcessTableEntry] . . .
	Recognized by the ps and p subcommand aliases. Displays the process table, including the thread count (the number of threads in the process) and state of each process. (See the /usr/include/sys/proc.h file for this structure definition.) The -r flag displays only runnable processes. The - (minus) flag displays a longer listing of the process table.
qrun	Displays the list of scheduled streams queues. If there are no queues found for scheduling, the qrun subcommand prints a message stating that there are no queues scheduled for service.
queue [Address]	Recognized by the que subcommand alias. The queue subcommand displays the STREAMS queue. If the optional parameter, Address, is not supplied, crash will display information for all write queues available. Refer to the /usr/include/sys/stream.h header file for the queue structure definition. If you wish to see the information stored for a read queue, issue the queue subcommand with the read queue address specified as the Address parameter. When you issue the queue subcommand with the Address parameter, the column headings do not distinguish between the read queue and the write queue. One queue address will be displayed under the column heading, QUEUE and the other queue in the pair will be displayed under the column heading, OTHERQ. The write queue will have a numerically higher address than the read queue.
quit	Recognized by the q subcommand alias. Exits from the crash command.
search[-sn] name	Search the symbols table for name.
-s	Prints symbols matching name in the nm format. Also prints the symbol table entry for the last symbol found.
-n	Prevents the search from examining kernel extensions..
search[-n] addr	Search for the symbol with the largest value less than or equal to addr.

-p pslot	Specifies the process slot number.
-t tslot	Specifies the thread slot number.
-s segflag:[value]	Limites the display to the segstate entries matching that segflag and value.
-l limit	Specifies the number of entries to print.
-n	Prints the uadnodes for the displayed data.

select	Recognized by the sel subcommand alias. Displays all select control blocks.
select p proc_slot	Recognized by the sel subcommand alias. Displays select control blocks for process in specified slot. Note: The p flag is not prefixed with a -.
select dev_id unique_id
	Recognized by the sel subcommand alias. Displays select control blocks matching the specified device and unique IDs
set	Display crash variables and values.
set allhex [no]	Causes crash to use only hex values for both input and output as oppossed to a mixture of hex and decimal. Specify no to turn this option off.
set edit [emacs\|gmacs\|none\|vi]
	Sets command line editing mode.
set fpregs [yes\|no\|auto]
	Specify whether or not floating-point registers should be displayed. If auto is used, the fpeu variable in the mstsave area determines when to display the registers.
set idarch [ppc\|pwr\|auto]
	Set instruction decode architecture. auto detects the architecture from the system.
set logfile [filename]
	Set logfile to given name, or turn off logging if no name is given.
set loglevel [0\|1\|2]	Set logging granularity to: 0 coarse-only commands will be logged. 1 medium-commands and output to terminal will be logged. 2 fine-commands and all outputs will be logged, including redirected.
set prtype [type]	Set the default print type. This is equivalent to print -d type.
set quiet [no]	Suppress error messages concerning missing or swapped out threads and processes. Specify no to turn off this option.
socket[-] . . .	Recognized by the sock subcommand alias. Displays the system socket structures. If the - (minus sign) flag is used, the socket buffers will also be displayed.
sr64 [-p pslot \| -t tslot] [-l limit [-n [start_esid [end_esid]]
	Recognized by the segst and seg subcommand aliases. Displays the effective segment IDs (esid) and their corresponding segvals for a 64-bit process. If you do not specify the -p or -t flags, sr64 uses the current process. Otherwise, it uses pslot as the process slot number for the desired process, or tslot as the thread slot number of a thread contained within the desired process. It lists all entries in the adspace unless a starting esid, start_esid and possible ending esid, end_esid is given. Also, it stops listing if the number of entries specified by the -l flag have printed. Since adspace_t holds 16 entries, each line consists of an esid, its corresponding value, and the 3 subsequent values following it in the adspace_t. The -n flag also prints the uadnodes for the displayed data. uadnode entries are not included in the count when limiting the data with -l.

-p pslot	Specifies the process slot number.
-t tslot	Specifies the thread slot number.
-l limit	Specifies the number of entries to print.
-n	Prints the uadnodes for the displayed data.

stack[ThreadTableEntry ] . . .
	Recognized by the stk, s, kernel, and k subcommand aliases. Displays a dump of the kernel stack of the thread identified by ThreadTableEntry. If you do not specify an entry, information about the last running kernel thread is displayed. You cannot trace the stack of the current kernel thread on a running system.
stat	Displays statistics found in the dump. These statistics include the panic message (if a panic occurred), time of crash, system name, and whether the MODS (here called xmalloc debug) is enabled.
status [ProcessorNumber]	Displays a description of the kernel thread scheduled on the designated processor. If no processor is specified, the status subcommand displays information for all processors. The information displayed includes the processor number, thread identifier, thread table slot, process identifier, process table slot, and process name.
stream	Displays the stream head table. The information printed is contained in an internal structure. The members of this internal structure are:

`address`	Address of stream head
`wq`	Address of streams write queue
`dev`	Associated device number of the stream
`read_error`	Read error on the stream
`write_error`	Write error on the stream
`flags`	Stream head flag values
`push_cnt`	Number of modules pushed on the stream
`wroff`	Write offset to prepend M_DATA
`ioc_id`	ID of outstanding M_IOCTL request
`pollq`	List of active polls
`sigsq`	List of active M_SETSIGs