ipmiseld(8) | ipmiseld | ipmiseld(8) |
ipmiseld - IPMI SEL logging daemon
ipmiseld [OPTION...]
The ipmiseld daemon polls the system event log (SEL) of specified hosts and stores the logs into the local syslog. By default, the daemon can also make best efforts to manage the remote SEL's buffer to ensure events are never lost. Recent logging data will be cached to disk to ensure that SEL events are not missed in the event the client or server is rebooted.
Many of the options for this daemon are very similar to the ipmi-sel(8) tool. It can be configured to log the local host, a remote host, or a range of hosts to the local syslog. It can be configured via the command line arguments listed below or via the /etc/freeipmi//ipmiseld.conf configuration file.
Listed below are general IPMI options, tool specific options, trouble shooting information, workaround information, examples, and known issues. For a general introduction to FreeIPMI please see freeipmi(7).
The following options are general options for configuring IPMI communication and executing general tool commands.
0 - Authentication Algorithm = None; Integrity Algorithm = None; Confidentiality Algorithm = None
1 - Authentication Algorithm = HMAC-SHA1; Integrity Algorithm = None; Confidentiality Algorithm = None
2 - Authentication Algorithm = HMAC-SHA1; Integrity Algorithm = HMAC-SHA1-96; Confidentiality Algorithm = None
3 - Authentication Algorithm = HMAC-SHA1; Integrity Algorithm = HMAC-SHA1-96; Confidentiality Algorithm = AES-CBC-128
6 - Authentication Algorithm = HMAC-MD5; Integrity Algorithm = None; Confidentiality Algorithm = None
7 - Authentication Algorithm = HMAC-MD5; Integrity Algorithm = HMAC-MD5-128; Confidentiality Algorithm = None
8 - Authentication Algorithm = HMAC-MD5; Integrity Algorithm = HMAC-MD5-128; Confidentiality Algorithm = AES-CBC-128
11 - Authentication Algorithm = HMAC-MD5; Integrity Algorithm = MD5-128; Confidentiality Algorithm = None
12 - Authentication Algorithm = HMAC-MD5; Integrity Algorithm = MD5-128; Confidentiality Algorithm = AES-CBC-128
15 - Authentication Algorithm = HMAC-SHA256; Integrity Algorithm = None; Confidentiality Algorithm = None
16 - Authentication Algorithm = HMAC-SHA256; Integrity Algorithm = HMAC_SHA256_128; Confidentiality Algorithm = None
17 - Authentication Algorithm = HMAC-SHA256; Integrity Algorithm = HMAC_SHA256_128; Confidentiality Algorithm = AES-CBC-128
The following options are specific to ipmiseld.
The output format of log messages can be adjusted via the --system-event-format, --oem-timestamped-event-format and --oem-non-timestamped-event-format options. Options such as --interpret-oem-data, --entity-sensor-names, and --non-abbreviated-units can further adjust the output format. The following conversion directives will allow the user to output specifics of each SEL event that occurs.
For System, OEM timestamped, and OEM non-timestamped events
%h - target host, useful if logging from multiple hosts
%i - record ID in decimal
%I - event state interpretation (NOMINAL, WARNING, or CRITICAL)
For System and OEM timestamped events
%t - time in format H:M:S using 24 hour clock
%d - date in format D-M-YEAR
For System events
%T - sensor type
%s - sensor name
%e - event data 1 string
%f - event data 2 string [2]
%h - event data 3 string
%c - combined event data 2 and event data 3 string
%p - event data 2 previous state string
%S - event data 2 severity string
%E - combined event data 1, 2, and 3 string
%k - event direction
For OEM timestamped events
%m - manufacturer id
For OEM timestamped and OEM non-timestamped events
%o - oem data in hex
%O - OEM supplied string describing the event (depends on manufacturer)
Multiple hosts can be input either as an explicit comma separated lists of hosts or a range of hostnames in the general form: prefix[n-m,l-k,...], where n < m and l < k, etc. The later form should not be confused with regular expression character classes (also denoted by []). For example, foo[19] does not represent foo1 or foo9, but rather represents a degenerate range: foo19.
This range syntax is meant only as a convenience on clusters with a prefixNN naming convention and specification of ranges should not be considered necessary -- the list foo1,foo9 could be specified as such, or by the range foo[1,9].
Some examples of range usage follow:
foo[01-05] instead of foo01,foo02,foo03,foo04,foo05
foo[7,9-10] instead of foo7,foo9,foo10
foo[0-3] instead of foo0,foo1,foo2,foo3
As a reminder to the reader, some shells will interpret brackets ([ and ]) for pattern matching. Depending on your shell, it may be necessary to enclose ranged lists within quotes.
In-band IPMI Communication will be used when the host "localhost" is specified. This allows the user to add the localhost into the hostranged output.
Most often, IPMI problems are due to configuration problems.
IPMI over LAN problems involve a misconfiguration of the remote machine's BMC. Double check to make sure the following are configured properly in the remote machine's BMC: IP address, MAC address, subnet mask, username, user enablement, user privilege, password, LAN privilege, LAN enablement, and allowed authentication type(s). For IPMI 2.0 connections, double check to make sure the cipher suite privilege(s) and K_g key are configured properly. The ipmi-config(8) tool can be used to check and/or change these configuration settings.
Inband IPMI problems are typically caused by improperly configured drivers or non-standard BMCs.
In addition to the troubleshooting tips below, please see WORKAROUNDS below to also if there are any vendor specific bugs that have been discovered and worked around.
Listed below are many of the common issues for error messages. For additional support, please e-mail the <freeipmi-users@gnu.org> mailing list.
"username invalid" - The username entered (or a NULL username if none was entered) is not available on the remote machine. It may also be possible the remote BMC's username configuration is incorrect.
"password invalid" - The password entered (or a NULL password if none was entered) is not correct. It may also be possible the password for the user is not correctly configured on the remote BMC.
"password verification timeout" - Password verification has timed out. A "password invalid" error (described above) or a generic "session timeout" (described below) occurred. During this point in the protocol it cannot be differentiated which occurred.
"k_g invalid" - The K_g key entered (or a NULL K_g key if none was entered) is not correct. It may also be possible the K_g key is not correctly configured on the remote BMC.
"privilege level insufficient" - An IPMI command requires a higher user privilege than the one authenticated with. Please try to authenticate with a higher privilege. This may require authenticating to a different user which has a higher maximum privilege.
"privilege level cannot be obtained for this user" - The privilege level you are attempting to authenticate with is higher than the maximum allowed for this user. Please try again with a lower privilege. It may also be possible the maximum privilege level allowed for a user is not configured properly on the remote BMC.
"authentication type unavailable for attempted privilege level" - The authentication type you wish to authenticate with is not available for this privilege level. Please try again with an alternate authentication type or alternate privilege level. It may also be possible the available authentication types you can authenticate with are not correctly configured on the remote BMC.
"cipher suite id unavailable" - The cipher suite id you wish to authenticate with is not available on the remote BMC. Please try again with an alternate cipher suite id. It may also be possible the available cipher suite ids are not correctly configured on the remote BMC.
"ipmi 2.0 unavailable" - IPMI 2.0 was not discovered on the remote machine. Please try to use IPMI 1.5 instead.
"connection timeout" - Initial IPMI communication failed. A number of potential errors are possible, including an invalid hostname specified, an IPMI IP address cannot be resolved, IPMI is not enabled on the remote server, the network connection is bad, etc. Please verify configuration and connectivity.
"session timeout" - The IPMI session has timed out. Please reconnect. If this error occurs often, you may wish to increase the retransmission timeout. Some remote BMCs are considerably slower than others.
"device not found" - The specified device could not be found. Please check configuration or inputs and try again.
"driver timeout" - Communication with the driver or device has timed out. Please try again.
"message timeout" - Communication with the driver or device has timed out. Please try again.
"BMC busy" - The BMC is currently busy. It may be processing information or have too many simultaneous sessions to manage. Please wait and try again.
"could not find inband device" - An inband device could not be found. Please check configuration or specify specific device or driver on the command line.
"driver timeout" - The inband driver has timed out communicating to the local BMC or service processor. The BMC or service processor may be busy or (worst case) possibly non-functioning.
"internal IPMI error" - An IPMI error has occurred that FreeIPMI does not know how to handle. Please e-mail <freeipmi-users@gnu.org> to report the issue.
Some timestamps in the SEL may report a date of 1-Jan-1970, the epoch for SEL timestamps. This timestamp is not necessarily incorrect. It usually indicates a hardware event that occurred before a timestamp in firmware has been initialized. For example, certain hardware components will have their internal clocks reset during a power cycle.
However, if the internal clock of the SEL appears to be regularly incorrect, you may need to set the SEL time. This can be done using bmc-device(8).
The following are common SEL related messages.
"sel config file parse error" - A parse error was found in the sel event interpretation configuration file. Please see freeipmi_interpret_sel.conf(5).
With so many different vendors implementing their own IPMI solutions, different vendors may implement their IPMI protocols incorrectly. The following describes a number of workarounds currently available to handle discovered compliance issues. When possible, workarounds have been implemented so they will be transparent to the user. However, some will require the user to specify a workaround be used via the -W option.
The hardware listed below may only indicate the hardware that a problem was discovered on. Newer versions of hardware may fix the problems indicated below. Similar machines from vendors may or may not exhibit the same problems. Different vendors may license their firmware from the same IPMI firmware developer, so it may be worthwhile to try workarounds listed below even if your motherboard is not listed.
If you believe your hardware has an additional compliance issue that needs a workaround to be implemented, please contact the FreeIPMI maintainers on <freeipmi-users@gnu.org> or <freeipmi-devel@gnu.org>.
assumeio - This workaround flag will assume inband interfaces communicate with system I/O rather than being memory-mapped. This will work around systems that report invalid base addresses. Those hitting this issue may see "device not supported" or "could not find inband device" errors. Issue observed on HP ProLiant DL145 G1.
spinpoll - This workaround flag will inform some inband drivers (most notably the KCS driver) to spin while polling rather than putting the process to sleep. This may significantly improve the wall clock running time of tools because an operating system scheduler's granularity may be much larger than the time it takes to perform a single IPMI message transaction. However, by spinning, your system may be performing less useful work by not contexting out the tool for a more useful task.
authcap - This workaround flag will skip early checks for username capabilities, authentication capabilities, and K_g support and allow IPMI authentication to succeed. It works around multiple issues in which the remote system does not properly report username capabilities, authentication capabilities, or K_g status. Those hitting this issue may see "username invalid", "authentication type unavailable for attempted privilege level", or "k_g invalid" errors. Issue observed on Asus P5M2/P5MT-R/RS162-E4/RX4, Intel SR1520ML/X38ML, and Sun Fire 2200/4150/4450 with ELOM.
nochecksumcheck - This workaround flag will tell FreeIPMI to not check the checksums returned from IPMI command responses. It works around systems that return invalid checksums due to implementation errors, but the packet is otherwise valid. Users are cautioned on the use of this option, as it removes validation of packet integrity in a number of circumstances. However, it is unlikely to be an issue in most situations. Those hitting this issue may see "connection timeout", "session timeout", or "password verification timeout" errors. On IPMI 1.5 connections, the "noauthcodecheck" workaround may also needed too. Issue observed on Supermicro X9SCM-iiF, Supermicro X9DRi-F, and Supermicro X9DRFR.
idzero - This workaround flag will allow empty session IDs to be accepted by the client. It works around IPMI sessions that report empty session IDs to the client. Those hitting this issue may see "session timeout" errors. Issue observed on Tyan S2882 with M3289 BMC.
unexpectedauth - This workaround flag will allow unexpected non-null authcodes to be checked as though they were expected. It works around an issue when packets contain non-null authentication data when they should be null due to disabled per-message authentication. Those hitting this issue may see "session timeout" errors. Issue observed on Dell PowerEdge 2850,SC1425. Confirmed fixed on newer firmware.
forcepermsg - This workaround flag will force per-message authentication to be used no matter what is advertised by the remote system. It works around an issue when per-message authentication is advertised as disabled on the remote system, but it is actually required for the protocol. Those hitting this issue may see "session timeout" errors. Issue observed on IBM eServer 325.
endianseq - This workaround flag will flip the endian of the session sequence numbers to allow the session to continue properly. It works around IPMI 1.5 session sequence numbers that are the wrong endian. Those hitting this issue may see "session timeout" errors. Issue observed on some Sun ILOM 1.0/2.0 (depends on service processor endian).
noauthcodecheck - This workaround flag will tell FreeIPMI to not check the authentication codes returned from IPMI 1.5 command responses. It works around systems that return invalid authentication codes due to hashing or implementation errors. Users are cautioned on the use of this option, as it removes an authentication check verifying the validity of a packet. However, in most organizations, this is unlikely to be a security issue. Those hitting this issue may see "connection timeout", "session timeout", or "password verification timeout" errors. Issue observed on Xyratex FB-H8-SRAY, Intel Windmill, Quanta Winterfell, and Wiwynn Windmill.
intel20 - This workaround flag will work around several Intel IPMI 2.0 authentication issues. The issues covered include padding of usernames, and password truncation if the authentication algorithm is HMAC-MD5-128. Those hitting this issue may see "username invalid", "password invalid", or "k_g invalid" errors. Issue observed on Intel SE7520AF2 with Intel Server Management Module (Professional Edition).
supermicro20 - This workaround flag will work around several Supermicro IPMI 2.0 authentication issues on motherboards w/ Peppercon IPMI firmware. The issues covered include handling invalid length authentication codes. Those hitting this issue may see "password invalid" errors. Issue observed on Supermicro H8QME with SIMSO daughter card. Confirmed fixed on newerver firmware.
sun20 - This workaround flag will work work around several Sun IPMI 2.0 authentication issues. The issues covered include invalid lengthed hash keys, improperly hashed keys, and invalid cipher suite records. Those hitting this issue may see "password invalid" or "bmc error" errors. Issue observed on Sun Fire 4100/4200/4500 with ILOM. This workaround automatically includes the "opensesspriv" workaround.
opensesspriv - This workaround flag will slightly alter FreeIPMI's IPMI 2.0 connection protocol to workaround an invalid hashing algorithm used by the remote system. The privilege level sent during the Open Session stage of an IPMI 2.0 connection is used for hashing keys instead of the privilege level sent during the RAKP1 connection stage. Those hitting this issue may see "password invalid", "k_g invalid", or "bad rmcpplus status code" errors. Issue observed on Sun Fire 4100/4200/4500 with ILOM, Inventec 5441/Dell Xanadu II, Supermicro X8DTH, Supermicro X8DTG, Intel S5500WBV/Penguin Relion 700, Intel S2600JF/Appro 512X, Quanta QSSC-S4R/Appro GB812X-CN, and Dell C5220. This workaround is automatically triggered with the "sun20" workaround.
integritycheckvalue - This workaround flag will work around an invalid integrity check value during an IPMI 2.0 session establishment when using Cipher Suite ID 0. The integrity check value should be 0 length, however the remote motherboard responds with a non-empty field. Those hitting this issue may see "k_g invalid" errors. Issue observed on Supermicro X8DTG, Supermicro X8DTU, and Intel S5500WBV/Penguin Relion 700, and Intel S2600JF/Appro 512X.
assumesystemevent - This workaround option will assume invalid SEL record types are system event records. Records may be formatted correctly but report invalid record types. Those hitting this issue may see "Unknown SEL Record Type" errors. Output may be unknown, pray for the best. This option is confirmed to work around compliances issues on HP DL 380 G5 motherboards.
No IPMI 1.5 Support - Some motherboards that support IPMI 2.0 have been found to not support IPMI 1.5. Those hitting this issue may see "ipmi 2.0 unavailable" or "connection timeout" errors. This issue can be worked around by using IPMI 2.0 instead of IPMI 1.5 by specifying --driver-type=LAN_2_0. Issue observed on HP Proliant DL 145.
The following motherboards are confirmed to have atleast some support by the --interpret-oem-data option. While highly probable the OEM data interpretations would work across other motherboards by the same manufacturer, there are no guarantees. Some of the motherboards below may be rebranded by vendors/distributors.
Dell Poweredge 2900, Dell Poweredge 2950, Dell Poweredge R610, Dell Poweredge R710, Fujitsu iRMC S1 and iRMC S2 systems, Intel S5500WB/Penguin Computing Relion 700, Intel S2600JF/Appro 512X, Intel S5000PAL, Inventec 5441/Dell Xanadu II, Inventec 5442/Dell Xanadu III, Quanta S99Q/Dell FS12-TY, Quanta QSSC-S4R/Appro GB812X-CN, Sun X4140 Supermicro X7DBR-3, Supermicro X7DB8, Supermicro X8DTN, Supermicro X7SBI-LN4, Supermicro X8DTH, Supermicro X8DTG, Supermicro X8DTU, Supermicro X8DT3-LN4F, Supermicro X8DTU-6+, Supermicro X8DTL, Supermicro X8DTL-3F, Supermicro X8SIL-F, Supermicro X9SCL, Supermicro X9SCM, Supermicro X8DTN+-F, Supermicro X8SIE, Supermicro X9SCA-F-O, Supermicro H8DGU-F, Supermicro X9DRi-F, Supermicro X9DRI-LN4F+, Supermicro X9SPU-F-O, Supermicro X9SCM-iiF, Wistron/Dell Poweredge C6220.
On older operating systems, if you input your username, password, and other potentially security relevant information on the command line, this information may be discovered by other users when using tools like the ps(1) command or looking in the /proc file system. It is generally more secure to input password information with options like the -P or -K options. Configuring security relevant information in the FreeIPMI configuration file would also be an appropriate way to hide this information.
In order to prevent brute force attacks, some BMCs will temporarily "lock up" after a number of remote authentication errors. You may need to wait awhile in order to this temporary "lock up" to pass before you may authenticate again.
/etc/freeipmi//ipmiseld.conf /var/cache/ipmiseld/
Report bugs to <freeipmi-users@gnu.org> or <freeipmi-devel@gnu.org>.
Copyright (C) 2012-2015 Lawrence Livermore National Security, LLC.
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
freeipmi(7), ipmi-sel(8), ipmiseld.conf(5), bmc-device(8), ipmi-config(8), freeipmi_interpret_sel.conf(5)
http://www.gnu.org/software/freeipmi/
2019-01-23 | ipmiseld 1.6.3 |