| XYMOND_RRD(8) | System Manager's Manual | XYMOND_RRD(8) |
xymond_rrd - xymond worker module for updating Xymon RRD files
xymond_channel --channel=status xymond_rrd [options]
xymond_channel --channel=data xymond_rrd [options]
xymond_rrd is a worker module for xymond, and as such it is normally run via the xymond_channel(8) program. It receives "status" and "data" messages from xymond via stdin, and updates the RRD databases used to generate trend-graphs.
Clients can send data to Xymon using both status- and data- messages. So you will normally run two instances of this module, once for the "status" channel and once for the "data" channel.
xymond_rrd understands data sent by the LARRD 0.43c client-side scripts (the so-called "bottom-feeder" scripts). So you still want to install the LARRD bottom-feeders on the clients you monitor.
Note: For certain types of data, the RRD files used by Xymon are imcompatible with those generated by the Big Brother LARRD add-on. See the COMPATIBILITY section below.
If the process exits, xymond_rrd will re-launch it.
The following RRD-file datasets are generated by xymond_rrd:
Many data-collection scripts report data in the form "NAME : value" or "NAME = value". So a generic module in xymond_rrd allows for easy tracking of this type of data.
The "ncv" module will automatically detect all occurrences of a "NAME : value" or "NAME = value" string in a status message, and generate an RRD file holding all of the name/value data found in the message (unless you use SPLITNCV, see above). The colon- or equal-sign must be present - if there is only whitespace, this module will fail.
Only the valid letters (A-Z, a-z) and digits (0-9) are used in the dataset names; whitespace and other characters are stripped off automatically. Only the first 19 characters of a dataset name are used (this is an RRD limitation). Underscore '_' is not allowed, even though RRDtool permits this, and will be stripped from the name.
When using the alternative SPLITNCV_testname, the dataset name is not limited in length, and non-valid characters are changed to underscores instead of being stripped off. The dataset inside the resulting rrd file is always "lambda".
Note that each "NAME : value" must be on a line by itself. If you have a custom script generating the status- or data-message that is fed into the NCV handler, make sure it inserts a newline before each of the data-items you want to track.
Any lines in the status message prepended with a "<!-- ncv_skip -->" will be skipped by the module. This can be used to prevent unneeded RRD files from an existing dataset from being created.
A line prepended with a "<!-- ncv_skipstart -->" will be ignored, along with all subsequent lines until a line starting with "<!-- ncv_skipend -->" is found, at which point processing will resume. This can be used to ignore explanatory or other text with a mostly-ncv message.
"<!-- ncv_ignore -->" can be used to ignore certain text at the beginning of a line, up until a closing '</-->' tag on the same line, at which point the line will continue to be processed as usual. Wrapping is not supported; but skipstart/skipend can be used to handle multiple lines.
A bare "<!-- ncv_end -->" on its own line will stop further NCV processing of that message.
All of these ncv_ terms are case-sensitive. Note that if you have full control over your NCV output, it is most efficient to have NCV data near the top of your message and use "<!-- ncv_end -->" once your data is complete.
To enable the ncv module for a status, add a "COLUMNNAME=ncv" to the TEST2RRD setting and the COLUMNNAME to the GRAPHS setting in xymonserver.cfg(5) , then restart Xymon. Xymon will now send all status-messages for the column COLUMNNAME through the xymond_rrd ncv-handler.
The name of the RRD file will be COLUMNNAME.rrd. When using SPLITNCV, the name of the RRD file will be COLUMNAME,DATASETNAME.rrd.
By default, all of the datasets are generated as the RRD type "DERIVE" which works for all types of monotonically increasing counters. If you have data that are of the type GAUGE, you can override the default via an environment variable NCV_COLUMNNAME (or SPLITNCV_COLUMNAME).
E.g. if you are using the bb-mysqlstatus script from
www.deadcat.net to collect data about your MySQL server, it generates a
report in the column called "mysql". One data item is the average
number of queries/second, which must be logged in the RRD file as type
"GAUGE". To do that, add the following to xymonserver.cfg:
NCV_mysql="Queriespersecondavg:GAUGE"
If you have multiple datasets that you myst define, add them to the
environment variable separated by commas, e.g.
NCV_mysql="Uptime:NONE,Queriespersecondavg:GAUGE"
The dataset type "NONE" used above causes xymond_rrd to ignore this data, it is not included in the RRD file.
You can use "*" as the dataset name to match all
datasets not listed. E.g.
NCV_weather="Rain:DERIVE,*:GAUGE"
will cause the "Rainfall" dataset to be of type DERIVE, and all
others of type GAUGE. If you want to track only a few of the variables in
your data, you can use "*:NONE" to drop any dataset not explicitly
listed.
For a more detailed "how to" description, see the on-line HTML documentation of "How to create graph custom data" available in the Help menu section on your Xymon server.
xymond_rrd provides a mechanism to send a copy of isolated metric data to a single external processor for further processing. This can be used to inject metric data that xymond_rrd has prepared into other storage systems, such as OpenTSDB, graphite, etc. The data is printed in a format nearly suitable for injection using rrdupdate(1) and easily transformable to other formats. If the process exits, xymond_rrd will re-launch it automatically.
xymond_rrd provides a simple mechanism for adding custom graphs to the set of data collected on your Xymon server. By adding the "--extra-script" and "--extra-tests" options, data reported to Xymon from selected tests are passed to an external script, which can define the RRD data-sets to store in an RRD file.
NOTE: For performance reasons, you should not use this mechanism for large amounts of data. The overhead involved in storing the received message to disk and launching the script is significantly larger than the normal xymond_rrd overhead. So if you have a large number of reports for a given test, you should consider implementing it in C and including it in the xymond_rrd tool or writing a separate stream listener that injects appropriate "trends" data messages back to xymond.
Apart from writing the script, You must also add a section to graphs.cfg(5) so that showgraph.cgi(1) knows how to generate the graph from the data stored in the RRD file. To make the graphs actually show up on the status-page and/or the "trends" page, add the name of the new graph to the TEST2RRD and/or GRAPHS setting in xymonserver.cfg(5).
The script is invoked for each message that arrives, where the test-name matches one of the testnames given in the "--extra-tests" option. The script receives three command-line parameters:
The script must process the data that is reported, and generate the following output:
If you want to store the data in multiple RRD files, the script can just print out more sequences of data-set definitions, RRD filenames and RRD values. If the data-set definitions are identical to the previous definition, you need not print the data-set definitions again - just print a new RRD filename and value.
The following sample script for tracking weather data shows how to use this mechanism. It assumes the status message include lines like these:
green Weather in Copenhagen is FAIR Temperature: 21 degrees Celsius Wind: 4 m/s Humidity: 72 % Rainfall: 5 mm since 6:00 AM
A shell-script to track all of these variables could be written like this:
#!/bin/sh
# Input parameters: Hostname, testname (column), and messagefile
HOSTNAME="$1"
TESTNAME="$2"
FNAME="$3"
if [ "$TESTNAME" = "weather" ]
then
# Analyze the message we got
TEMP=`grep "^Temperature:" $FNAME | awk '{print $2}'`
WIND=`grep "^Wind:" $FNAME | awk '{print $2}'`
HMTY=`grep "^Humidity:" $FNAME | awk '{print $2}'`
RAIN=`grep "^Rainfall:" $FNAME | awk '{print $2}'`
# The RRD dataset definitions
echo "DS:temperature:GAUGE:600:-30:50"
echo "DS:wind:GAUGE:600:0:U"
echo "DS:humidity:GAUGE:600:0:100"
echo "DS:rainfall:DERIVE:600:0:100"
# The filename
echo "weather.rrd"
# The data
echo "$TEMP:$WIND:$HMTY:$RAIN"
fi
exit 0
Some of the RRD files generated by xymond_rrd are incompatible with the files generated by the Big Brother LARRD add-on:
| Version 4.3.30: 4 Sep 2019 | Xymon |