NMONVisualizer

Overview

NMONVisualizer supports generating custom reports from the command line. Custom reports are defined in an XML file format, detailed here.

Concepts

Four chart types are supported: line charts, bar charts, interval charts and histograms. Line charts always have a time-based X axis; only the values charted and the Y axis are defined. Interval and histogram charts are similar except that the X axis plots all the intervals that are currently defined or the range of values in the histogram, respectively. A bar chart plots at least one aggregate statistic; both the Y axis and the columns (bar) names can be specified.

To create lines or bars, one or more <data> elements are specified for each chart. The data can either be all the data for a specific data type (<type>) or one or more fields from the data type (<field>). In either case, the type and / or field does not have to be exact; regular expression matching and other specific substitutions are supported.

In general, naming of lines and bars on a particular chart is very important. If a type or field is defined more than once or if renaming causes duplicates, only a single instance will appear on the chart. To assist with this, support exists for renaming data on the resulting chart.

By default, a chart is created using data from all hosts (i.e. parsed data sets). An optional <host> element can be used to restrict chart data to specific host names (exact or regex) or operating systems. Note that the program makes a distiction between (and processes differently) report XML files that should be run for a single host (i.e. a dataset report) and XML files that should be run across all hosts (i.e. a summary report, the 'All Systems' view). Therefore, it is best practice to have at least two report XML files defined: one to generate a set of charts for each host and another to generate charts that aggregate data for all hosts. Multiple, similar chart definitions differing only in their <host> specifiction are usually not required. Instead, a common report will be applied to each host.

Charts

The root element of the XML must be <charts>. Within that, there can be any number of chart elements; each one will generate a separate graph. The four chart types each have their own element: <linechart>, <barchart>, <intervalchart> and <histogram>. All chart elements support the following attributes:

• name - the name of the chart. This value is required. This will be used as the title of the chart.
• shortName - a more concise name for the chart. This will be used as the file name when the chart is output as well as the tab name in the GUI. If not specified the short name will default to the full name.
• stacked - a a boolean value indicating that the lines or bars in the chart should be stacked rather than separate. On a line chart this causes the area between the lines to be filled in. This attribute is ignored for interval charts. The default value is false.
• width and height - the size of the chart when saved or copied, in pixels. The default is ¼ HD (1920 / 2 x 1080 / 2).

Bar charts also support a subtractionNeeded attribute to specify that the bar value should be subtracted from the previous bar's value. This is useful for stacked charts where the underlying data for each field is a sum of other data.

Line and interval charts support a showDataPoints attribute that defaults to true. A value of false will prevent the chart from displaying any data point markers on the chart lines.

Histograms support a bins attribute to specify the number of bins to use in the chart. Each bin is displayed as a separate bar on the chart. This value defaults to 10. Histograms also support the ability to display annotation markers on a chart via the showMarkers attribute and <marker> sub-elements. Each <marker> element requires a stat attribute that corresponds to a valid statistic. By default, showMarkers is true and markers are displayed for AVERAGE, MEDIAN and PERCENTILE_95.

Naming & Legends

By default, lines and bars as displayed in a chart legend, are named by their field. On bar charts, the X axis displays a category which defaults to the type name. Optional attributes exist to change the defaults.

For line and interval charts, the attribute is linesNamedBy. This is most useful when charting multiple types, each with the same field name. For bar charts, there are two attributes: barsNamedBy and categoriesNamedBy; these should not be set to the same value. Histograms support barsNamedBy.

All charts also support a subtitledBy attribute that adds a subtitle to the chart. By default, the subtitle naming mode is HOST. Note that the naming mode is only applied if there is a single instance of the given value for the mode. For example, if the mode is HOST, the hostname will not be displayed if the chart displays data from multiple system. Similarly, for the compound naming modes, the actual subtitle may be shortened if one of the values is not unique for the charted data.

The following naming modes are supported:

• HOST - the hostname of the dataset
• TYPE - the name of the data type
• FIELD - the name of the field
• STAT - the name of a statistic
• HOST_TYPE - the hostname plus the type
• HOST_FIELD - the hostname plus the field
• HOST_STAT - the hostname plus the stat
• TYPE_FIELD - the type name plus the field
• TYPE_STAT - the type name plus the stat
• FIELD_STAT - the field name plus the stat
• DATE - the initial datetime of the data being charted, formatted as HH:mm:ss MMM dd.
  The format can be changed by setting the dateFormat attribute in the chart element. Any format supported by SimpleDateFormat is allowed. In addition, any of the above naming modes can be enclosed in single quotes. For example <linechart name="CPU Utilization" shortName="CPU" stacked="true" subtitledBy="DATE" dateFormat="HH:mm 'HOST' 'TYPE'"> would create a subtitle like 14:13 app1.foobar.com CPU_ALL.

Note that these values must be capitalized in the XML.

Data

Each chart must contain at least one <data> element to define the data to be graphed. The data element must have one <type> element and zero or more <field> elements as children. Type elements are used to match data types; field elements are used to match data within the given type. If no field element is specified, all fields for the matching types will be graphed.

The <data> element can have an optional stat attribute. The default value for bar and interval charts is AVERAGE. For line charts, this attribute is ignored – the raw data is always plotted.

Y Axis

Optional <yAxis> and <yAxis2> elements are supported for all chart types. Both support a label attribute which specifies the name of the axis. The primary Y axis can also be set to a percentage scale with the asPercent attribute, which must be a boolean. Note that to actually use the secondary Y axis, subsequent <data> elements must specify the useYAxis2 attribute as a boolean.

X Axis

An optional <xAxis> element is supported for line, interval and bar charts. The label attribute specifies the name of the axis. Percentages and secondary axes are not supported.

Statistics

In addition to graphing data as-is, aggregate data can also be charted. This is done by specifing the stat attribute in a data element.

The following statistics are supported:

• AVERAGE
• WEIGHTED_AVERAGE - this statistic calculates an average that weights larger values more than smaller values, especially values closer to 0. This is useful for data that have long idle periods followed by bursts. See the NMON blog for more information.
• MINIMUM
• MEDIAN
• PERCENTILE_95
• PERCENTILE_99
• MAX
• STD_DEV
• SUM
• COUNT
• GRANULARITY_MAX - this statistic calculates a peak average using the current granularity as the time scale. The value reported is the maximum peak average of all periods in the current interval.

Note that these values must be capitalized in the XML. The default value is AVERAGE.

For line charts, only AVERAGE, MAXIMUM, MINIMUM, COUNT and SUM are supported. The statistic value will be calculated for each granularity value. The default is AVERAGE, which calculates the average value of all data within the granularity interval.

Selecting Data

Hosts, types and fields can all be selected by name or by regular expression. For matching a specific name, use the name attribute. For regexes, use the regex attribute. Both should not be specified in a single element. Note that the name must be capitalized exactly how it is in the underlying data file; this usually means types are uppercase and fields are lowercase.

For the regex attribute, any valid regular expression that can be parsed by Java's Pattern class is valid. Note that when escaping values with backslash, only a single \ is needed.

For the name attribute, there are several special values that can be used:

• $ALL - matches all values. This is the default but can be useful in combination with renaming.
• $PROCESSES - matches all processes and returns only the aggregate process data if there are multiple processes with the same name.
• $TOP_PROCESSES_BY_CPU & $TOP_PROCESSES_BY_MEMORY - matches all processes and returns only the top 10 processes by CPU utilization or memory usage, respectively.
• $DISKS - matches all disks (e.g. /dev/sda but not /dev/sda1) for nmon DISK data types.
• $PARTITIONS - matches all partitions for nmon DISK data types.

Note for Perfmon data, the disks are identified by subtype (i.e. Logical Disk (C:)). So, the $DISKS and $PARTITIONS values are not valid for Windows data since they only work on base data types.

For <host> elements, an additional os attribute can also be provided to specify that data should only be matched for a specific operating system. Accepted values for this attribute are UNIX, LINUX, AIX, VIOS and PERFMON. These values are not case sensitive.

Renaming Data

Hosts, types and fields can all be renamed directly or with regular expressions. For explicit renaming, use the alias attribute. For regexes, either the existing regex attribute will be reused or an explicit aliasRegex attribute can be specified. Both an alias and a regex should not be specified in a single element.

For the aliasRegex attribute, any valid regular expression that can be parsed by Java's Pattern class is valid. Note that when escaping values with backslash, only a single \ is needed. By default, renaming with a regex uses the first matching group (i.e. parentheses). This matching group will be used as the new field name. The matching group can be changed by specifying the aliasByGroup attribute. Full string replacement is also possible by specifying the aliasByReplacement attribute. This attribute allows back references to matching groups, for example $1 to match the first group.

Often it is not possible to craft a regexAlias that can rename appropriately. However, a similar regex is still useful for matching data. To support this case, an optional <fieldAlias> is also supported. This element requires both a name and a value attribute. The name matches the field exactly (regexes are not supported) and the value is the renamed alias.

Note that it is almost always wrong to use a renaming attribute when also using a regex attribute to match. If the regular expression matches more than one item, all the items will be renamed. However, since names on a chart must be unique, only a single line or bar will appear on the chart.

When using the regex attribute, that same regular expression will be used to rename. If no renaming is required, either specify a <fieldAlias> or set the attribute aliasByGroup=0 to indicate that the entire string should be used (i.e. no renaming).

Examples

Graph CPU data in a stacked line chart for a single host. Graph with a percentage scale. Rename the fields to their full names without percent signs. <linechart name="CPU Utilization" shortName="CPU" stacked="true"> <yAxis label="% CPU" asPercent="true" /> <data> <type name="CPU_ALL" /> <field name="User%" alias="User" /> <field name="Sys%" alias="System" /> <field name="Wait%" alias="Wait" /> </data> </linechart>

Graph CPU data in a bar chart for all hosts. User, system and wait time bars (named by field) will be stacked. Each host will have a separate set of bars (categories). <barchart name="Average CPU Utilization" shortName="CPU" stacked="true" categoriesNamedBy="HOST" barsNamedBy="FIELD"> <yAxis label="% CPU" asPercent="true" /> <data> <type name="CPU_ALL" /> <field name="User%" alias="User" /> <field name="Sys%" alias="System" /> <field name="Wait%" alias="Wait" /> </data> </barchart>

Graph CPU data in an interval chart for all hosts. Create an average total CPU line for each host. <intervalchart name="Average CPU Utilization" shortName="CPU" linesNamedBy="HOST"> <yAxis label="% CPU" asPercent="true" /> <data> <type name="CPU_ALL" /> <field name="CPU%" /> </data> </intervalchart>
Note CPU% in this example is a generated value. It is not in the original NMON file but is generated as User% + Sys%.

XML files used by NMONVisualizer internally are in the com.ibm.nmon.report package.