Coming from elabit/robotmk#60
Currently, RobotMK monitors
- the overall status of a suite
- runtimes of suites/test/keywords
Performance graphs are created only for runtimes of suites/test/keywords.
The idea is to create a RF library with special keywords which allow to "export" suite-internal data into the XML log.
The RobotMK check in turn will know how these monitoring keywords are being parsed.
The msg field in the XML node could be the container for the collected values.
Application scenarios for robotframework-robotmk:
Performance graphs
Checkmk Perfdata items_failed ${FAILED_ITEMS} [${CRIT_ITEMS}]
In addition to the runtime perf values, there will be also a metric items_failed with value of $FAILED_ITEMS. If there is a Robot internal logic which lets the test fail at $CRIT_ITEMS, draw this as a line within the graph.
Checkmk Metric
Checkmk Metric items_failed ${FAILED_ITEMS} [type=number] [perfdata=yes]
Checkmk Metric coffeemachine_status ${MACHINE_STATUS} [type=string]
Usually, a RF test determines the final status of a test, which is transferred as raw XML to the Checkmk server.
The overall status, only then gets changed by RobotMK if there are runtime exceedances (configured in Checkmk/WATO).
If the Monitoring Admin wants to change a RF internal threshold, he either has to change the RF test, or modify the threshold variable passed to the test. In addition, RF tests only distinguish between PASS (=OK) and FAIL (=CRIT). There is no state between for WARN.
In order to make setting thresholds as much as Checkmk-"alike", the WATO page for a RobotMK check should, besides "runtime monitoring", also contain a section "metric monitoring". In this mode, the check will explicitely search for metric monitoring keywords.
=> Values of type=number can be compared with numeric warn/crit thresholds.
=> Values of type=string can be compared with regex patterns for warn/crit states.
In both cases, custom error messages can be set for warn/crit.
Note: In consequence, this will not have any effect on the log.html of RF. It is nothing more than a re-interpretation of the final RF result for monitoring purposes.
Coming from elabit/robotmk#60
Currently, RobotMK monitors
Performance graphs are created only for runtimes of suites/test/keywords.
The idea is to create a RF library with special keywords which allow to "export" suite-internal data into the XML log.
The RobotMK check in turn will know how these monitoring keywords are being parsed.
The msg field in the XML node could be the container for the collected values.
Application scenarios for robotframework-robotmk:
Performance graphs
Checkmk Perfdata items_failed ${FAILED_ITEMS} [${CRIT_ITEMS}]In addition to the runtime perf values, there will be also a metric
items_failedwith value of$FAILED_ITEMS. If there is a Robot internal logic which lets the test fail at$CRIT_ITEMS, draw this as a line within the graph.Checkmk Metric
Checkmk Metric items_failed ${FAILED_ITEMS} [type=number] [perfdata=yes]Checkmk Metric coffeemachine_status ${MACHINE_STATUS} [type=string]Usually, a RF test determines the final status of a test, which is transferred as raw XML to the Checkmk server.
The overall status, only then gets changed by RobotMK if there are runtime exceedances (configured in Checkmk/WATO).
If the Monitoring Admin wants to change a RF internal threshold, he either has to change the RF test, or modify the threshold variable passed to the test. In addition, RF tests only distinguish between
PASS(=OK) andFAIL(=CRIT). There is no state between for WARN.In order to make setting thresholds as much as Checkmk-"alike", the WATO page for a RobotMK check should, besides "runtime monitoring", also contain a section "metric monitoring". In this mode, the check will explicitely search for metric monitoring keywords.
=> Values of
type=numbercan be compared with numeric warn/crit thresholds.=> Values of
type=stringcan be compared with regex patterns for warn/crit states.In both cases, custom error messages can be set for warn/crit.
Note: In consequence, this will not have any effect on the log.html of RF. It is nothing more than a re-interpretation of the final RF result for monitoring purposes.