SysOrb Network Monitoring System User's Guide: For version 4.6.0
Prev Chapter 4. Host/Node Management Next

4.4. Setting Up Checks

Before the SysOrb Server can tell you what problems, if any, there are on your network, you need to tell the SysOrb Server what to monitor. The results of all types of checks are stored by the SysOrb Server and are checked to see if the host in question is behaving as expected. If abnormal behavior is detected a warning or alert will be raised.

4.4.1. NetChecks

These checks can be performed without a SysOrb Agent being installed.

NetChecks on services are performed by using one of these 8 protocols:

When setting up a NetCheck you can set the following options:

In addition to these, some of the NetCheck types require extra parameters.

To enable a NetCheck on a node, do the following:

4.4.2. AgentChecks

In SysOrb the term AgentCheck covers a number of things on a host machine that can only be checked by the host itself by running the SysOrb Agent on it. Some of the checks you are able to monitor on hosts running SysOrb Agents are:

Depending on the operating system of the host running the SysOrb Agent, it may support many other types of AgentChecks. Please consult the SysOrb brochure for the full list.

Any AgentCheck belongs to one of three groups. One group results in a numerical value, such as free space, load, temperature etc., we call these checks continuous. The other group results in one of a number of cases, such as processes (present, absent), RAID (OK, degraded, failed) etc., we call these checks enumerations. The last group is the LogChecks, which return a line.

All AgentChecks share the common configuration options listed below.

In addition to these settings continuous AgentChecks have the following options. The appropriate units depending on the actual check, e.g. MB for free disk space, are listed to the right of these fields.

Note: When the value crosses any of these boundaries, a warning or alert may not be sent immediately. This depends on the Alert Strategy setting, and the score settings if the ScoreKeeper strategy is selected. Please refer to Section 1.3 for an explanation.

Enumeration AgentChecks have one option for each state the check may result in. That option determines if a warning or alert is to be be raise, should the check result in the given state. The possible settings for each state are Good, Warn or Alert.

When deciding on warning and alert tresholds be sure to choose meaningful values. For example, if you monitor a database server with approximately 10GB data on a 15GB device it would not make sense to raise an alert when there is less than 1GB free space. The other way around it would make sense to raise a warning, or possibly an alert, if there is more than 10GB free space as this might be caused by unexpected loss of data in the database.

As noted above all thresholds are disabled by default which means that when enabling a AgentCheck you need to consider not only what to monitor but also how you monitor it. Even if you create a check, with all the thresholds disabled, you will still be able to view the graphs for the device, so this is a good way to document how much a machine is stressed.

To enable an AgentCheck on a host, do the following:

4.4.2.1. LogChecks

The SysOrb Agent can scan log files for error messages, or for unexpected messages. Before you are able to configure any LogChecks, you will need to tell the Agent which files may be monitored. This is a safety measure, guarding agaist a compromised SysOrb Server being able to retrieve a copy of any file on the hosts running SysOrb Agents. (See the Administrator's Guide for more information on how to configure the SysOrb Agent to allow LogCheck).

If you see a group called LogChecks beneath AgentChecks, then you are ready to go on. Browse through the directories below the LogChecks group to find the file you want to monitor.

When scanning a log file, the Agent will start from the position where it stopped the last time. (End of file at that time.) That means that scanning the log file frequently (e.g. every 30 seconds) does not mean, that the entire file is read through twice every minute.

The Agent must know how to separate the entries of the log files. Many log files contain one entry per line, other separate the entries by a line with only dashes, or by an empty line. This can be configured for each log file in the web interface.

After splitting the log file into entries, the Agent will look at one entry at a time, and based on a set of rules determine whether that particular entry should be reported to the SysOrb Server, and if it is a warning or alert, or just an informational message.

How to split the entries, and the rules to apply to each of them afterwards is specified through the following configuration options specific for LogChecks:

  • Separator: A POSIX regular expression that describes the separator text between two adjacent log entries. The default '\r?\n' is to use each new line as a log entry.

    Please Note: Certain characters must be prepended with a backslash (i.e. \), if they are to be used to specify a separator. These are: . [ \ ] ( ) * ^ $ + ? { } |

    For more information about regular expressions, please see http://www.regular-expressions.info/tutorial.html

    Default value: \r?\n

  • Separator inclusion: This field can take one of three values, that determines what to do with the log entry separator. It is only useful if your separator pattern may match something non-trivial, that you want to include with the log entry. "Append to previous entry" will append the separator to the entry just before the separator. "Prepend to next entry" will prepend the separator to the next entry found. "Discard the separator" will just throw away the separator and only consider the text between separators as entries.

    Default value: Discard the separator

  • Rules: This text box is used to enter rules. The rules specifies which log entries that should be handled, and what to do with them. Each rule is terminated by a semicolon and consists of two parts separated by a colon.

    An example of a rule could be 'Error': alert log;. This rule will make sure that all lines containing the string Error (case sensitive) will result in an alert, and be logged in the database.

    The left part of the rule specifies which log entries should match this rule. The syntax is that of a POSIX regular expression enclosed in single quotes. It is also possible to have a number of regular expressions separated by and, or. Any regular expression can optionally be preceeded by not. Example: 'Error on' and 'Drive' and not 'empty', will match entries containing both Error on and Drive in no particular order, but not containing the the string empty anywhere.

    The right part of the rule specifies what to do with an entry matching the rule. It can be one of the states: ok, warm, or alert, followed by one of the actions: log or quiet, which tells whether the entry should be logged on the SysOrb Server in order for you to view later.

    Another example of a rule could be '[Ee]rror': alert log;. This rule will make sure that all lines containing either the string Error or error, will result in an alert, and be logged in the database.

    If multiple applications write into the same log (e.g. the Windows Event Log) you may want to split that log into multiple LogChecks, which will show warning and errors individually on overview pages and mail notifications. This can be achived with the LogCheck forwarding feature of SysOrb.

    Example of forwarding: '^[^:]*:Browser:': moveto 'Browser'; this will cause all entries having Browser between the first and second colon on the line to be moved into a LogCheck called Browser (which will be created if it does not exists). The moved entries will be processed through all of the rules of the LogCheck called Browser, and be logged over there if the rules says so. The entries will not be processed through the remaing rules of the originating LogCheck, in order for this to happen use copyto instead of moveto.

    You can insert comments in the list of rules by starting a line with '#'. SysOrb will ignore these lines when processing the log.

    Default value: Empty

When configuring a new LogCheck you can start by putting the catch-all rule '.*': ok log; as the very last (or maybe only) one. That will log everything to the SysOrb server. You will probably quickly find out a number of messages, that you do not want to see in the web interface. You should put the rules for those above the catch-all rule, that way any unexpected message will still turn up in the web interface, for you to decide if it should be ignored or not. After a while you could even modify the catch-all rule to result in warnings if any unexpected messages turn up.

From SysOrb 4.2 the log check rule system has been extended so that moveto and copyto actions now can be given alert groups as destinations.

Example rule set: 'test': moveto @'name_of_your_group';

The above rule collection will make sure that anyone in the "Intern" alert group get copied on events from the log that match the rule.

Note that the notification which is sent is not an alert. The notification send using this rule will bypass the alert system in SysOrb.

The above log check rule could be used in situations where it is needed to be notified of every single occurrence of a given event.

In order to make this rule work the Use for Logcheck redirection needs to be ticked on the user's AlertPath.

4.4.2.1.1. Event Log checks

A SysOrb agent running on Windows is able to check the Event Logs in addition to text log files. Event Logs are handled slightly different than ordinary log files

Checking of any Event Log is enabled per default on the SysOrb agent. If you consider it a security problem that SysOrb users will be able to configure a check on for instance the security Event Log on all your machines, then you should disable this feature on the Agent, before starting it. See the Administrator's Guide for more information on how to configure the SysOrb Agent with respect to which files will be permitted for LogChecks.

First off, the Event Log is already divided into separate entries, and thus SysOrb does not need a separator pattern to find out when one entry ends and the next begins.

The other difference is that an Event Log entry has more to it than a simple string of text, but the regular expressions in the rules are only able to handle strings. Therefore the SysOrb agent converts every Event Log entry into a string in the following way.

The Agent composes a colon-separated string of the following form: type:source:category:event_id:user:computer:description for instance: Information:SNMP:0:1001:N/A:MyComputer:The SNMP Service has started successfully.

This string is then tested against the patterns in the rules, and sent to the server if it matches.

A very simple rule could look like this:


'SNMP': warn log;

The pattern matches every string containing SNMP. And causes it to be logged with a yellow warning icon. The above example string contains SNMP (twice), so that would be logged.

This rule is a bit too simple, as it would also match unrelated Event Log entries, which happen to contain the word SNMP somewhere in the desription text. (That could just be part of a filename.) We really want the rule to match only entries, whose source is SNMP. That can be done, but first we will consider a simpler case.

In this rule


'^Information': ok quiet;

the caret causes the pattern to match only when the word Information appears at the very beginning of the string. This will match all information type messages, without catching errors or warnings, which happen to contain the word Information in the description text.

We can now proceed with the SNMP example:


'^Error:SNMP:': alert log;

This pattern will match all strings beginning with Error:SNMP:, i.e. error messages whose source is SNMP. (As source is the second field in the colon delimited string.)

To catch both errors and warnings related to SNMP one can write:


'^(Warning|Error):SNMP:': alert log;

The pipe and parenteses mean that either Warning or Error must preceed the first colon in order for this pattern to match.

If you want to match every record from the SNMP source, regardless of the event type, you can use the regular expression [^:]*, which matches any number of characters as long as there are no colons among them. This can be used in a rule like this:


'^[^:]*:SNMP:': alert log;

The wildcard can be used more than once in a singe rule, for instance if we want to catch event number 6009 from the SNMP source. SysOrb places the event number as the fourth field in the colon delimited string, so we do not care about the first and third field, only the secon and fourth. This can be achived with the following pattern:


'^[^:]*:SNMP:[^:]*:6009:': alert log;

4.4.3. snmpChecks

If a node supports the SNMP protocol it is possible to monitor the node using that. SNMP can provide much more information than NetChecks, but does not require installation of extra software like AgentChecks. This is useful for monitoring SNMP aware printers/switches/routers/firewalls, but can also be used to monitor a ordinary computers.

To enable an snmpCheck on a node, do the following:

snmpChecks are like AgentChecks grouped into continuous and enumeration. The configuration parameters are equal to those of the AgentChecks, please refer to Section 4.4.2 for a thorough description.

4.4.4. ESXi checks

SysOrb provides you with a "single pane of glass" monitoring of your VMware infrastructure.

SysOrb supports monitoring of health and performance status of VMware ESXi hosts via an easy to use web interface. SysOrb enables the users to define threshold values in order for SysOrb to trigger alert notifications upon violation of those thresholds. SysOrb uses vSphere Hypervisor APIs to collect the critical metrics for the host as well as the VMs.

This is useful in situations where a server with several virtual machines needs to be monitored. . It is important to understand that all checks for the server and its Virtual Machines will be located on the same node, this means that it is still recommended to install agents on virtual machines if a detailed and complete picture of the individual VM's is needed.

4.4.4.1. SysOrb agents for deep monitoring of the OS and applications running on the virtual machine

If you need deep monitoring of the OS and applications running on each single virtual instance you simply install the SysOrb agent. The SysOrb agent is very light weight software which support windows, Linux, and Novell platform.

To set up a node with ESXi first this is to create a new node that should contain the checks. You should have set up VMware to allow monitoring.

  • Make sure you navigate the to domain where you want your node to appear

  • Click on configure in the left pane

  • Click on "add node" to add a new node.

  • Configure the node by writing the address to the server, removing the agent check, and adding the ESXi settings.

4.4.5. Cumulative checks

Some SysOrb checks measures events per second or bytes per second, when what you really want to know is the total number of events/bytes in one day, week or month. If this is the case, you can use cumulative checks.

If you want to monitor the monthly traffic on a switch or router port for instance, you should first configure ordinary checks on ifInOctets and ifOutOctets on the port in question. You do not have to configure any alert limits on the check itself.

You should then see an option called [Add acc.] to the right of the newly enabled check. Click that link, and you will be presented with a configuration dialog with options for the accumulation, that you are about to create.

The rest of the options works as for ordinary checks. Only that Warn when below and Alert when below does not make sense for calendar accumulations, as they would always trigger when the sum is reset at the beginning of every month (or whatever period is selected.)

Prev Home Next
Deleting a Node Up Dependencies