Set up distributed data collectors

A data collector executes checks on a remote Linux server, for example at another site. The web interface, central storage and administration remain on the monitoring server. The data collector establishes the connection to the central server; a separate web port on the data collector is not required.

Step 1: Create the data collector on the central server

  1. Open Settings → Distributed collectors.

  2. Enter a unique, descriptive Name, for example Standort Berlin.

  3. Leave Active enabled and select Save collector.

  4. Copy the displayed Collector ID and Access key. The key is only displayed now.

Step 2: Install the remote Linux system

Copy the complete program package to the remote Linux server. The data collector installer requires systemd, Python 3.9 or newer with SQLite/SSL, and sha256sum. Change to the extracted QisutuMonitoring folder and run:

sudo sh install-collector.sh

Enter the central server’s HTTPS address, the data collector ID, access key and, if necessary, the confirmed SHA-256 certificate fingerprint. For a central server on the default port, the address includes :8787, for example https://monitoring.firma.intern:8787. Use an independently verified fingerprint when confirming a self-signed certificate this way.

The program is installed under /opt/netzmonitor-collector. Private configuration and the local measurement buffer are under /var/lib/netzmonitor-collector; the Linux service account is named netzmonitor-collector. Separately install the programs required for the checks actually assigned, such as OpenSSH, Net-SNMP and database clients.

Step 3: Assign devices

  1. Open a device under Devices → Configure.

  2. In the Device section, select the desired name under Collector.

  3. Apply the assignment using Save all.

  4. Under Settings → Distributed collectors, check the connection and then the device’s readings.

Central server executes checks on the central monitoring server. Assignment is per device. Under Distributed collectors, Connected appears if communication was recorded within the last 90 seconds; otherwise, No current connection appears. Paused means the data collector has been disabled.

Check the service on the remote server using:

sudo systemctl status netzmonitor-collector
sudo journalctl -u netzmonitor-collector -n 50

Change the access key and pause the data collector

Click its name in the Distributed collectors section. Enable Generate new access key and save. Then store the new key, which is only displayed once, in /var/lib/netzmonitor-collector/collector.json on the remote server and restart the service:

sudo systemctl restart netzmonitor-collector

The configuration contains credentials. The key authorizes retrieval of the check configuration, including the credentials required for the assigned devices. Therefore restrict access to the configuration, buffer and backups.

If you switch Active off and save, the data collector can no longer exchange configuration and results with the central server. A data collector already operating offline only learns of the change at the next successful communication or after its configuration validity expires. The central server does not automatically take over its checks. To move them back, select Central server for the affected devices and save.

Behavior during connection interruptions

The data collector synchronizes its configuration every 30 seconds. Without a connection, it continues using the last received configuration for at most 24 hours. Measurements are stored locally in SQLite and later transmitted with their original measurement time. By default, the buffer holds 100.000 measurements; no further checks start when it reaches capacity.

The default includes eight local workers. workers and max_buffer_records can be changed in collector.json; the data collector service must then be restarted. The clocks of the central server and data collector may differ by at most 60 seconds.

Retransmitted measurements that have already been acknowledged are not stored twice. Results from an obsolete device configuration are discarded. Delayed old values do not replace a newer current state. Severely delayed measurements do not generate retrospective problem notifications. Results older than 30 days are removed from the local buffer and recorded in the log.

Calculated metrics are always evaluated centrally. NetFlow/IPFIX also continues to be received centrally. A device with active flow reception therefore cannot be assigned to a remote data collector. If necessary, use a separate, centrally monitored device entry for this purpose. This version does not include automatic high availability of the central server or automatic takeover of a failed data collector.