Security

Trusted Certificate Generation with Windows

The instructions below will help you create a trusted certificate chain in Windows and configure HTTPS in NetEye 4 with the trusted certificate. There are also instructions for how to do this from within the NetEye server.

Requirements

The following requirements should be met before proceeding with configuration of the certificate:

• A Windows Certification Authority should already be up and running, with a suitable Certificate Template.

• The Certificate Template should meet recent encryption standards, for example:

  • RSA, SHA256, 4096bit

  • The private key should be marked as exportable

• You have a Windows domain-joined Server/PC which is allowed to request certificates.

• You have a Linux machine running NetEye where you can install the CA Chain Certificate. This is necessary for the server certificate to be trusted by the Apache web server.

Procedure

Step 1: Request a new certificate from a Windows domain-joined Server/PC:

• Open the Microsoft Management Console (Start ‣ mmc.exe)

• Within MMC, go to File ‣ Add/Remove Snap-in

• In the popup dialog, navigate to Certificates ‣ Add ‣ Computer Account ‣ Next ‣ Local computer ‣ Finish, and then OK to close the dialog.

• Expand Certificates ‣ Personal, then right click on Certificates

• Select All Tasks, then Request new certificate (you may need to skip a “Before You Begin” screen first) and Next when “Active Directory Enrollment Policy” is selected as shown here:

• Select a Certificate Template and click on the link “More information is required to enroll for this certificate…”

• Fill-in the information for each tab in the Certificate Properties dialog as shown (the fields shown are mandatory; you can optionally add values like Country, Department, Organization, etc.):

• Click OK and then Enroll and Finish.

Step 2: Export the certificate with its private key in PFX format:

• Right click on the certificate you just created in the center panel, then click on All Tasks ‣ Export.

• Select Yes, export the private key, click Next, and select PKCS #12:

• Provide a password to protect the private key that goes with the certificate (a strong password is advised):

• Then designate the path where the certificate should be stored:

• Now click Next, and then Finish. You should see the message “The export was successful.”

Step 3: Export your CA Certificate(s) in Base64 format.

Note

If the Certification Authority Infrastructure consists of multiple CAs (for example, Root CA > Subordinate Intermediate CA), you must export all of them and then combine them into a single Certificate.

• Double click on your new certificate in the center panel. In the popup dialog, click on the Certification Path tab, which should display a Certificate Chain such as the one shown here:

• Select for instance the Intermediate Certificate, and then click on the View Certificate button. Then click on the Details > Copy to File:

• Instead of the “DER encoded binary” option, select “Base-64 encoded”:

• In the next dialog, choose a path and filename to save the .CER file, then click Finish.

• Repeat the procedure above for the Root CA instead of the intermediate certificate.

• To create the certificate chain, open all of your saved CA certificates in a Text Editor and combine them into a single file, both respecting the proper order (Root/Parent before Subordinate/Child) and paying attention to not leave any blank lines between certificates as shown here:

Step 4: Copy the CA Chain certificate to the Linux server (NetEye 4) and adjust the Apache configuration:

• Make a copy of the CA Chain certificate, rename and move both of them to the proper folder according to the settings in /etc/httpd/conf.d/neteye-ssl.conf file:

  SSLCertificateChainFile /neteye/shared/httpd/conf/tls/certs/neteye_chain.crt
  SSLCACertificateFile /neteye/shared/httpd/conf/tls/certs/neteye_ca_bundle.crt
  

Step 5: Copy your PFX Server Certificate to the Linux server (NetEye 4), convert it, and adjust the Apache configuration:

• Put the PFX certificate in a temporary directory, for example /tmp.

• Extract the public part of the certificate. You will be asked for the Key Password, which is the one you entered when you exported your PFX from the Windows Machine:

  # openssl pkcs12 -in {yourfile.pfx} -nokeys -out {certificate.crt}
  

• Extract the encrypted private key part of the certificate. You will be asked for the Key Password, which is the one you entered when you exported your PFX file from the Windows Machine. You will also be asked to enter a new Password for the newly generated private key (you can use the same password):

  #  openssl pkcs12 -in {yourfile.pfx} -nocerts -out {keyfile-encrypted.key}
  

• Now decrypt your private key:

  # openssl rsa -in {keyfile-encrypted.key} -out {keyfile-decrypted.key}
  

• Rename your certificate and key and move them in the proper folder according to the settings in /etc/httpd/conf.d/neteye-ssl.conf file:

  SSLCertificateFile /neteye/shared/httpd/conf/tls/certs/neteye_cert.crt
  SSLCertificateKeyFile /neteye/shared/httpd/conf/tls/private/neteye.key
  

Note

Both the certificate and key must be owned by root and only root must have full read and write access to the files. Also, certificate and key are located in shared directory. Therefore, in a cluster environment, they should be changed only on the node running the httpd resource.

• Finally, restart Apache.

HTTPS Configuration

Beginning with version 4.2, NetEye has been configured to use HTTPS throughout, using a self-signed certificate based on Apache’s mod_ssl. This certificate is generated automatically during the NetEye install process.

However, it is recommended to create and install as soon as possible a new trusted certificate consisting of the self-signed certificate chained with a valid, external CA certificate.

The importance of using trusted certificates, is clear also from this example use case: if you use the Director’s Self-Service API to first connect to an external Icinga2 agent, the Kickstart initialization script may fail if it determines it cannot trust the self-signed certificate alone. While this restriction can be bypassed as an emergency measure, this is a highly insecure practice and is strongly discouraged.

The following steps will help you configure your NetEye installation to create and deploy a more secure certificate that can be trusted externally and/or in your domain. If you would prefer to create a certificate chain from within Windows, step by step instructions are available in the dedicated section.

Step 1: Obtain Your Signed Certificate from a Certificate Authority

The instructions below assume you already have a valid certificate from an external Certificate Authority (CA). Then for each server/machine you will need to:

1. Create a private key

2. Generate a certificate signing request (CSR)

3. Send or upload the private key and CSR to the CA

4. Retrieve the certificate signed by the CA

When NetEye is first installed, it configures the initial self-signed certificates in the following directories:

File	Directory
neteye_cert.crt	/neteye/shared/httpd/conf/tls/certs/
neteye.key	/neteye/shared/httpd/conf/tls/private/
neteye_chain.crt	/neteye/shared/httpd/conf/tls/certs/
neteye_ca_bundle.crt	/neteye/shared/httpd/conf/tls/certs/
neteye-ssl.conf	/etc/httpd/conf.d/

Note

Certificate and key are located in shared directory. Therefore, in a cluster environment, they should be changed only on the node running the httpd resource.

Because the private key is so fundamentally important to your network’s security, you should strongly consider creating a new one. You can create the private key and the CSR file in the appropriate directory with a single command, after moving to the correct directory:

# cd /neteye/shared/httpd/conf/tls/private/
# openssl req -newkey rsa:4096 -nodes -keyout hostname.fqdn.key -out hostname.fqdn.csr
Generating a 4096 bit RSA private key
..................................................................................................++
..................................................++
writing new private key to 'hostname.fqdn.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:IT
State or Province Name (full name) []:BZ
Locality Name (eg, city) [Default City]:Bolzano
Organization Name (eg, company) [Default Company Ltd]:Company
Organizational Unit Name (eg, section) []:Monitoring
Common Name (eg, your name or your server's hostname) []:hostname.fqdn
Email Address []:mail@company.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

The hostname.fqdn.key file is your private key which should be kept secure and not given to anyone. The hostname.fqdn.csr file is what you should send to the Certificate Authority when requesting your SSL certificate (you may need to paste its contents into the web form of the CA).

Note

If you have a large number of systems to monitor, it makes sense to automate this process. For instance, you can keep multiple keys and CSRs manageable by using the host’s FQDN as part of the filename for both the private key and the CSR. And rather than manually answer the CSR questions one by one, you can create an external configuration file (usually called openssl.cnf) that is invoked with the -extfile parameter.

Relevant links:

• Description of OpenSSL/x509 and parameters :manpage:`openssl-x509

• Example external configuration file

Step 2: How to Create the Trusted Certificate

The certificate that the CA returns to you (let’s call it countersigned.crt) will be the (self-signed) certificate you sent them, countersigned with the CA’s key. You can then use this new trusted certificate in applications (e.g., browsers or the Icinga2 agent) that in turn trust the CA you used.

To be used with the Icinga2 agent, this certificate should be in PEM format. To check, you can look at the certificate file:

# cat countersigned.crt
-----BEGIN CERTIFICATE-----
MIID3jCCAsagAwIBAgICPnowDQYJKoZIhvcNAQELBQAwgaMxCzAJBgNVBAYTAi0t
...
-----END CERTIFICATE-----

If you do not see BEGIN CERTIFICATE, you will need to export the certificate to PEM format (you can use other tools besides openssl as long as they generate a certificate in PEM format):

# openssl x509 -in countersigned.crt -outform PEM -out countersigned.pem

Step 3: Install the Certificates on the Web Server

You must then rename your certificates and key and move them in the proper folder according to the settings in file /etc/httpd/conf.d/neteye-ssl.conf:

<VirtualHost _default_:443>
SSLEngine on
SSLCertificateFile /neteye/shared/httpd/conf/tls/certs/neteye_cert.crt
SSLCertificateKeyFile /neteye/shared/httpd/conf/tls/private/neteye.key
SSLCertificateChainFile /neteye/shared/httpd/conf/tls/certs/neteye_chain.crt
SSLCACertificateFile /neteye/shared/httpd/conf/tls/certs/neteye_ca_bundle.crt
</VirtualHost>

• SSLCertificateFile: Your trusted certificate countersigned by the CA, from countersigned.crt (or countersigned.pem if you exported it in .pem format) to neteye_cert.crt

• SSLCertificateKeyFile: Your private key, renamed from hostname.fqdn.key to neteye.key

• SSLCertificateChainFile: Certificate chain of the server certificate. If you don’t have a chain you can copy the file neteye_cert.crt naming it neteye_chain.crt

• SSLCACertificateFile: The CA’s public certificate named neteye_ca_bundle.crt

Note

Do not change setting in file /etc/httpd/conf.d/neteye-ssl.conf because they will be overwritten during update procedure possibly causing NetEye outages.

Step 4: Restart Apache

Finally, restart the HTTPD service so it reloads the configuration files above with the new trusted certificates. If you are on a single-node instance use:

# systemctl restart httpd.service

If you are on a cluster use:

# pcs resource restart httpd

Known Risks and Foreseeable Misuse

The following deployment or operating conditions are known to create significant cybersecurity risks. They are outside the intended use of NetEye.

• Exposing the web interface or API to the public internet without a reverse proxy, network restriction or VPN

• Continuing to operate a release that is no longer supported – such releases no longer receive security fixes; see our Bugfix Policy

• Keeping default or shared credentials for administrative accounts, or reusing service account credentials across environments

• Disabling authentication, TLS or the audit log to simplify integration

• Running the underlying operating system without vendor security updates, or on a release that has reached end of life

• Granting broad module permissions to users who only require read access, particularly for modules that can execute commands on monitored hosts

• Storing monitoring data with confidentiality requirements without configuring encryption at rest on the underlying storage

If any of these conditions cannot be avoided, compensating controls should be documented in the operator’s own risk assessment.

How Configuration Changes Affect Data Security

Several configuration options change how NetEye handles data. Before applying them, consider the effect on confidentiality, integrity and availability.

Change	Effect on data security
Disabling TLS on an interface	Monitoring data and credentials are transmitted in clear text
Adding a satellite	Monitoring data is transmitted to and buffered on an additional node that must be secured to the same standard
Federating authentication to an external IdP	Account lifecycle and password policy move to the IdP; access to NetEye depends on its availability and configuration
Widening module permissions	Users may gain access to monitoring data or command execution beyond their role
Changing retention settings	Longer retention increases the volume of data exposed by a compromise; shorter retention may conflict with audit requirements
Disabling the audit log	Administrative actions are no longer traceable
Restoring a backup	The configuration state, including accounts and permissions, is reverted to the time of the backup

Trusted CAs for Kubernetes Workloads

The NetEye components that run on Kubernetes, such as Keycloak and the OpenTelemetry Collector, do not use the Certificate Authorities (CAs) trusted by the node they run on, for example those added with update-ca-trust. They trust only:

• The well-known public CAs shipped with their container image

• The NetEye root CA

• The CAs you add to the /neteye/local/rke2/conf/ca-trust/ directory, described below

If a workload connects to a service whose certificate is issued by a private CA, you must add that CA to the directory. A typical example is when Keycloak connects to an LDAPS server (see Configure federated LDAP/AD) or to an external identity provider. Without a valid CA, TLS handshakes will fail.

The workloads also connect to the NetEye frontend domain. If its certificate is not issued by the NetEye root CA, for example because you configured a custom certificate for the web interface, the CA that issued it must be in the directory as well.

Note

Every certificate in the directory is trusted by all the workloads above. Add only the CAs that are really needed, and never a CA you do not trust.

The ca-trust Directory

The directory is created by NetEye and contains:

• 00-neteye-root-ca.crt, the NetEye root CA. It is managed by NetEye and overwritten at every neteye install, neteye update and neteye upgrade: do not edit or remove it.

• Your own CAs. Add them as files with the .crt or .pem extension, in PEM format. A file may contain more than one certificate.

When NetEye adds 00-neteye-root-ca.crt for the first time, it also pre-populates the directory with the CAs trusted by the node, so that the workloads keep trusting the CAs you already added to the node. It copies the .crt and .pem files of /etc/pki/ca-trust/source/anchors/ and /usr/share/pki/ca-trust-source/anchors/, with the following exceptions:

• root-ca.crt, the NetEye root CA, already present as 00-neteye-root-ca.crt

• neteye-icinga2-master-ca.crt, the Icinga 2 master CA, which the workloads do not need

• Files with the same content as one already copied. If both directories contain a file with the same name or the same content, the one in /etc/pki/ca-trust/source/anchors/ is copied.

Pre-population happens only once: the CAs you add to the node afterwards are not copied, and the files you remove from the directory are not copied again. Review the copied files and remove those that the workloads do not need.

The trust store of the node also accepts certificates in other formats, such as DER, which the workloads cannot use. Such files are copied as they are, and the upgrade prerequisite checks report them. Convert each of them to PEM, for example:

neteye# cd /neteye/local/rke2/conf/ca-trust/
neteye# openssl x509 -inform DER -in my-ca.crt -out my-ca.pem && rm my-ca.crt

or remove it if the workloads do not need that CA.

Subdirectories, hidden files and files with any other extension are ignored. Files that you add are never modified or removed by NetEye.

Since anyone able to change this directory or its files could make the workloads trust a CA of their choice, the directory and every .crt and .pem file in it (a symbolic link is followed) must be owned by root and must not be writable by their group or by others. Otherwise neteye install, neteye update and neteye upgrade stop with an error naming the path.

To add a CA, for example the one for an LDAPS server, copy its certificate into the directory:

neteye# cp ldap-ca.crt /neteye/local/rke2/conf/ca-trust/

Note

On a cluster, the directory must contain the same files on all the nodes, so copy the files to every node. neteye install, neteye update and neteye upgrade stop with an error that lists the checksum of each node if the directory differs.

Confirm the Directory Before Upgrading to NetEye 4.51

The directory is created, and pre-populated, by the latest updates of NetEye 4.50. Before upgrading to NetEye 4.51:

1. On every node, review the content of /neteye/local/rke2/conf/ca-trust/: remove the CAs that the workloads do not need and add the missing ones, for example the CA of an LDAPS server that is not trusted by the node, or the CA that issued the certificate of the NetEye frontend domain if it is not the NetEye root CA. Make sure that all the nodes contain the same files.

2. Confirm that the directory is ready: in Icinga Web 2, open the configuration of the neteye module, check I am ready to upgrade and save. Alternatively, run:

   neteye# icingacli neteye catrustready enable
   

The upgrade prerequisite checks fail until the confirmation is given. They also fail, on each node, if the directory or its files can be changed by a user other than root, and if a .crt or .pem file is empty, does not contain any certificate in PEM format, contains a malformed certificate or contains a private key. For each problem, they name the file and tell how to fix it. The checks are skipped on NetEye Satellites, as they do not run Kubernetes workloads.

Apply the Changes

The content of the directory is merged into a single bundle, stored in the neteye-ca-bundle ConfigMap of the neteye-tenant-shared Kubernetes namespace. The NetEye Operator mounts it in the workloads. This happens automatically during neteye install, neteye update and neteye upgrade. To apply a change at any other time, run:

neteye# neteye install --restrict-services-to neteye-operator

When the bundle changes, Keycloak and the OpenTelemetry Collector are restarted to load it. Nothing is restarted if the content does not change.

While merging the CAs, NetEye re-encodes every certificate, removes duplicates and comments, and refuses to continue, naming the guilty file, if:

• A file does not contain any PEM certificate, or contains a malformed one

• A file contains a private key (never put private keys in this directory)

• The resulting bundle is larger than 900 KB

To check which CAs were merged, count the certificates in the bundle (the NetEye root CA and one for each certificate you added, minus duplicates):

neteye# kubectl get configmap neteye-ca-bundle -n neteye-tenant-shared \
    -o jsonpath='{.data.ca-bundle\.pem}' | grep -c 'BEGIN CERTIFICATE'