Single Node Upgrade from 4.50 to 4.51¶
This guide will lead you through the steps specific for upgrading a NetEye Single Node installation from version 4.50 to 4.51.
Upgrading a NetEye Single Node takes a nontrivial amount of time. Granted the environment connectivity is seamless, the upgrade procedure may take up to 30 minutes.
Warning
Remember that you must upgrade sequentially without skipping versions, therefore an upgrade to 4.51 is possible only from 4.50; for example, if you have version 4.27, you must first upgrade to the 4.28, then 4.29, and so on.
Breaking Changes¶
NetEye Kubernetes Operator¶
This release will introduce a brand new NetEye Operator which is the component responsible for managing the NetEye
deployment in Kubernetes. All configuration and management is made by the standard NetEye CLI procedures such as
neteye install and neteye upgrade, but you can configure and override the default configuration by
editing the /etc/neteye-environment.yaml file. NetEye CLI procedures will automatically source this configuration
file and apply desired changes to the Operator. This upgrade does not require users to update the configuration
beforehand since default values will be applied. For the full list of configurable parameters, please refer to the
/usr/share/neteye/setup/neteye-environment.yaml.tpl file. To apply the configuration changes during normal
operation, you can use the neteye install --restrict-services-to neteye-operator command. This command will
apply the configuration changes to the Operator without affecting the other services.
NetEye CA Trust Bundle for Kubernetes Workloads¶
To ensure that all Kubernetes workloads trust the various Certificate Authorities (CAs) used by external services, for
example, when connecting to LDAP servers with Keycloak, users are now required to configure the NetEye CA Trust.
To do so, a new configuration directory has been introduced at /neteye/local/rke2/conf/ca-trust/.
This is a breaking change: the workloads running on Kubernetes will no longer trust the CAs installed on the node (for example with update-ca-trust), but only the public CAs shipped with their image, the NetEye root CA and the CAs in this directory. If, before the upgrade, Keycloak relied on a CA added to the node to reach an LDAPS server or an external identity provider, that CA must be added to the directory, otherwise the connection will fail after the upgrade.
The directory is created by the latest updates of NetEye 4.50 and pre-populated with the NetEye root CA and the CAs trusted
by the node, that is, the .crt and .pem files in /etc/pki/ca-trust/source/anchors/ and
/usr/share/pki/ca-trust-source/anchors/, except for the NetEye root CA and the Icinga 2 master CA, which are
already handled by NetEye. Before upgrading:
Identify the private CAs of the services that Keycloak and the other workloads (OpenTelemetry Collector) connect to, including the CA that issued the certificate of the NetEye frontend domain, if it is not the NetEye root CA.
Review
/neteye/local/rke2/conf/ca-trust/on every node of the installation: remove the pre-populated CAs that the workloads do not need and add the missing ones, in PEM format with the.crtor.pemextension. All the nodes must contain the same files. The directory and the files must be owned byrootand not writable by their group or by others, and every.crtand.pemfile must contain only valid certificates in PEM format (the files copied from the node may also be in other formats, such as DER). Otherwise the upgrade prerequisite checks will fail on that node and explain how to fix each file, or advise removing it if the workloads do not need that CA.Confirm it in Icinga Web 2: in the configuration of the neteye module, check I am ready to upgrade and save, or run icingacli neteye catrustready enable.
The upgrade prerequisite checks will fail until the confirmation is given. The check is skipped on NetEye Satellites, as they do not run Kubernetes workloads. The directory will be merged into the bundle used by the workloads during the upgrade.
The directory will remain after the upgrade in case you need to trust new CAs. For the content, the pre-population, the format of the files and the commands to apply a change, see Trusted CAs for Kubernetes Workloads.
Prerequisites¶
Before starting the upgrade, carefully read the latest release notes on NetEye’s blog and check the features that will change or be deprecated.
All NetEye packages installed on a currently running version must be updated according to the update procedure prior to running the upgrade.
NetEye must be up and running in a healthy state.
Disk Space required:
3GB for
/and/var150MB for
/boot
If the NetEye Elastic Stack module is installed:
The rubygems.org domain should be reachable by the NetEye Master only during the update/upgrade procedure. This domain is needed to update additional Logstash plugins and thus is required only if you manually installed any Logstash plugin that is not present by default.
There is a number of configuration items that should not be modified in order to avoid issues during the update/upgrade of your instance. Please check out Protected Configuration Items for details.
NetEye services are progressively moving to Kubernetes to improve scalability, security, resource management and the speed of updates.
Access to the following domains is strictly required for upgrading to NetEye 4.51: Please ensure that all NetEye nodes can reach these domains over HTTPS (TCP port 443):
Domain
Port
Intended Use
ghcr.io
443 TCP
GitHub container images
api.github.com
443 TCP
GitHub container images
pkg-containers.githubusercontent.com
443 TCP
GitHub container images
quay.io
443 TCP
Quay container images
cdn01.quay.io
443 TCP
Quay container images
docker.io
443 TCP
Docker Hub container images
hub.docker.com
443 TCP
Docker Hub container images
auth.docker.io
443 TCP
Docker Hub container images
index.docker.io
443 TCP
Docker Hub container images
registry-1.docker.io
443 TCP
Docker Hub container images
production.cloudfront.docker.com
443 TCP
Docker Hub container images
*.cloudflarestorage.com
443 TCP
Container images
rpm.rancher.io
443 TCP
Rancher packages
docker.elastic.co
443 TCP
Elastic container images (only with the Elastic Stack)
docker-auth.elastic.co
443 TCP
Elastic container images (only with the Elastic Stack)
1. Run the Upgrade¶
To perform the upgrade, run from the command line the following command:
neteye# (nohup neteye upgrade &) && tail --retry -f nohup.out
After the command was executed, the output will inform if the upgrade was successful or not:
In case of successful upgrade you might need to restart NetEye to properly apply the upgrades. If the reboot is not needed, please skip the next step.
In case the command fails refer to the troubleshooting section.
2. Reboot¶
Restart NetEye to apply the upgrades correctly.
neteye# neteye node reboot
3. Additional Tasks¶
Keycloak in Kubernetes and the NetEye Operator¶
After a successful upgrade, you will be required to perform and verify the migration of the Keycloak systemd/pcs instance to the new Keycloak deployment in Kubernetes, which will be an upgrade prerequisite check for future releases. To automate this process, the product ships 3 utility tools and scripts that allow you to perform the validation and migration.
neteye config auth switch-to-kube: This command will switch the authentication configuration from the systemd/pcs instance to the new Keycloak deployment in Kubernetes by updating the necessary configuration files. After successful execution, the systemd/pcs instance will be disabled, and you will be required to test the new environment to ensure that authentication is working as expected.
neteye config auth rollback-from-kube: Should any issues arise with the new Keycloak deployment in Kubernetes, this command will allow you to roll back the authentication configuration to the previous systemd/pcs instance by restoring the necessary configuration files.
neteye config auth cleanup-keycloak-pcs: Finally, once you are satisfied with the new Keycloak deployment, this command will clean up the old systemd/pcs instance by removing any remaining configuration files and data related to the previous Keycloak deployment. Backups of the old configuration files will be created in the
/root/keycloak_upgrade_<timestamp>.bckdirectory. This step is non-reversible, and you will not be able to roll back to the previous systemd/pcs instance after executing this command.