User Guide Functional Overview Requirements Architecture System Installation NetEye Additional Components Installation Setup The neteye Command Director NetEye Self Monitoring Tornado Business Service Monitoring IT Operation Analytics - Telemetry Geo Maps NagVis Audit Log Shutdown Manager Reporting ntopng Visual Monitoring with Alyvix Elastic Stack IT Operations (Command Orchestrator) Asset Management Service Level Management Cyber Threat Intelligence - SATAYO NetEye Update & Upgrade How To NetEye Extension Packs Troubleshooting Security Policy Glossary
module icon Elastic Stack
Overview Architecture Authorization Kibana Elasticsearch Cluster Elasticsearch Configuration Replicas on a Single Node Elasticsearch Performance tuning Overview Enabling El Proxy Sending custom logs to El Proxy Configuration files Commands Elasticsearch Templates and Retentions El Proxy DLQ Blockchain Verification Handling Blockchain Corruptions El Proxy Metrics El Proxy Security El Proxy REST Endpoints Agents Logstash Elastic APM Elastic RUM Elastic XDR Log Manager - Deprecated
ntopng Visual Monitoring with Alyvix Elastic Stack IT Operations (Command Orchestrator) Asset Management Service Level Management Cyber Threat Intelligence - SATAYO Introduction to NetEye Monitoring Business Service Monitoring IT Operation Analytics Visualization Network Visibility Log Management & Security Orchestrated Datacenter Shutdown Application Performance Monitoring User Experience Service Management Service Level Management & Reporting Requirements for a Node Cluster Requirements and Best Practices NetEye Satellite Requirements TCP and UDP Ports Requirements Additional Software Installation Introduction Single Node Cluster NetEye Master Master-Satellite Architecture Underlying Operating System Acquiring NetEye ISO Image Installing ISO Image Single Nodes and Satellites Cluster Nodes Configuration of Tenants Satellite Nodes Only Nodes behind a Proxy Additional NetEye Components Single Node Cluster Node Satellites Nodes only Verify if a module is running correctly Accessing the New Module Cluster Satellite Security Identity and Access Management External Identity Providers Configure federated LDAP/AD Emergency Reset of Keycloak Configuration Advanced Configuration Roles Single Page Application in NetEye Module Permissions and Single Sign On Within NetEye Importing User Federation Groups inside another Group Importing OIDC IdP Groups inside another Group Resources Tuning Advanced Topics Basic Concepts & Usage Advanced Topics Monitoring Environment Templates Monitored Objects Import Monitored Objects Data Fields Deployment Icinga 2 Agents Configuration Baskets Dashboard Monitoring Status VMD Permissions Notifications Jobs API Configuring Icinga Monitoring Retention Policy NetEye Self Monitoring Concepts Collecting Events Add a Filter Node WHERE Conditions Iterating over Event fields Retrieving Payload of an Event Extract Variables Create a Rule Tornado Actions Test your Configuration Export and Import Configuration Example Under the hood Development Retry Strategy Configuration Thread Pool Configuration API Reference Configure a new Business Process Create your first Business Process Node Importing Processes Operators The ITOA Module Configuring User Permissions Telegraf Metrics in NetEye Telegraf Configuration Telegraf on Monitored Hosts Visualizing Dashboards Customizing Performance Graph The NetEye Geo Map Visualizer Map Viewer Configuring Geo Maps NagVis Audit Log Overview Shutdown Manager user Shutdown Manager GUI Shutdown Commands Advanced Topics Overview User Role Management Cube Use Cases ntopng and NetEye Integration Permissions Retention Advanced Topics Overview User Roles Nodes Test Cases Dashboard Use Cases Overview Architecture Authorization Kibana Elasticsearch Cluster Elasticsearch Configuration Replicas on a Single Node Elasticsearch Performance tuning Overview Enabling El Proxy Sending custom logs to El Proxy Configuration files Commands Elasticsearch Templates and Retentions El Proxy DLQ Blockchain Verification Handling Blockchain Corruptions El Proxy Metrics El Proxy Security El Proxy REST Endpoints Agents Logstash Elastic APM Elastic RUM Elastic XDR Log Manager - Deprecated Overview Authorization in the Command Orchestrator Module Configuring CLI Commands Executing Commands Overview Permissions Installation Single Tenancy Multitenancy Communication through a Satellite Asset collection methods Display asset information in monitoring host page Overview Customers Availability Event Adjustment Outages Resource Advanced Topics Introduction Getting Started SATAYO Items Settings Managed Service Mitre Attack Coverage Changelog Before you start Update Procedure Single Node Upgrade from 4.43 to 4.44 Cluster Upgrade from 4.43 to 4.44 Satellite Upgrade from 4.43 to 4.44 DPO machine Upgrade from 4.43 to 4.44 Create a mirror of the RPM repository Sprint Releases Feature Troubleshooting Tornado Networking Service Management - Incident Response IT Operation Analytics - Telemetry Identity Provider (IdP) Configuration NetEye Cluster on Microsoft Azure Introduction to NEP Getting Started with NEPs Online Resources Obtaining NEP Insights Available Packages Advanced Topics Upgrade to NetEye 4.31 Setup Configure swappiness Restarting Stopped Services Enable stack traces in web UI How to access standard logs Director does not deploy when services assigned to a host have the same name How to enable/disable debug logging Activate Debug Logging for Tornado Modules/Services do not start Sync Rule fails when trying to recreate Icinga object How to disable InfluxDB query logging Managing an Elasticsearch Cluster with a Full Disk Some logs are not indexed in Elasticsearch Elasticsearch is not functioning properly Reporting: Error when opening a report Debugging Logstash file input filter Bugfix Policy Reporting Vulnerabilities Glossary

Elasticsearch Configuration

Elasticsearch settings need to be added to their configuration files at run-time.

Starting from NetEye 4.16 release, configuration files for Elasticsearch are not anymore modified by neteye install. The support of the run-time configuration is instead done via environment.

The default values for NetEye are stored in the /neteye/local/elasticsearch/conf/sysconfig/elasticsearch file and they can be overridden by creating the /neteye/local/elasticsearch/conf/sysconfig/elasticsearch-user-customization file and specify the new values.

By restarting Elasticsearch, the new settings are now loaded at run-time, thus overriding the default ones.

Elasticsearch temporary directory

NetEye uses the /neteye/local/elasticsearch/data/tmp directory as the temporary storage for Elasticsearch. It is essential to ensure that this directory resides on a filesystem that does not have the noexec mount option enabled. This directory shall be changed to a different location if the default one can not meet the requirements, by setting the ES_TMPDIR environment variable in the user customization file.

Elasticsearch Backup and Restore

Elasticsearch provides snapshot functionality which is great for backups because they can be restored relatively quickly.

The main features of Elasticsearch snapshots are:
  • They are incremental

  • They can store either individual indices or an entire cluster

  • They can be stored in a remote repository such as a shared file system

The destination for snapshots must be a shared file system mounted on each Elasticsearch node.

For further details see the Official Elasticsearch snapshot documentation.

Elasticsearch security helper tool

The secure communication provided by the X-Pack Security requires additional parameters such as authentication certificates to interact with the Elastic Stack APIs. We have developed a few helper tools, based on curl, to simplify your interaction with the APIs.

The Elasticsearch helper script lets you omit all the authentication parameters for the admin user, which would otherwise be required.

Location: /usr/share/neteye/elasticsearch/scripts/es_curl.sh

The NetEye helper script can be used instead if you only need read permission for the fields @timestamp and host on the Logstash index entries. This script is used by NetEye for self-monitoring activities.

Location: /usr/share/neteye/elasticsearch/scripts/es_neteye_curl.sh

Plugins

Plugins extend the core functionality of Elasticsearch. They range from adding custom mapping types, custom analyzers, native scripts, custom discovery and more.

Plugins can come from different sources: the official ones created or at least maintained by Elastic, community-sourced plugins from other users, and plugins that you provide.

Core plugins are part of Elasticsearch project, and are delivered at the same time as Elasticsearch. Their version number always matches the version number of Elasticsearch itself.

Community contributed plugins are external to the Elasticsearch project. They are provided by individual developers or private companies and have their own licenses as well as their own versioning system.

Plugins contain JAR files, but may also contain scripts and config files, and must be installed on every node in the cluster.

Run the following command to source sysconfig variables:

. /usr/share/neteye/elasticsearch/scripts/es_autosetup_functions.sh; source_elasticsearch_sysconfig

Now, an actual plugin command can be run:

ES_PATH_CONF=${ES_PATH_CONF} /usr/share/elasticsearch/bin/elasticsearch-plugin install [plugin_name]

This command will install the version of the plugin that matches your Elasticsearch version. Upon every neteye update / neteye upgrade the plugins will be updated to the latest version available.

A plugin can also be downloaded directly from a custom location by specifying the URL, from your local file system, or from an HTTP URL. Please consult official installation guide for more details on various plugin installation methods.

After installation, each node must be restarted before the plugin becomes visible.

Some of the official plugins are always provided with the service, and can be enabled per deployment.

Note

When running neteye update / neteye upgrade for deployments with community contributed plugins installed, the latter must be manually removed from all nodes before the running the precedure, and re-installed after the procedure is successfully completed. This will prevent neteye update / neteye upgrade from failing due to not being able to automatically re-install a plugin from a custom source.

Check out the official Elasticsearch guide to find more information on plugin management options.