Elasticsearch Configuration¶
Elasticsearch settings need to be added to their configuration files at run-time.
Starting from NetEye 4.16 release, configuration files for Elasticsearch are not anymore modified by neteye install. The support of the run-time configuration is instead done via environment.
The default values for NetEye are stored in the
/neteye/local/elasticsearch/conf/sysconfig/elasticsearch
file and they can be overridden by creating the
/neteye/local/elasticsearch/conf/sysconfig/elasticsearch-user-customization
file and specify the new values.
By restarting Elasticsearch, the new settings are now loaded at run-time, thus overriding the default ones.
Elasticsearch temporary directory¶
NetEye uses the /neteye/local/elasticsearch/data/tmp
directory as the
temporary storage for Elasticsearch. It is essential to ensure that this
directory resides on a filesystem that does not have the noexec
mount option enabled. This directory shall be changed to a different location if
the default one can not meet the requirements, by setting the
ES_TMPDIR environment variable in the user customization file.
Elasticsearch Backup and Restore¶
Elasticsearch provides snapshot functionality which is great for backups because they can be restored relatively quickly.
- The main features of Elasticsearch snapshots are:
They are incremental
They can store either individual indices or an entire cluster
They can be stored in a remote repository such as a shared file system
The destination for snapshots must be a shared file system mounted on each Elasticsearch node.
For further details see the Official Elasticsearch snapshot documentation.
Elasticsearch security helper tool¶
The secure communication provided by the X-Pack Security requires additional parameters such as authentication certificates to interact with the Elastic Stack APIs. We have developed a few helper tools, based on curl, to simplify your interaction with the APIs.
The Elasticsearch helper script lets you omit all the authentication parameters for the admin user, which would otherwise be required.
Location: /usr/share/neteye/elasticsearch/scripts/es_curl.sh
The NetEye helper script can be used instead if you only need read permission for the fields @timestamp and host on the Logstash index entries. This script is used by NetEye for self-monitoring activities.
Location: /usr/share/neteye/elasticsearch/scripts/es_neteye_curl.sh
Plugins¶
Plugins extend the core functionality of Elasticsearch. They range from adding custom mapping types, custom analyzers, native scripts, custom discovery and more.
Plugins can come from different sources: the official ones created or at least maintained by Elastic, community-sourced plugins from other users, and plugins that you provide.
Core plugins are part of Elasticsearch project, and are delivered at the same time as Elasticsearch. Their version number always matches the version number of Elasticsearch itself.
Community contributed plugins are external to the Elasticsearch project. They are provided by individual developers or private companies and have their own licenses as well as their own versioning system.
Plugins contain JAR files, but may also contain scripts and config files, and must be installed on every node in the cluster.
Run the following command to source sysconfig variables:
. /usr/share/neteye/elasticsearch/scripts/es_autosetup_functions.sh; source_elasticsearch_sysconfig
Now, an actual plugin command can be run:
ES_PATH_CONF=${ES_PATH_CONF} /usr/share/elasticsearch/bin/elasticsearch-plugin install [plugin_name]
This command will install the version of the plugin that matches your Elasticsearch version. Upon every neteye update / neteye upgrade the plugins will be updated to the latest version available.
A plugin can also be downloaded directly from a custom location by specifying the URL, from your local file system, or from an HTTP URL. Please consult official installation guide for more details on various plugin installation methods.
After installation, each node must be restarted before the plugin becomes visible.
Some of the official plugins are always provided with the service, and can be enabled per deployment.
Note
When running neteye update / neteye upgrade for deployments with community contributed plugins installed, the latter must be manually removed from all nodes before the running the precedure, and re-installed after the procedure is successfully completed. This will prevent neteye update / neteye upgrade from failing due to not being able to automatically re-install a plugin from a custom source.
Check out the official Elasticsearch guide to find more information on plugin management options.