...
The sources generating log files are a freeRadius and dhcp a DHCP server where Filebeat was installed as an agent. Thus the data flow starts with Filebeat collecting log events and forwarding them to Logstash. At Logstash, the logs are filtered/enriched according to the needs of WiFiMon, before sending them toward Elasticsearch nodes in the cluster.
| Note | ||
|---|---|---|
| ||
To implement this setup in your environment:
|
...
The following table shows the DNS configuration and the role each machine plays in the cluster.
| FQDN | Role |
|---|---|
| wifimon-node1.example.org | master-eligible / data node |
| wifimon-node2.example.org | master-eligible / data node |
| wifimon-node3.example.org | master-eligible / data node |
| wifimon-kibana.example.org | coordinating node |
| wifimon-logstash.example.org | pipeline node |
Cluster node is considered to be the one that joins the cluster. In this setup, cluster nodes are the master-eligible/data nodes and the coordinating node. The pipeline node is not, it doesn’t join the cluster.
...
Being a cluster of Elasticsearch nodes, Java (at least version 8) is required, so the java-1.8.0-openjdk package was installed on in each node.
Having the Java dependency satisfied, the next step was to install the elasticsearch package on in each cluster node, that is not in the pipeline node. For more information see Install Elasticsearch with RPM.
On In the coordinating node, along with elasticsearch, the kibana package was installed, too. For more information see Install Kibana with RPM.
On pipeline node The logstash package was installed in the logstash packagepipeline node. For more information see Installing Logstash.
The filebeat package was installed in a the dhcp server and in the freeRadius server which implements the Eduroam Service Provider. For more information see Repositories for APT and YUM.
...
Each node’s hostname is set to its FQDN, according to the values shown in the VMs DNS table. This value is referenced in the configuration file of Elasticsearch, and is also used in certificates for hostname validation.
It is recommended to disable system swapping, which can result in parts of JVM Heap or even its executable pages being swapped out to disk.
...
The cluster communication is secured by configuring SSL/TLS encryption. The elasticsearch-certutil was used to generate a CA certificate, utilized for signing while generating the certificates of the cluster components certificates. This utility comes with the elasticsearch installation, and in this case was used the one installed in the wifimon-kibana.example.org node.
...
| Note | ||
|---|---|---|
| ||
On a running elasticsearch instance: If the command "systemctl -l status elasticsearch.service" produces the following warning:
then (according to JEP 291) comment out the option "-XX:+UseConcMarkSweepG" and set the option "-XX:+UseG1GC". If the file "/var/log/elasticsearch/wifimon_deprication.log" contains warnings like the following:
then proceed with the recommendation, that is set the option "-Des.transport.cname_in_publish_address=true". |
...
The node.name is set to the value of ${HOSTNAME}, that is the value of the node’s FQDN. This The value for this setting can also be configured given explicitly to some value.
The node.master makes this node eligible to be elected as a master node which controls the cluster. Every master-eligible node, which is not a voting_only node, can be the master node of the cluster.
...
The cluster.remote.connect setting makes this node to function as a cross-cluster client able to connect to remote clusters. This is not the case of this setup so it is set to false.
...
A coordinating node is an Elasticsearch node which joins the cluster as every cluster node. In this setup, the coordinating node is named wifimon-kibana.example.org because the Kibana visualization platform has been installed and configured on in it.
Below is the configuration of wifimon-kibana.example.org as an Elasticsearch coordinating node. It follows the same pattern as the master-eligible/data nodes, but with their functionalities set to false.
...