...
| Code Block |
|---|
| title | /etc/filebeat/filebeat.yml |
|---|
|
filebeat.inputs:
- type: log
enabled: true
paths: /tmp/radius_sample_logs
multiline.pattern: '^[[:space:]]'
multiline.negate: false
multiline.match: after
output.file:
path: /tmp
filename: sample_logs_output.json
processors:
- add_fields:
target: ''
fields:
logtype: radius
- drop_fields:
fields: ['input', 'host', 'agent', 'acs', 'log', 'ecs'] |
...
| Code Block |
|---|
| title | /tmp/sample_logs_output.json |
|---|
|
{"@timestamp":"2020-06-28T13:07:37.183Z","@metadata":
{"beat":"filebeat","type":"_doc","version":"7.8.0"},"logtype":"radius","message":"S
unSun Mar 10 08:16:05 2019\n\tService-Type = Framed-User\n\tNAS-Port-Id = \"wlan2\"\n\
tNAS-Port-Type = Wireless-802.11\n\tUser-Name = \"sgjeci@rash.al\"\n\tAcct-SessionId = \"82c000cd\"\n\tAcct-Multi-Session-Id = \"CC-2D-E0-9A-EB-A3-88-75-98-6C-31-AA82-C0-00-00-00-00-00-CD\"\n\tCalling-Station-Id = \"88-75-98-6C-31-AA\"\n\tCalledStation-Id = \"CC-2D-E0-9A-EB-A3:eduroam\"\n\tAcct-Authentic = RADIUS\n\tAcctStatus-Type = Start\n\tNAS-Identifier = \"Eduroam\"\n\tAcct-Delay-Time = 0\n\tNASIP-Address = 192.168.0.22\n\tEvent-Timestamp = \"Mar 8 2019 08:16:05 CET\"\n\tTmpString-9 = \"ai:\"\n\tAcct-Unique-Session-Id
= \"e5450a4e16d951436a7c241eaf788f9b\"\n\tRealm = \"rash.al\"\n\tTimestamp =
1552029365"} |
The logs are located in the message field. The logtype field holds the radius value, thus differentiating these events from the dhcp ones when arriving at Logstash pipeline.
...
| Code Block |
|---|
| title | /etc/filebeat/filebeat.yml |
|---|
|
filebeat.inputs:
- type: log
enabled: true
paths: /tmp/dhcp_sample_logs
include_lines: ['DHCPACK']
output.file:
path: /tmp
filename: sample_logs_output.json
processors:
- add_fields:
target: ''
fields:
logtype: dhcp
- drop_fields:
fields: ['input', 'host', 'agent', 'acs', 'log', 'ecs'] |
...
| Code Block |
|---|
| title | /tmp/sample_logs_output.json |
|---|
|
{"@timestamp":"2020-06-28T09:20:17.834Z","@metadata":
{"beat":"filebeat","type":"_doc","version":"7.8.0"},"message":"Jun 18 19:15:20
centos dhcpd[11223]: DHCPACK on 192.168.1.200 to a4:c4:94:cd:35:70 (galliumos) via
wlp6s0","logtype":"dhcp"} |
...
Apart from adding or dropping named fields, processors can also be used to filter log events when certain criteria are met. For example, to send out only the log events containing the value Eduroam in the NAS-Identifyer field, the following configuration could be applied.
| Code Block |
|---|
processors:
- drop_event:
when:
not:
regexp:
message: '.*NAS-Identifier.*=.*Eduroam.*' |
...
| Code Block |
|---|
output.logstash:
hosts: ["wifimon-logstash.rash.al:5044"]
ssl.certificate_authorities: ["/etc/filebeat/certs/ca.crt"]
ssl.certificate: "/etc/filebeat/certs/filebeat.crt"
ssl.key: "/etc/filebeat/certs/filebeat.key"
ssl.key_passphrase: "${key_passphrase}" |
...
For this configuration to work, the Elasticsearch index template must be manually loaded. Template autoloading is only supported for the elasticsearch output. Replace elastic-password-goes-here with the proper password and run:
| Code Block |
|---|
set +o history
filebeat setup --index-management \
-E output.logstash.enabled=false \
-E 'output.elasticsearch.hosts=["wifimon-kibana.rash.al:9200"]' \
-E output.elasticsearch.protocol=https \
-E output.elasticsearch.username=elastic \
-E output.elasticsearch.password=elastic-password-goes-here \
-E 'output.elasticsearch.ssl.certificate_authorities=["/etc/filebeat/certs/
ca.crt"]'
set -o history |
...
| Code Block |
|---|
monitoring.enabled: true
monitoring.cluster_uuid: "cluster-id-goes-here"
monitoring.elasticsearch.ssl.certificate_authorities:
["/etc/filebeat/certs/ca.crt"]
monitoring.elasticsearch.ssl.certificate: "/etc/filebeat/certs/filebeat.crt"
monitoring.elasticsearch.ssl.key: "/etc/filebeat/certs/filebeat.key"
monitoring.elasticsearch.ssl.key_passphrase: "${key_passphrase}"
monitoring.elasticsearch.hosts: ["https://wifimon-kibana.rash.al:9200"]
monitoring.elasticsearch.username: beats_system
monitoring.elasticsearch.password: "${beats_system_password}" |
...
| Code Block |
|---|
| title | /etc/logstash/logstash.yml |
|---|
|
path.data: /var/lib/logstash
path.logs: /var/log/logstash
queue.type: persisted
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: "logstash_system"
xpack.monitoring.elasticsearch.password: "${logstash_system_password}"
xpack.monitoring.elasticsearch.hosts: "https://wifimon-kibana.rash.al:9200"
xpack.monitoring.elasticsearch.ssl.certificate_authority:
"/etc/logstash/certs/ca.crt"
xpack.monitoring.elasticsearch.ssl.verification_mode: certificate
xpack.monitoring.elasticsearch.sniffing: true |
...
| Code Block |
|---|
| title | /etc/logstash/pipelines.yml |
|---|
|
- pipeline.id: beats-pipeline
path.config: "/etc/logstash/conf.d/beats-pipeline.conf"
- pipeline.id: radius-pipeline
path.config: "/etc/logstash/conf.d/radius-pipeline.conf"
- pipeline.id: dhcp-pipeline
path.config: "/etc/logstash/conf.d/dhcp-pipeline.conf" |
...
| Code Block |
|---|
| title | /etc/logstash/conf.d/radius-pipeline.conf |
|---|
|
input {
pipeline { address => radiuslogs }
}
filter {
mutate { gsub => [ "message", "[\n\t]+", " " ] }
kv {
allow_duplicate_values => false
include_keys include_keys => [
"Calling-Station-Id",
"Framed-IP-Address",
"Timestamp",
"Called-Station-Id",
"NAS-IP-Address",
"Acct-Status-Type"
]
remove_field => [
"logtype",
"message",
"@version"
]
}
if "beats_input_codec_plain_applied" in [tags] {
mutate { remove_tag => ["beats_input_codec_plain_applied"] }
}
geoip { source => "NAS-IP-Address" }
fingerprint {
key => "${cipher_key}"
method => "SHA512"
source => "Calling-Station-Id"
target => "Calling-Station-Id"
}
fingerprint {
key => "${cipher_key}"
method => "SHA512"
source => "Framed-IP-Address"
target => "Framed-IP-Address"
}
}
output {
stdout { codec => rubydebug }
} |
...
