Note: Don’t do this anymore, it is no longer needed.
The logstash tld plugin (logstash-filter-tld) is used to extract the individual labels of a fqdn. This is great for analysis of e.g. domains from a Domain Generation Algorithm (DGA). Unfortunately this plugin does not work on newer logstash versions. (See https://github.com/logstash-plugins/logstash-filter-tld/pull/10
This is caused by java version requirements. To make it work, all that is needed is changing that requirement, then (re)building the gem. This is a few simple steps (done on Debian below - substitute with your distros package manager).
On the development machine:
1. apt-get -y install jruby bundler 2. gem install bundler 3. git clone https://github.com/logstash-plugins/logstash-filter-tld.git 4. cd logstash-filter-tld/ 5. nano (or vi) logstash-filter-tld.gemspec 6. change logstash-filter-tld.gemspec, ‘=1.4.6’ '' to logstash-filter-tld.gemspec, ‘~> 3’ 7. gem build logstash-filter-tld.gemspec
The finalized plugin is now saved as logstash-filter-tld-3.0.3.gem. Copy that to your logstash instances, then install it using:
/usr/share/logstash/bin/logstash-plugin install –no-verify ./logstash-filter-tld-3.0.3.gem
For the logstash .conf-file(s) here’s an example using the tld plugin:
Take a look at these logstash configurations too: https://github.com/HASecuritySolutions/Logstash