Domain Generation Algorithm

Logstash-Filter-tld

Note: Don’t do this anymore, it is no longer needed.

The logstash tld plugin (logstash-filter-tld) is used to extract the individual labels of a fqdn. This is great for analysis of e.g. domains from a Domain Generation Algorithm (DGA). Unfortunately this plugin does not work on newer logstash versions. (See https://github.com/logstash-plugins/logstash-filter-tld/pull/10 )
This is caused by java version requirements. To make it work, all that is needed is changing that requirement, then (re)building the gem. This is a few simple steps (done on Debian below - substitute with your distros package manager).
On the development machine:

1. apt-get -y install jruby bundler 2. gem install bundler 3. git clone https://github.com/logstash-plugins/logstash-filter-tld.git 4. cd logstash-filter-tld/ 5. nano (or vi) logstash-filter-tld.gemspec 6. change logstash-filter-tld.gemspec, ‘=1.4.6’ '' to logstash-filter-tld.gemspec, ‘~> 3’ 7. gem build logstash-filter-tld.gemspec

The finalized plugin is now saved as logstash-filter-tld-3.0.3.gem. Copy that to your logstash instances, then install it using:

/usr/share/logstash/bin/logstash-plugin install –no-verify ./logstash-filter-tld-3.0.3.gem

For the logstash .conf-file(s) here’s an example using the tld plugin:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

 filter {          
  
    tld {  
  
         source => "computer\_name"  
  
     }  
  
  
     mutate {  
  
         rename => { "\[tld\]\[domain\]" => "highest\_registered\_domain" }  
  
         rename => { "\[tld\]\[trd\]" => "sub\_domain" }  
  
         rename => { "\[tld\]\[tld\]" => "top\_level\_domain" }  
  
         rename => { "\[tld\]\[sld\]" => "parent\_domain" }  
  
     }  
  
 }

Take a look at these logstash configurations too: https://github.com/HASecuritySolutions/Logstash