Logstash: mutate nested field

Ah, the ELK stack. It seems like everybody is using it now. Tons of data in and lightning fast searches. Yelp even created a monitoring solution that works pretty well. This is a solution for developers and data architects. It’s not quite right for the average systems professional as it’s missing some [monitoring] features. I’m getting used to it.

I recently had an issue with an Elastic Search index where I wanted a field to be a float and the first value being sent by Logstash was a long, 99% of the time. Since Elastic Search types dynamically based on the first value, it’s a long.

The solution to this problem is to add a mutate section to the logstash config to catch this value before it gets to Elastic Search and force the float. I tried a number of methods, even tried using the Elastic Search mapping API, but that does not work if you use a daily index as the new index gets created with the (99% of the time) long vs float.

The specific issue was that Topbeat was sending the volume details for each volume and the filesystem.fs.used_p value was nearly always 0 as most volumes are pseudo-filesystems if all systems have a single disk mounted.

The logstash configuration that finally worked for me was the following:

filter {
        if [type] == "filesystem" {
                mutate {
                        convert => [ "[fs][used_p]" , "float" ]
                }   
        }   
}

• https://www.elastic.co/
• https://www.elastic.co/products/logstash
• https://www.elastic.co/products/kibana
• https://github.com/Yelp/elastalert