Home

Welcome to the Sawmill wiki!

Sawmill is a JSON transformation open source library.

It enables you to enrich, transform, and filter your JSON documents.

Using Sawmill pipelines you can integrate your favorite groks, geoip, user-agent resolving, add or remove fields/tags and more in a descriptive manner, using configuration files or builders, in a simple DSL, allowing you to dynamically change transformations.

Getting Started

Lets start with a basic example illustrating how to use Sawmill:

Doc doc = new Doc(myLog);
PipelineExecutor pipelineExecutor = new PipelineExecutor();
pipelineExecutor.execute(pipeline, doc);

As you can see above, there are few entities here:

• Doc - essentially a Map representing a JSON.
• Processor - a single document logical transformation. Either grok-processor, key-value-processor, add-field and so on.
• Pipeline - specifies a series of processing steps using an ordered list of processors. Each processor transforms the document in some specific way. For example, a pipeline might have one processor that removes a field from the document, followed by another processor that renames a field.
• PipelineExecutor - executes the processors defined in the pipeline on a document. The PipelineExecutor is responsible for the execution flow - handling onFailure and onSuccess flows, stops on failure, expose metrics of the execution and more.
• PipelineExecutionTimeWatchdog - responsible for warning on long processing time, interrupts and stops processing on timeout (not shown in the example above).

Here’s a simple code sample showing GeoIP resolution:

package io.logz.sawmill.processors;

import io.logz.sawmill.Doc;
import io.logz.sawmill.ExecutionResult;
import io.logz.sawmill.Pipeline;
import io.logz.sawmill.PipelineExecutor;

import static io.logz.sawmill.utils.DocUtils.createDoc;

public class SawmillTesting {

    public static void main(String[] args) {

        Pipeline pipeline = new Pipeline.Factory().create(
                "{ steps :[{\n" +
                "    geoIp: {\n" +
                "      config: {\n" +
                "        sourceField: \"ip\"\n" +
                "        targetField: \"geoip\"\n" +
                "        tagsOnSuccess: [\"geo-ip\"]\n" +
                "      }\n" +
                "    }\n" +
                "  }]\n" +
                "}");

        Doc doc = createDoc("message", "testing geoip resolving", "ip", "172.217.11.174");
        ExecutionResult executionResult = new PipelineExecutor().execute(pipeline, doc);

        if (executionResult.isSucceeded()) {
            System.out.println("Success! result is:"+doc.toString());
            // will print out:
            // Success! result is:Doc{source={message=testing geoip resolving, ip=172.217.11.174, geoip={timezone=America/Los_Angeles, ip=172.217.11.174, latitude=37.419200000000004, continent_code=NA, city_name=Mountain View, country_name=United States, country_code2=US, dma_code=807, region_name=CA, location=[-122.0574, 37.419200000000004], real_region_name=California, postal_code=94043, longitude=-122.0574}, tags=[geo-ip]}}
        }
    }
}