Adtech Logs processing Pipeline with Apache Beam, Cloud Dataflow.

• Serialization of messages with Protocol Buffer protobuf files
• Dataflow Pipeline Source Code

Benchmark:

• The task was to develop and optimize a pipeline that could run on Cloud Dataflow in less than 20 minutes with standard 8 vCPU machines.
• The source code is highly optimized.