In Part 2 of this 4 blog series, I described the tools and  frameworks we used to build Data Processing Platform. In this post, I will describe at a very high level the unified data processing platform architecture, different components of the platform and how they interact with each other.

Figure-1 : Data Processing Architecture

In the last blog of this series, I will provide specifics around how the sub-components interact with each other and how we use the platform to generate curated datasets that conform to a common enterprise wide Master Data Management model.  The curated datasets are then available for use by downstream services (reporting, self-service, advanced analytics etc).