Author: Nikhil Smotra

Analytics, Big Data professional with a passion for identifying opportunities in data to optimize business processes, better understand customers, their behaviors and preferences. I have a passion for transforming traditional enterprise applications to data driven applications that not only provide real time processing at scale using Big Data frameworks, but also use Machine Learning to make intelligent decisions. I love to crunch big data and have created data engineering and analytics practices from scratch, recruiting and building global teams.

Written by Nikhil SmotraJuly 11, 2015July 11, 2015

Moving from in-house Hadoop cluster to cloud based cluster – Part 1

We have been running Cloudera Hadoop Distribution (CDH) cluster in our data center. As we migrate more and more of our applications/products away from traditional architectures to Big Data, we have felt the need to use cloud for our infrastructure needs instead of an in-house Hadoop cluster. This should bring down our infrastructure and operational […]

Written by Nikhil SmotraApril 21, 2015May 31, 2015

Moving from Hadoop to Spark – challenges and work arounds

We are using an Oozie workflow on our production Hadoop cluster to manage a pipeline containing 57 MapReduce(MR) jobs. This pipeline of MR jobs does the following – Loads data from relational stores using Sqoop imports Calculates aggregations across multiple dimensions for our OLAP offering (MR jobs written in java) Writes end results back to […]

Written by Nikhil SmotraMay 8, 2014April 11, 2015

Securing Hadoop with Kerberos

Lately I was working on making our 10 node Hadoop cluster secure behind Kerberos. The cluster consists of CentOS 6.4 servers with CDH 5.0 distribution. Instead of using our company wide Active Directory running on Windows (didn’t want to mess up data for 32,000 users), I decided to run Kerberos Key Distriburtion Center (KDC) and realm local […]

Written by Nikhil SmotraApril 21, 2014October 22, 2014

Uninstalling CDH5 cluster

While implementing Kerberos security on a small CDH5 cluster (each machine running CentOS 6.4) recently, I ended up making quite a few configuration changes manually on few of the nodes in my cluster. The changes I made resulted in my cluster becomming unstable with few nodes having different configuration (from how I had set up […]

Written by Nikhil SmotraNovember 12, 2013April 11, 2015

Running Traditional Applications on Hadoop

I work on a cloud-based SaaS business analytics software that transforms multi-dimensional disparate enterprise data into actionable insights through user-friendly infographics, charts, and dashboards. There has been an exponential growth in the amount of data generated across the enterprise. and we’ve re-engineered our product to run on Big Data technologies powered (Hadoop […]