Big Data has emerged as the next frontier in information technology to address the need for managing huge data sets and get meaningful insights. Skills in big data has emerged as the new age skill, having huge demand across businesses. Due to increased use of mobile and online for consumption, making meaning of the data is getting more critical for companies.
Mr. Jasneet Singh from Vskills, interviewed Hardeep Singh, SDE at Right Relevance to update us on the right Big data skills. The interview also lists how to develp skills for beginners, to have opportunities in Data science, machine learning, Hadoop, Mapreduce and big data analytics.
Please tell our readers something about yourself
B-Tech CSE from IIT Ropar, I am working as a full stack development engineer at Right Relevance. In past I’ve worked mostly on Frontend Technologies like Ember Angular etc. My career around Big Data has started in recent past.
What inspired you to choose a career in Big Data ?
With the increasing rate of adaption of Technology the Digital Data is increasing exponentially. There are many problems we couldn’t solve earlier just because we didn’t have enough data to create solutions that work, but today we can and the results are phenomenal e.g major machine learning problems are solved using Big Data, self driving cars, amazon recommendations, facebook friend suggestions etc.
What were your professional qualifications and how did it help to get a job in Big Data ?
I’ve worked at multiple companies solving problems with frontend architectures and implementing some proprietary Microsoft protocols, which lead me to Right Relevance. Right Relevance is an analytics company deeply involved in Big Data.
What courses do you recommend to students and professionals who want to pursue Big Data as a profession ?
Big Data Specialisation course from Coursera will teach you everything about Big Data, initial understanding of Machines, Clusters, Networks would be a plus, generally a B-Tech/ MCA degree would work as a pre-requisite.
Is studying at a top institution essential to make a career in Big Data ?
No, it’s not related, I think if you can get good ranks solving some known online problems on Big Data, that would be a major plus, ultimately showing your solutions to problems would lead to a better impression than your college degree.
What are the books / websites / online courses that you would recommend to freshers in your profession ?
For Books I would recommend
1) Hadoop The Definite Guide
2) Introduction to Apache Hive, Apache Spark, Apache Kafka.
And Big Data Specialisation Course from Coursera includes everything needed to kick start a Big Data Project.
According to you, which area of specialization in Big Data has the most promising future ?
Big data is transforming lives everywhere, every big or small company today is using Big Data, to improve user experience, to recommend better products. Big Data is everywhere, but personally I feel Machine Learning is going to be most promising future for Big Data analytics.
In your experience, what are the common qualities and skills that you have identified in successful professionals in Big Data ?
From my experience skills of a successful professions in Big Data professional includes Java language, AWS or Azure knowledge, understanding of operating systems, understanding of hardware and clusters. The most important quality is the great sense of debugging these systems because these systems don’t get built in one iteration, they take multiple iterations to get right and perfect.
Other qualities include
1) Performance Measurement
2) Agility
3) End to End System Testing.
What challenges have you faced in the course of your career and how did you overcome them ?
The number of technologies are huge, you have to find out what you want to use. Then, you will have to figure out their configurations because the hardware requirements vary according to use of Technology. These hardware requirements are generally too high to manage finances. Keeping these things in mind, building a technology stack is I guess includes major challenges in my course career. I had to read a lot, sometimes multiple times to understand each configuration and how at the core one technology is different from other, this has given me a lot of clarity and now all that hard work helps me a lot to build resilient systems.
What is the most satisfying and enjoyable aspect of your job ?
Seeing your analytics solutions solving everyday problems of a lot of people and hence saving a lot of time of a lot of people, is I think the most satisfying aspect. The unexpected challenges that occur while building the big data solution stack is the most enjoyable aspect.
What is the dividing line between an average professional in Big Data and an excellent one ?
The Excellent One will build a “Resilient System” really fast growing from a prototype, but an average one would try to build a “Solution” slowly keeping every possibility in mind and solving problems that might not occur, which is generally not needed and is a major cost to company.
What is your advice to aspiring professionals who want to build career in Big Data ?
The Big Data Ecosystem is huge and it is evolving at a very fast pace, you don’t need to learn everything but you should know what is happening everyday in this ecosystem, what technologies are used to solve which type of problems. These are basic things generally asked in Job Interviews.
-Team Vskills
