Job Description

Roles and Responsibilities:

• Resource and Project Timeline Estimation: Accurately estimate project timelines and resource allocation for Azure-based cloud projects.
• Defining Project and Program Management Process: Establish project and program management frameworks for Azure environments.
• Task Prioritization and Resource Management: Allocate tasks across resources by assessing skills and areas for improvement, with a focus on developing resource capabilities.
• Contribution to Organizational Initiatives: Involvement in sales collateral development, proposal design, leading Proof of Concepts, and training teams on Azure cloud technologies.
• Team Leadership: Recruit, train, develop, and supervise a team of Associates & Consultants, with a focus on cloud and data technologies.
• Specialization in Cloud Technologies: Expertise in Azure cloud and data services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and other Azure-based data solutions.
• End-to-End Data Pipeline Design: Understanding upstream data and designing efficient data pipelines on Azure, building data marts in Synapse Analytics, and making transformed data accessible to downstream applications.
• Data Pipeline Optimization: Build and optimize high-performance data pipelines using Azure Databricks, with ingestion, cleansing, and curation into Azure Data Lake, Azure SQL, or other Azure data platforms using Spark with Python, Scala, or Java.
• Scalable Frameworks: Develop scalable, reusable frameworks for data ingestion and transformation, leveraging Azure-native services like Azure Data Factory and Azure Functions.
• Data Quality Assurance: Ensure data quality and integrity across the entire Azure data pipeline, implementing robust validation and cleansing mechanisms.
• Data Warehouse/Mart Design: Experience designing and building data warehouses or data marts using Azure Synapse or SQL Database.
• RDBMS Expertise: Strong knowledge of SQL Server, able to write and understand complex SQL queries.
• Documentation & Agile Delivery: Expertise in requirement gathering, technical design, and functional documentation; proven experience delivering projects using Agile frameworks (Scrum, Kanban).
• Client and Stakeholder Engagement: Lead discussions with clients and stakeholders, including on-site and remote engagements.
• Problem Solving and Adaptability: Strong analytical skills and the ability to quickly adapt to new challenges and technologies.
• Communication Skills: Excellent verbal and written communication skills.

Job Requirement

Experience: 10 to 15 years of relevant experience in data engineering, with a focus on Azure-based cloud and data projects.

Location: Gurugram – Hybrid Mode

Technical Capabilities:

• Cloud Expertise: Experience managing projects on Azure Data Services, including Azure Data Lake, Azure Synapse Analytics, and Azure Databricks.
• Programming Languages: Expertise in Python, PySpark
• Databricks: Extensive experience working with Databricks on Azure.
• Database Expertise: Proficiency in SQL, with knowledge of SQL Server and Azure Cosmos DB (NoSQL).
• ETL/ELT Processes: Deep understanding of ETL/ELT processes and how to implement them using Azure Data Factory.
• DevOps & CI/CD Pipelines: Knowledge of CI/CD pipelines and DevOps practices using Azure DevOps or GitHub Actions.
• Agile Tools: Familiarity with Jira or similar tools used in agile environments.