Engineering Manager – Data Engineering

Must-Have skills:

-10–15+ years of experience in Data Engineering.

-3–5+ years of experience leading Data Engineering teams.

- Must be a Manager

-Strong hands-on experience with Databricks on Azure.

-Strong expertise in PySpark, Spark SQL, Python, and SQL.

-Hands-on experience with Delta Lake, Delta Live Tables (DLT), Unity Catalog, Databricks Workflows, and Auto Loader.

Experience designing and building enterprise-scale data platforms.

Strong knowledge of ETL/ELT, Medallion Architecture, and batch & streaming data pipelines.

Additional Guidelines :

Interview process  - 2 technical + 1 client round + HR

Relocation is supported.

Temporary accommodation for a couple of weeks.

About the position:

Designation Engineering Manager Data Engineering

Experience required- 15-20 years.

WORK LOCATION : Pune

Shift: General Shift

Shift Timing/On-Call/Weekend Support Needed: General Shift (Should flexible to extend and support)

Rounds of Interview– 2 technical + 1 client round + HR

Relocation is supported.

Temporary accommodation for a couple of weeks.

ABOUT THE ROLE: Engineering Manager Insurance Domain

Job Title: Engineering Manager Data Engineering

Location: Pune, Maharashtra, India

Work Mode: Hybrid

Experience Required: 10-15 Years

Open Positions: 1

Job Summary

We are seeking an experienced and dynamic Engineering Manager Data Engineering to lead cross-functional data engineering teams and drive the design, development, and delivery of enterprise-scale data platforms. This is a 50-50 blend of technical leadership and delivery management role where you will combine hands-on technical expertise with strategic people leadership. You will be responsible for leading multiple squads (20+ members), ensuring exceptional delivery quality, and building scalable data solutions using Databricks on cloud platforms. The ideal candidate will have a proven track record of end-to-end delivery, strong stakeholder management capabilities, and deep expertise in modern data engineering practices, particularly in the Banking/Financial Services sector.

Key Responsibilities

Technical Leadership & Architecture (50%)

• Lead the design and implementation of enterprise-scale data platforms using Databricks as the primary technology stack.
• Define and enforce technical architecture, coding standards, and engineering best practices across all data engineering initiatives.
• Drive adoption of modern data engineering patterns including Medallion Architecture, Delta Lake, and Delta Live Tables (DLT).
• Conduct comprehensive solution design reviews and provide technical guidance on complex data engineering challenges.
• Review code quality, performance optimization strategies, and ensure adherence to data security, governance, and compliance standards.
• Mentor team members on advanced Databricks features, PySpark optimization, and cloud-native data architectures.
• Stay current with emerging technologies and industry trends; champion innovation within the team.
• Ensure implementation of robust data modeling strategies (Star Schema, Snowflake, Data Vault) and ETL/ELT design patterns.

Delivery Management & Execution (50%)

• Own end-to-end delivery accountability for multiple data engineering initiatives, from requirements gathering through production deployment and post-go-live support.
• Lead and manage 2+ squads (20+ members) with a strong focus on delivery excellence and quality outcomes.
• Develop comprehensive project plans, timelines, and resource allocations with accurate estimation, budgeting, and costing.
• Execute sprint planning and management; track progress against milestones and KPIs.
• Proactively identify, assess, and mitigate delivery risks and dependencies.
• Ensure production readiness through rigorous testing, validation, and release management processes.
• Drive continuous improvement in engineering processes, tools, and team productivity.
• Manage stakeholder expectations, communicate progress transparently, and prioritize business requirements effectively.

People Leadership & Development

• Lead, mentor, and develop a high-performing team of Data Engineers and Technical Leads.
• Conduct regular one-on-one meetings, performance reviews, and career development discussions.
• Support hiring initiatives, onboarding programs, and capability development plans.
• Foster a culture of collaboration, innovation, accountability, and continuous learning.
• Identify and nurture talent; create pathways for career growth within the team.

Operational Excellence & Stakeholder Management

• Ensure platform reliability, availability, and optimal performance in production environments.
• Drive root cause analysis for production incidents and implement preventive measures.
• Improve monitoring, alerting, observability, and incident response capabilities.
• Optimize cloud infrastructure costs and resource utilization across Azure, AWS, or GCP.
• Collaborate effectively with Product Owners, Solution Architects, Business stakeholders, and Platform teams.
• Communicate technical decisions, risks, and delivery status to senior leadership and business partners.

Required Skills & Competencies

Databricks & Data Engineering (Must-Have)

• Databricks Expertise: Databricks Workspace, Delta Lake, Delta Live Tables (DLT), Unity Catalog, Databricks Workflows, Auto Loader, Structured Streaming
• Programming: Python, PySpark, Spark SQL (SQL is mandatory; Python/PySpark is good-to-have)
• Data Engineering Patterns: ETL/ELT design, Medallion Architecture, Batch and Streaming data pipelines
• Data Modeling: Star Schema, Snowflake Schema, Data Vault methodologies
• Database Technologies: SQL Server, Oracle, Snowflake, PostgreSQL

Cloud Platforms (Must-Have)

• Hands-on experience with Microsoft Azure (primary) and/or AWS
• Cloud Storage: ADLS (Azure Data Lake Storage), S3, or GCS
• Understanding of cloud-native architectures and infrastructure optimization

DevOps & CI/CD

• Git and version control best practices
• Azure DevOps or GitHub for CI/CD pipeline management
• Infrastructure as Code (Terraform preferred)
• Release management and deployment automation

Leadership & Delivery Management (Must-Have)

• Engineering leadership with proven experience leading multiple squads (20+ members)
• Agile delivery and project management expertise
• Strong planning, estimation, budgeting, and costing skills
• Stakeholder management and executive communication
• Risk management and conflict resolution
• Coaching, mentoring, and team development capabilities
• Solution and design thinking with end-to-end delivery ownership
• Excellent communication and interpersonal skills

Domain Expertise (Preferred)

• Banking/Financial Services industry experience (highly preferred)
• Understanding of financial data governance, compliance (GDPR, SOX), and security requirements

Experience Requirements

• Total Experience: 10-15+ years in Data Engineering and related roles
• Leadership Experience: 3-5+ years leading engineering teams or squads
• Databricks Experience: Hands-on, production-grade experience with Databricks on Azure, AWS, or GCP (mandatory)
• Enterprise-Scale Platform Development: Proven experience building and delivering enterprise-scale data platforms
• End-to-End Delivery: Demonstrated ability to own complete delivery lifecycle from requirements gathering, design, development, testing, production deployment, and post-go-live support
• Performance Tuning & Optimization: Experience optimizing data pipeline performance, cost management, and production support
• Solution Architecture: Strong background in designing and implementing complex data solutions

Preferred Certifications & Qualifications

• Databricks Certified Data Engineer Professional
• Azure Data Engineer Associate or Azure Solutions Architect Expert
• AWS Certified Data Analytics Specialty
• Bachelor's degree in Computer Science, Engineering, or related field

Nice-to-Have Skills

• MLflow and ML pipeline orchestration
• Databricks Asset Bundles (DABs)
• Apache Kafka or Azure Event Hubs experience
• AI/ML pipelines and Generative AI knowledge
• Data Mesh or Data Fabric architecture experience
• Databricks Genie or AI-assisted development tools
• NoSQL databases (MongoDB, Cassandra)
• Advanced monitoring and observability tools

Key Competencies

• Technical Expertise: Deep hands-on knowledge of modern data engineering and cloud technologies
• Leadership: Ability to inspire and lead high-performing teams toward ambitious goals
• Delivery Excellence: Demonstrated track record of on-time, quality delivery of complex projects
• Strategic Thinking: Capability to align technical solutions with business objectives
• Communication: Exceptional verbal and written communication skills; ability to articulate complex concepts to diverse audiences
• Problem-Solving: Strong analytical and troubleshooting capabilities
• Adaptability: Comfortable in fast-paced, dynamic environments with evolving requirements
• Accountability: Takes ownership of outcomes and drives results

What We're Looking For

• A hands-on technical leader who can code and architect solutions while managing teams
• Someone with proven expertise in Databricks and enterprise data platform development
• A delivery-focused professional with strong planning and estimation capabilities
• A leader who has successfully managed multiple squads and driven complex, multi-initiative programs
• Banking/Financial Services industry experience is a significant plus
• An individual with excellent communication skills who can engage effectively with all stakeholder levels
• A champion of quality, continuous improvement, and operational excellence

About the client:

NCS is a leading AI Tech Services company. With a 15,000-strong team across the Asia Pacific, NCS scales its platforms and capabilities to provide clients with greater agility and AI expertise across a range of Industries. Embracing a strong ecosystem of global partners, NCS transforms technology services delivery combining AI with digital resilience to drive real business impact. NCS is a subsidiary of the Singtel Group. For more information, visit ncs.co.

Specialties
Business Application Services, Communications Engineering, ICT Professional Services, Digital, Infrastructure Services, Cloud, Technology, Products, Platforms, Consulting, Digital Transformation, and Cybersecurity