Senior Data Engineer
India · Remote Employee - India · 6+ years experience · Remote · Posted Oct 5, 2026
At a glance
- Experience
- 6+ years experience
- Workplace
- Remote
- Location
- India · Remote Employee - India
About the role
- As a Senior Data Engineer on the Member Communication Platform team, you'll design and operate secure, scalable data products that power audience selection, eligibility, preferences, consent, campaign inputs, delivery data, and operational reporting.
- In your first year, success means delivering reliable Databricks-based pipelines and integrations, improving data quality and observability, and providing technical leadership across cross-functional initiatives that support timely, relevant, and compliant member communications.
Responsibilities
- You'll lead the design and implementation of scalable, secure data architectures for member communications, defining data models, contracts, lineage, and ownership.
- You'll independently lead complex data engineering initiatives from technical requirements and design through production delivery, coordinating work across multiple engineers and stakeholders while proactively managing dependencies, risks, and technical trade-offs.
- You'll design, develop, and optimize production-grade data pipelines and integrations using Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, and SQL, while modernizing appropriate legacy workloads.
- You'll build reliable data-loading, reconciliation, retry, idempotency, recovery, and deployment patterns, and help operate data lakehouse and operational data assets for performance, integrity, availability, and security.
- You'll establish data-quality and governance practices, including validation, monitoring, alerts, dashboards, runbooks, and lineage, and partner with Privacy, Compliance, Risk, and Security on PII, preferences, consent, suppression, retention, and access controls.
- You'll provide technical leadership through design and code reviews, mentor junior engineers, lead complex production investigations, identify root causes, and implement durable fixes that reduce recurring operational burden.
- You'll partner with Product, Communications, Analytics, Engineering, Risk, Privacy, Compliance, and other stakeholders to translate needs and technical trade-offs into actionable solutions.
- You'll drive release readiness through automated testing, CI/CD, rollout planning, and post-release measurement, monitor pipeline performance, quality, timeliness, cost, and reliability.
- You'll look for practical ways to use AI tools thoughtfully to work smarter, reduce manual work, and make better decisions faster.
Requirements
- You'll have 6+ years of experience in data engineering, with a focus on data architecture, data pipelines, data platforms, and database or lakehouse management.
- You'll have a bachelor's or master's degree in Computer Science, Data Science, Engineering, or a related field, or equivalent practical experience.
- You'll bring strong proficiency in Python/PySpark and SQL.
- You'll bring strong leadership, problem-solving, decision-making, and communication skills.
- You'll bring strong hands-on experience with Databricks, Apache Spark, Spark SQL, Delta Lake, workflow orchestration, CI/CD, Git-based workflows, automated testing, and end-to-end data integrations or products in production environments.
- You'll have experience with data quality, observability, alerting, lineage, cloud platforms and services (especially AWS), secure access controls, secrets management, encryption, auditability, sensitive member data, and Agile ways of working.
Preferred qualifications
- Experience in financial services, lending, marketing technology, customer communications, or another regulated environment.
- Familiarity with Unity Catalog or comparable data-governance tooling, and experience with Java or Scala.
- You've used AI tools such as ChatGPT, Copilot, or similar tools to improve engineering productivity, testing, documentation, troubleshooting, data quality, or operational effectiveness.
- Familiarity with using AI agents or automations to research, prototype, test, document, and deliver solutions across a broad technology stack with sound judgment for security, privacy, accuracy, and human review.
Benefits
- Medical, Dental & Vision Insurance
- Provident Fund
- Life & Accident Insurance
- Internet, Fuel, Meal & Telephone Allowances
Workplace
- Remote-India position, the role must be performed from within India.
- Excited to consider talent currently based in India, as well as candidates who are open to relocating to India for this opportunity.
Skills
- Python
- PySpark
- SQL
- Databricks
- Apache Spark
- Spark SQL
- Delta Lake
- Git
- AWS
- Java
- Scala
- ChatGPT
Show all 14 skills
- Copilot
- Unity Catalog
