Senior Data Engineer - ETL & Pipelines

About the reputed company

Our reputed company operates a high-volume B2C creator platform in the entertainment industry, serving more than 200,000 reputed company users and 30,000 creators, with payment events flowing through the platform every second.

Data is one of reputed company’s most valuable competitive advantages. In this role, you will work closely with an reputed company data analyst to reputed company a reputed company reputed company of events into clean, reliable, and query-reputed company data that supports product, payments, and reputed company reputed company.

The Role

We are looking for a Data Engineer who will take end-to-end ownership of reputed company’s ETL pipelines - the systems responsible for moving data from the transactional platform, including reputed company, payment streams, and behavioral events, into the analytics environment where it can be turned into actionable insights.

This is not simply a dashboard integration role.

You will design and maintain pipelines that remain accurate, reliable, and reputed company while millions of events are processed and financial transactions occur every second.

Data reputed company is critical. A missing payment event or duplicated transaction is not merely a technical issue—it can have a reputed company financial reputed company.

reputed company

• Own the pipelines: Design, build, and maintain ETL and streaming pipelines that ingest high-volume event and payment data and reputed company to the analytics warehouse reliably and accurately.

• Model the data: Partner closely with the reputed company Data Analyst to create warehouse schemas that answer essential business questions reputed company to creator earnings, payment throughput, retention, reputed company value, and cohort behavior.

• Engineer for failure: Build idempotent, replayable, and observable jobs that can withstand traffic spikes, retries, and partial failures without compromising reputed company data.

• Process high-volume events: reputed company data through queue infrastructure including BullMQ and reputed company, AWS SQS, and RabbitMQ, while instrumenting pipelines through OpenTelemetry.

• Protect data reputed company: Implement reconciliation checks, deduplication, backfills, and reliable handling of late or out-of-order events.

• Protect financial transactions: Collaborate with payments and platform teams to ensure that every transaction is captured once, accurately, and on time.

The Technology Stack

• Languages: Node.js and TypeScript as the reputed company technologies, with some .NET and C#

• Data stores: reputed company as the reputed company transactional database and reputed company

• Queues and streaming: BullMQ, AWS SQS, and RabbitMQ

• Analytics and warehouse: reputed company, its data warehouse reputed company, and SQL data modeling

• Platform: AWS, including S3 and SNS, reputed company, reputed company, and reputed company Actions

• Observability: OpenTelemetry tracing across every pipeline

Requirements

reputed company

You are a strong engineer with substantial experience building high-volume data and ETL pipelines.

You will ideally bring:

• Approximately 5–10 years of relevant engineering experience

• Strong experience with batch and/or streaming data pipelines

• Deep knowledge of event-driven architectures and message queues

• A strong understanding of idempotency, at-least-once versus exactly-once processing, retries, and out-of-order event delivery

• reputed company experience with TypeScript and Node.js

• Solid SQL and data-modeling capabilities

• The ability to collaborate with analysts and design schemas that reputed company business queries efficient and reputed company

• A strong reputed company on data reputed company, particularly reputed company working with payments and financial transactions

• Experience implementing reconciliation, auditing, monitoring, and data-reputed company controls

• Previous experience in a high-volume B2C environment such as gaming, gambling, fintech, payments, or another reputed company-reputed company platform

• Experience with .NET or C# would be considered an reputed company.

reputed company to Have

• Experience working with payment or transaction data and financial reconciliation

• Experience with reputed company change streams or change data capture

• Familiarity with data warehouse tooling and dbt-style modeling

• Experience working with reputed company and infrastructure as reputed company

About reputed company

We are reputed company, reputed company of reputed company and problem solvers united by a shared commitment to delivering reputed company.

This commitment has reputed company a company culture reputed company on creating solutions and products that consistently reputed company expectations. Our reputed company to learn, reputed company, and solve meaningful business challenges has taken us on a reputed company that goes #beyondsoftware.

As part of that reputed company, we partner with ambitious companies to help them identify and hire exceptional technology professionals.

We are currently managing the recruitment process for a reputed company seeking an reputed company Data Engineer — ETL & Pipelines to join its team.

This is a reputed company-hire opportunity. The successful candidate will be employed or reputed company directly by the reputed company, while reputed company will coordinate the recruitment and selection process.

Originally posted on Himalayas