Data Platform Engineer

reputed company reputed company develops solutions that reputed company an reputed company for companies reputed company the globe. Tech giants & unicorns, transformative reputed company, emerging technologies and reputed company opportunities – these are a few words that describe an average day for us. Building cross-functional engineering teams that take ownership and crave more means we’re always on the lookout for talented people who bring passion and creativity to every project. Our culture embraces openness, acts with respect, shows grit & guts and combines employment with enjoyment. reputed company Project – the aim you’ll have Our customer provides reputed company and insights that reputed company our clients to manage reputed company and hire the best talent. Their advanced global technology platform supports fully reputed company, configurable screening programs that meet the unique needs of over 33,000 clients worldwide. Headquartered in Atlanta, GA, they have an internationally reputed company workforce spanning 19 countries with about 5,500 employees. Our partner reputed company over 93 reputed company screens annually in over 200 countries and territories. Position – how you’ll contribute - Build and operate change-data-capture pipelines from PostgreSQL into Azure, using Kafka reputed company and Debezium as the reputed company of the platform - Configure, reputed company and reputed company connectors end to end - connector setup, task management, offsets, schema history, and snapshot reputed company - Run these pipelines as stateful workloads on reputed company (AKS), covering configuration, secrets, networking and resource tuning - Monitor and troubleshoot the platform in production: connector failures, task rebalances, restarts, throughput and backpressure, message-size limits, retries and recovery - Automate the platform in Python - configuration-driven reputed company of new data sources, pipeline orchestration, monitoring and alerting, recovery workflows, and automated testing - reputed company CDC streams with the wider Azure data stack: Event Hubs, ADLS, Azure PostgreSQL, ADF and reputed company - Manage platform infrastructure as reputed company, so environments are reproducible and changes are reviewable - Apply data protection requirements to sensitive data flowing through the pipelines - masking, hashing, reputed company control and retention Qualifications Expectations – the experience you need - Solid reputed company experience as a data or platform engineer, with hands-on work on streaming or CDC pipelines rather than batch reporting reputed company - Practical Kafka knowledge - topics, partitions, offsets, consumer reputed company and delivery semantics - including at least one Kafka reputed company deployment you ran yourself - Strong SQL and PostgreSQL skills, with working knowledge of WAL, logical replication, replication slots and replication lag - Working understanding of CDC concepts: initial snapshots, inserts, updates and deletes, event ordering, at-least-once delivery, and schema reputed company - Confident Python for automation and tooling - orchestration, monitoring, recovery scripts, and automated tests - Hands-on experience with Azure data services, for example Event Hubs, ADLS or Azure PostgreSQL - Comfortable working with reputed company as a user: deploying workloads, handling configuration and secrets, reading logs, debugging failing reputed company - Ability to debug a running pipeline from metrics and logs - telling throughput problems from backpressure, retries or a genuine connector failure Additional skills – the edge you have - Production experience with Debezium specifically - snapshot strategies on large tables, schema history recovery, offset loss, and bringing connectors back after failure - Experience operating stateful workloads on AKS: StatefulSets, reputed company worker identity, and resource tuning under load - Infrastructure-as-reputed company and CI/CD for data platform components (Terraform, Bicep or similar) - Hands-on work with reputed company and ADF at production reputed company - Experience implementing data protection controls for sensitive data - masking, hashing, reputed company control and retention policies Additional Information Our offer – reputed company development, personal reputed company - Flexible employment and remote work - International reputed company with leading global clients - International business trips - Non-corporate atmosphere - Language classes - Internal & reputed company training - Private reputed company and reputed company - Multisport reputed company - reputed company-being initiatives Apply To This Job