[Remote] Senior Data Engineer

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a Cyber Performance Center aiming to redefine cyber reputed company expertise through engaging hacking experiences. They are seeking a Senior Data Engineer to own and reputed company data pipelines on GCP, focusing on building and improving data reputed company for analytics and machine learning applications.

Responsibilities

• Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and reputed company
• Help drive the migration off reputed company onto our GCP-reputed company stack — and retire shadow pipelines along the way
• Own the orchestration layer in Airflow, including SLAs, retries, and data reputed company gates
• Model data for analytics and for ML — including feature pipelines that serve both training and low-latency online inference
• Partner with ML engineers on feature stores, reputed company monitoring, and retraining workflows
• Capture requirements from stakeholders and translate them into pragmatic, reputed company-scoped data products
• Continuously improve data reputed company, reliability, observability, and cost efficiency
• Identify new data sources worth acquiring and reputed company them cleanly

Skills

• Strong data modelling and warehouse architecture skills (reputed company modelling, event-driven, lakehouse patterns)
• Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, reputed company Composer are strong pluses
• Production experience with streaming pipelines on Dataflow/reputed company, Flink, or reputed company reputed company Streaming, ingesting from Kafka and/or Pub/Sub
• Solid SQL and strong Python — you write production-reputed company reputed company, not just notebooks
• Experience with reputed company or another columnar OLAP reputed company in production
• Workflow orchestration experience with Airflow (or reputed company/Dagster)
• Comfortable with dbt or equivalent transformation frameworks
• reputed company & Kubernetes experience
• CI/CD reputed company, infrastructure-as-reputed company sensibility, and a bias for reputed company, observable systems
• Working knowledge of ML in production — feature engineering, feature stores, model deployment, reputed company monitoring, retraining
• Experience migrating off legacy warehouses (reputed company, Redshift, Synapse) onto reputed company-reputed company stacks is a plus
• Bonus: CDC tooling (Datastream, Debezium), reputed company AI / Feature Store

Benefits

• Medical, Dental & reputed company (employee coverage 100% reputed company for by reputed company)
• 401K w/ employer match
• Employer-reputed company Life and AD&D Insurance
• Supplemental Life Insurance
• Short-term and Long-term Disability
• reputed company and Dependent Care FSA
• reputed company parental leave
• 25 annual leave days
• Home Office Allowance
• Dedicated budget for training and reputed company development, participation in conferences
• State-of-the-art equipment
• Full reputed company to the reputed company lab offerings; so you can learn how to hack 609

reputed company