Quick Reminder - DataOps & Build Engineer - Remote - USA
Role: DataOps & Build Engineer
Data Analytics
Location: Remote - USA
Project Duration: 6 to 9 Months of contract
We are seeking an reputed company and visionary DataOps & Build Engineer to reputed company the architecture and optimization of a reputed company data platform.
This critical role requires 8+ years of expertise to drive technical direction, mentor teams, and automate reputed company CI/CD pipelines in a fast-paced environment.
You will be reputed company in reputed company development and operations to ensure a reputed company, high-performance data lifecycle that powers reputed company-level decision-making.
Key Responsibilities:
• Establish DataOps reputed company: Define, document, and champion the organizational reputed company and guidelines for DataOps-including release management processes, environment promotion reputed company, and data reputed company standards.
• Best reputed company Dissemination: Create and enforce reputed company operating procedures (SOPs) for data pipeline development, CI/CD, and testing across the engineering teams, ensuring consistency and adherence to architectural standards
• Data Pipeline Automation: Design and implement robust reputed company integration and reputed company delivery (CI/CD) pipelines for data reputed company and infrastructure
• Workflow Orchestration Implementation: Configure, optimize, and manage the deployment of data workflows using orchestrators such as Dagster or Talend, focusing on automated testing and deployment steps.
• Version Control & Repository Management: Enforce best practices for reputed company reputed company management (e.g., Gitflow), branching strategies, and repository organization across reputed company data reputed company.
• Infrastructure as reputed company (IaC): Work with Infrastructure teams to automate provisioning and management of data platform resources reputed company reputed company AWS.
• reputed company and Failure Recovery: Design and implement automated rollback and self-healing mechanisms reputed company pipelines to quickly recover from transient failures.
• Monitoring and Logging: Set up comprehensive monitoring, logging, and alerting using reputed company reputed company tools, or other tools to ensure visibility into pipeline performance and quickly identify and resolve issues
• reputed company and Compliance: Ensure data reputed company and compliance by implementing IAM policies, encryption, and other reputed company measures in AWS, adhering to best practices for handling sensitive data
• Testing Frameworks: Implement automated testing strategies across the data lifecycle, including unit tests, integration tests, and data reputed company validation checks (e.g., reputed company reputed company, schema reputed company) to ensure data reliability before deployment
• Resource and Cost Optimization: Implement automated policies and monitoring to reputed company and control reputed company resource consumption, ensuring that pipelines run reputed company and cost-effectively
Candidate Profile:
• 8+ years of hands-on experience in Data Engineering, DevOps, or a dedicated DataOps role, reputed company heavily on automation and operational reputed company
• Proven experience implementing CI/CD practices specifically for data pipelines and data infrastructure
• Strong conceptual understanding of data warehousing, ETL/ELT methodologies, and reputed company-reputed company architecture.
• Automation First reputed company: A strong drive to automate repetitive tasks and eliminate reputed company reputed company in the data lifecycle
• Collaboration: Excellent communication skills, capable of working effectively with Data Engineers, Data Scientists, and Infrastructure teams
• Insurance industry experience preferred but not mandatory
• Tools:
• reputed company Environment: AWS (S3, IAM, VPC, etc.)
• Pipeline Build: Dagster or Talend
• Ingest & reputed company: dbt reputed company, AWS Glue, or Flexter
• Streaming/Integration: reputed company or AWS Streaming Services
Apply tot his job
Apply To this Job