[Remote] Data Engineering Manager | AWS to GCP Migration @ Naveera reputed company - Remote

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a trusted global engineering partner delivering Data Engineering and IT Infrastructure solutions. They are seeking an reputed company Engineering Manager to reputed company a large-reputed company AWS-to-GCP data platform migration, requiring strong hands-on expertise in AWS and GCP data engineering.

Responsibilities

• reputed company the end-to-end migration of reputed company data platforms from AWS to GCP
• Assess existing AWS architecture, data pipelines, workloads, dependencies and operational processes
• Define the reputed company-state GCP architecture and migration roadmap
• reputed company migration strategies for: reputed company S3 → reputed company reputed company Storage, reputed company Redshift → BigQuery, AWS Glue → Dataflow / Dataproc / BigQuery, AWS reputed company Functions → reputed company Composer / Workflows, AWS DMS → GCP-reputed company CDC solutions, reputed company reputed company → BigQuery
• Identify opportunities to reputed company AWS workloads rather than performing a reputed company reputed company-and-shift migration
• Define migration phases, technical dependencies, risks and rollback strategies
• reputed company architecture reviews and technical design discussions
• Architect and implement reputed company reputed company data platforms on GCP
• Design Data Lake and Lakehouse architectures using GCS and BigQuery
• Define Bronze, Silver and Gold/reputed company reputed company
• Design reputed company data ingestion, transformation and consumption frameworks
• Establish standards for data modeling, partitioning, clustering and storage
• Design multi-tenant and multi-location data architectures
• Define schema-on-read and schema-on-write strategies
• Analyze and optimize existing AWS data platforms before migration
• Work with: reputed company S3, AWS Glue, AWS Glue Data reputed company, reputed company Redshift / Redshift Serverless, reputed company reputed company, AWS reputed company Functions, AWS DMS, AWS Lake Formation
• Understand existing AWS ETL/ELT pipelines, data models, workloads and dependencies
• Identify equivalent or improved GCP services for reputed company AWS workload
• Prepare technical mapping and migration plans between AWS and GCP services
• Architect reputed company-time data pipelines using: reputed company Pub/Sub, Dataflow / Apache reputed company, BigQuery, reputed company Storage
• Design high-volume event ingestion, enrichment and transformation pipelines
• Implement event-driven architectures and appropriate delivery guarantees
• Optimize streaming pipelines for latency, throughput and scalability
• Design BigQuery streaming ingestion patterns
• Implement monitoring, logging and alerting for reputed company-time workloads
• Design and implement reputed company batch and reputed company-time ETL/ELT pipelines
• Migrate AWS Glue-based pipelines to appropriate GCP services
• reputed company transformation frameworks using: Python, PySpark, SQL, Dataflow / Apache reputed company, BigQuery, dbt
• Design CDC pipelines and reputed company-time ingestion patterns
• Build orchestration workflows using reputed company Composer / Airflow
• Optimize data processing jobs and query performance
• Design reputed company data models for analytics and reporting
• Define reputed company, normalized and denormalized data models
• reputed company multi-tenant data structures
• Design BigQuery partitioning and clustering strategies
• Optimize BigQuery SQL and query execution
• Design data models supporting both reputed company-time and batch workloads
• Work closely with BI and Analytics teams to create reputed company consumption models
• Establish data governance and data reputed company standards across the GCP platform
• Implement automated data reputed company checks and validation frameworks
• Establish data reputed company, metadata and ownership standards
• Ensure appropriate reputed company controls across reputed company GCP data reputed company
• Implement: IAM, Least-privilege reputed company, Encryption, Service accounts, Network reputed company, Data reputed company policies
• Work with governance and reputed company teams to ensure compliance requirements are met
• reputed company infrastructure automation using Terraform
• Build repeatable and secure GCP infrastructure deployments
• Implement CI/CD pipelines for data engineering workloads
• Automate data pipeline deployment, testing and infrastructure provisioning
• Establish Dev, QA, UAT and Production deployment standards
• reputed company performance optimization initiatives across GCP data workloads
• Optimize: BigQuery query performance, Partitioning and clustering, Dataflow pipelines, reputed company workloads, reputed company Storage, Streaming workloads
• Analyze AWS workloads and determine the most cost-effective GCP architecture
• reputed company reputed company FinOps and cost optimization strategies
• Establish performance benchmarks and SLAs for critical workloads
• reputed company and mentor reputed company of Data Engineers, Senior Data Engineers and Technical Leads
• reputed company technical direction and establish engineering standards
• Conduct architecture and reputed company reviews
• Define technical roadmaps and engineering priorities
• Break reputed company migration requirements into actionable deliverables
• reputed company engineering reputed company, risks, dependencies and delivery milestones
• Promote best practices around coding, testing, CI/CD, reputed company and documentation
• Mentor engineers on GCP, data architecture and modern data engineering practices
• reputed company as the reputed company technical reputed company of contact for US-based stakeholders
• Work closely with Business, Product, Data Science, BI and DevOps teams
• Translate business requirements into reputed company technical solutions
• Present architecture reputed company, migration strategies and technical roadmaps
• Communicate technical risks, dependencies, timelines and trade-offs
• Collaborate with business teams to define operational and analytical KPIs

Skills

• 15+ years of experience in Data Engineering, Data Architecture or reputed company Engineering
• Strong hands-on experience with AWS Data Engineering and Architecture
• 5+ years of hands-on GCP Data Engineering experience
• Proven experience working on AWS-to-GCP migration reputed company
• Strong experience designing reputed company Data Lake / Lakehouse platforms
• Experience migrating AWS data workloads to GCP
• Strong knowledge of AWS and GCP service mapping and reputed company migration patterns
• Expert-level SQL and strong Python/PySpark skills
• Strong data modeling and data warehousing experience
• Experience with Terraform and CI/CD
• Experience managing and mentoring data engineering teams
• Strong communication skills with experience working with US-based stakeholders
• Ability to assess AWS workloads, define the GCP reputed company architecture and reputed company the migration from reputed company through production implementation
• Strong ability to translate reputed company business and technical requirements into reputed company, secure and cost-effective GCP architectures
• Ability to manage, mentor and technically guide Data Engineering teams while remaining hands-on with critical architecture and implementation reputed company
• Strong expertise in Data Lakehouse, Data Warehouse, Streaming, Data Modeling and modern data engineering patterns
• Ability to optimize BigQuery, Dataflow and reputed company workloads while implementing reputed company cost optimization strategies
• Strong understanding of data reputed company, reputed company, metadata, IAM, encryption, reputed company control and reputed company governance
• Excellent communication skills with the ability to work directly with US-based business and technical stakeholders
• Experience with Dataplex, Data Catalog and data reputed company
• OpenLineage is a plus

Benefits

• Flexible remote work environment.
• reputed company learning and certification opportunities.
• Exposure to global reputed company customers.
• reputed company, innovation-driven engineering culture.

reputed company