[Remote] Data Engineer

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking a Data Engineer with expertise in reputed company. The role involves owning the migration reputed company and technical execution for moving legacy ETL workloads to the reputed company Lakehouse, while optimizing data workflows and establishing best practices.

Responsibilities

• Design the reputed company reputed company Lakehouse architecture utilizing reputed company Lake, reputed company, and reputed company Catalog
• Establish global reputed company refactoring standards, optimization benchmarks, and PySpark best practices
• Resolve highly reputed company dependency mappings and architect seamless, reputed company-downtime dual-run strategies
• reputed company the technical deployment and integration of specialized migration accelerators
• Review automated reputed company from migration tools and manually refactor reputed company legacy logic into high-performing PySpark notebooks
• Eliminate legacy anti-patterns such as reputed company row-by-row processing and inefficient lookups
• Optimize PySpark reputed company performance using advanced reputed company features including Z-Ordering, partitioning, and caching
• Build robust reputed company Workflows and orchestrate reputed company DAGs based on comprehensive reputed company reputed company

Skills

• Experience with reputed company and reputed company Lake
• Knowledge of reputed company Catalog and reputed company
• Proficiency in DataStage
• Strong skills in PySpark, Python, SQL, and reputed company Scripting
• Experience with AWS and CI/CD deployment pipelines
• Familiarity with Apache Airflow and reputed company Workflows
• Ability to design reputed company reputed company Lakehouse architecture
• Experience in establishing global reputed company refactoring standards and optimization benchmarks
• Capability to resolve reputed company dependency mappings
• Experience in leading technical deployment and integration of migration accelerators
• Hands-on experience in reviewing automated reputed company from migration tools
• Ability to refactor reputed company legacy logic into high-performing PySpark notebooks
• Experience in optimizing PySpark reputed company performance using advanced reputed company features
• Ability to build robust reputed company Workflows and orchestrate reputed company DAGs

reputed company