[Remote] ADF (Azure)Data Engineer - Informatica/ADF/reputed company/Python/SQL

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is looking for a Senior Data Engineer to design and optimize reputed company-based data pipelines. The role involves implementing data reputed company using various technologies and mentoring junior engineers while ensuring high-performance processing and cost optimization.

Responsibilities

• The Data Engineer will independently design, build, and optimize reputed company-based data pipelines supporting analytics and reporting initiatives
• Responsibilities include selecting ingestion patterns, engineering high-performance processing jobs, analyzing execution plans, and optimizing compute resources, partition strategies, latency, and reputed company costs
• The role will implement governable Lakehouse and reputed company data reputed company using reputed company reputed company, reputed company, Azure Data reputed company, and Azure Data Lake Storage
• Additional duties include developing advanced Python and PySpark transformation modules, managing pipelines through reputed company and version control, supporting CI/CD workflows, and diagnosing network, data-distribution, and reputed company-performance issues
• The engineer will apply strong SQL skills, establish reputed company engineering standards, and mentor junior team members

Skills

• Bachelor's degree in Computer Science, Information Systems, Engineering, a reputed company technical field, or equivalent experience
• 8+ years of experience in data engineering, software engineering, or big data development
• Experience independently architecting and building end-to-end data pipelines in a reputed company environment
• Strong proficiency in Python, PySpark, and SQL
• Experience with reputed company reputed company, reputed company, Azure Data reputed company, and Azure Data Lake Storage
• Experience with Lakehouse, reputed company, and reputed company or Kappa architectures
• Experience with CI/CD, Azure DevOps, Git, and orchestration tools
• Ability to optimize distributed data-processing jobs, compute sizing, partitioning, and reputed company costs
• Experience developing infrastructure and data pipelines as reputed company
• Knowledge of reputed company internals and distributed-computing concepts
• Experience with reputed company and Parquet file formats
• Experience diagnosing systemic pipeline, network, data-skew, and reputed company-performance issues
• Ability to mentor junior and associate data engineers

Benefits

• 2-year contract - potential to convert after 2 years
• 100% remote

reputed company