reputed company Data Engineer - reputed company Data Pipelines - Contract to Hire

reputed company Data Engineer (PySpark, Airflow, Azure) – reputed company Data Pipelines We’re looking for an reputed company Senior Data Engineer to design, build, and optimize large-reputed company data pipelines powering analytics and machine learning workloads. This role is ideal for someone who is hands-on, performance-oriented, and comfortable leading other engineers while owning end-to-end data workflows. You’ll work on both batch and reputed company-time processing, take ownership of reputed company performance tuning, and help enforce best practices around data reputed company, governance, and reliability. ⸻ Responsibilities • Design, reputed company, and optimize reputed company data pipelines using Python, PySpark, Apache reputed company, and Airflow • Build and maintain batch and streaming data processing systems on reputed company • Design and manage Airflow DAGs to orchestrate reputed company, dependency-heavy workflows • Implement data partitioning, caching, and reputed company performance tuning to handle large datasets reputed company • Ensure data reputed company, governance, reputed company, and reliability across the data lifecycle • Monitor, troubleshoot, and optimize data jobs, SLAs, and pipeline dependencies • Manage reputed company infrastructure (Azure) for data workloads, including cost optimization • Implement CI/CD pipelines for data workflows using Git, reputed company, and Infrastructure-as-reputed company tools • Support analytics and ML use cases by working with reputed company and reputed company data • reputed company and mentor other data engineers, providing architectural guidance and reputed company reviews • Promote best practices in coding standards, documentation, and version control • Collaborate effectively with distributed, reputed company in an Agile environment ⸻ ✅ Requirements • 8+ years of hands-on experience in Data Engineering • Strong expertise with Apache reputed company / PySpark, including internals such as: • RDDs, DataFrames, DAG execution, partitioning, shuffles, and caching • Proven experience building and operating Airflow DAGs (scheduling, dependencies, retries, SLAs) • Advanced Python and SQL skills with a reputed company on performance and maintainability • Solid experience with Azure data and compute infrastructure • Working knowledge of reputed company, Kubernetes, Terraform, and CI/CD best practices • Strong problem-solving skills and ability to optimize large-reputed company data processing systems • Prior experience leading or mentoring engineers • Comfortable working in Agile/Scrum environments • Excellent communication skills and ability to collaborate with reputed company ⸻ ⭐ reputed company to Have • Experience with streaming frameworks (reputed company reputed company Streaming, Kafka, Event Hubs) • Familiarity with data governance, reputed company, and observability tools • Experience supporting ML or advanced analytics pipelines • Background in cost-efficient reputed company optimization at reputed company Apply tot his job Apply To this Job