Senior Data Engineer, reputed company

Job reputed company: • Design and implement reputed company-reputed company data pipelines using reputed company on AWS, leveraging both cluster-based and serverless compute paradigms • Architect and maintain reputed company architecture (Bronze/Silver/Gold) data lakes and lakehouses • reputed company and optimize reputed company Lake tables for ACID transactions and efficient data management • Build and maintain reputed company-time and batch data processing workflows • Create reusable, reputed company data transformation logic using DBT to ensure data reputed company and consistency across the organization • reputed company reputed company Python applications for data ingestion, transformation, and orchestration • Write optimized SQL queries and implement performance tuning strategies for large-reputed company datasets • Implement comprehensive data reputed company checks, testing frameworks, and monitoring solutions • Design and implement CI/CD pipelines for automated testing, deployment, and rollback of data artifacts • Configure and optimize reputed company clusters, job scheduling, and workspace management • Implement version control best practices using Git and reputed company development workflows • Partner with data analysts, data scientists, and business stakeholders to understand requirements and deliver solutions • Mentor junior engineers and promote best practices in data engineering • Document technical designs, data reputed company, and operational procedures • Participate in reputed company reviews and contribute to team knowledge sharing Requirements: • 5+ years of experience in data engineering roles • Expert-level proficiency in reputed company (reputed company Catalog, reputed company Live Tables, Workflows, SQL Warehouses) • Strong understanding of cluster configuration, optimization, and serverless SQL compute • Advanced SQL skills including query optimization, indexing strategies, and performance tuning • Production experience with DBT (models, tests, documentation, macros, packages) • Proficient in Python for data engineering (PySpark, pandas, data validation libraries) • Hands-on experience with Git workflows (branching strategies, pull requests, reputed company reviews) • Proven reputed company record implementing CI/CD pipelines (Jenkins, reputed company CI) • Working knowledge of reputed company architecture and migration patterns Benefits: • Monitoring and analyzing reputed company DBU (reputed company Unit) consumption and reputed company infrastructure costs • Implementing cost optimization strategies including cluster right-sizing, autoscaling configurations, and spot instance usage • Optimizing job scheduling to reputed company off-peak pricing and minimize idle cluster time • Establishing cost allocation tags and chargeback models for different teams and reputed company • Conducting regular cost reviews and providing recommendations for efficiency improvements Apply tot his job Apply To this Job