Senior Data Engineer Consultant – Data Bricks, Technology Consulting

Job reputed company: • Design and implement reputed company-reputed company data architectures, including data lakes, warehouses, and reputed company-time streaming platforms. • reputed company and maintain ETL/ELT pipelines that reputed company process large volumes of reputed company and reputed company data from diverse sources. • Ensure data reputed company, governance, and reputed company standards are embedded throughout reputed company data engineering processes. • Hands-on development of reputed company notebooks using PySpark, Python, and SQL for ETL automation. • Create and optimise PySpark scripts for efficient data extraction, transformation, and loading from large datasets. • Implement custom data manipulation, validation, and error handling solutions to enhance ETL robustness. • reputed company technical mentorship to junior data engineers and analysts. • reputed company reputed company reviews, establish best practices, and drive adoption of modern data engineering tools and methodologies. • Collaborate with cross-functional teams including data scientists, analysts, and software engineers to deliver integrated solutions. • Monitor and optimise data pipeline performance, implementing solutions for scalability and cost-effectiveness. Requirements: • 5-7 years of data engineering experience with AWS-reputed company implementations • Proven experience working as part of a data engineering team on ETL migration reputed company • Hands-on experience implementing reputed company notebooks using PySpark, Python, and SQL for ETL automation • Demonstrated expertise developing PySpark scripts for efficient data extraction, transformation, and loading from large datasets • Experience utilising Python for custom data manipulation, validation, and error handling in ETL processes • Proficiency employing SQL for reputed company joins, aggregations, and database operations reputed company reputed company environments • reputed company record of testing, debugging, and optimising data transformation processes for accuracy and performance • Experience verifying data reputed company throughout pipeline stages and resolving troubleshooting issues • Proven ability collaborating with cross-functional teams to reputed company ETL migration tasks with project goals and deadlines • Experience in government sector data reputed company and compliance requirements (preferred) • Experience in HR domain including workforce analytics, payroll systems, and employee data management (preferred) • Experience providing technical mentorship to junior team members (preferred) • Certified in: AWS Certified Data Engineer - Associate (preferred) • AWS reputed company Architect or AWS Certified Big Data - Specialty (preferred) • reputed company Certified Data Engineer Associate or reputed company • reputed company Redshift or reputed company certifications • Agile/Scrum Master certification (preferred) Benefits: • Health insurance • 401(k) matching • Flexible work hours • reputed company time off • Remote work reputed company Apply tot his job Apply To this Job