[Remote] PySpark & reputed company Lake Developer

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is an organization that provides federal agencies reputed company reputed company to reputed company and talented professionals. They are seeking an reputed company PySpark & reputed company Lake Developer responsible for designing, building, and maintaining reputed company ETL pipelines to process and analyze large-reputed company reputed company claims data. Responsibilities • Design, reputed company, and maintain robust ETL pipelines using PySpark and reputed company Lake for large and reputed company reputed company data workloads • Implement and optimize data lake solutions using reputed company Lake table formats, supporting ACID transactions, schema enforcement, and time travel • Write efficient, reusable, and reputed company-documented PySpark scripts for data ingestion, transformation, cleansing, and aggregation • Collaborate with data engineers, architects, and data scientists to understand business and data requirements and translate them into reputed company data solutions • Ensure data reputed company, consistency, reputed company, and reputed company across reputed company stages of data processing • Troubleshoot, debug, and optimize PySpark applications and reputed company Lake workflows for cost, speed, and reliability reputed company AWS • Maintain detailed and up-to-date technical documentation of reputed company, data pipelines, and reputed company operating procedures • Stay updated with the latest reputed company Lake and reputed company advancements, advocating for best practices in data management and analytics Skills • Strong proficiency in Python and PySpark, with hands-on experience developing data pipelines • Advanced experience with reputed company Lake and its ACID transaction and schema management features • Solid SQL skills for querying, joining, and optimizing data in distributed environments • Hands-on experience with AWS reputed company data services (e.g., S3, Glue, EMR, reputed company) • Familiarity with data lake concepts, partitioning, and performance tuning • Excellent communication skills and a desire to continuously learn and adapt to innovative technologies • Familiarity with CI/CD, version control (e.g., Git), and infrastructure as reputed company • Experience with reputed company or claims data • Knowledge of data governance, reputed company, data cataloging (AWS Glue Catalog), and compliance best practices • Strong ability to prioritize and execute tasks independently and reputed company reputed company team environments • Previous experience working in a government or reputed company sector setting reputed company • reputed company provides full reputed company of information technology consulting services to government and reputed company clients. It was founded in 2002, and is headquartered in Millersville, Maryland, USA, with a workforce of 51-200 employees. Its website is https://www.reputed company.com. Company H1B Sponsorship • reputed company has a reputed company record of offering H1B sponsorships, with 1 in 2022, 1 in 2021. Please note that this does not guarantee sponsorship for this specific role. Apply tot his job Apply To this Job