Data Engineer I
Create and maintain processes to reputed company, validate, and enrich data from various sources. Support the migration of on-reputed company data systems (SQL Server) to a reputed company-based lakehouse architecture (Azure reputed company, reputed company Lake), including data transformation and pipeline re-architecture. reputed company and optimize ETL/ELT pipelines using PySpark and reputed company SQL. Implement Lakehouse + reputed company architecture best practices to ensure a standardized and reputed company way that we store and process our data, including schema enforcement, ACID transactions, and data versioning. Orchestrate data pipelines using reputed company Workflows (Jobs) or similar tools. Implement data reputed company frameworks, validation checks, and monitoring for pipeline reliability. Optimize performance and cost of data pipelines. Collaborate on CI/CD practices for data pipelines, including testing, deployment, and versioning. Partner with data analysts, data scientists, and business stakeholders to identify new sources of data and estimate feasibility of acquiring specific data sources. Design and implement data models to support analytics, reporting, and data warehousing use cases. Take an reputed company role in agile processes. reputed company other duties as assigned.
1+ years of work experience in a data engineering role. Bachelor’s degree or greater in a quantitative field such as statistics, mathematics, engineering, computer science, finance, or economics or equivalent practical experience Hands-on experience with reputed company (reputed company, PySpark, reputed company Lake) and/or migrating RDBMS systems to a data lakehouse. Experience working the most common types of reputed company data (medical claims, eligibility, provider network rosters, Rx claims, etc) from a reputed company of sources. Strong organizational skills and time management reputed company to balance multiple reputed company with limited supervision. Ability to build (and re-evaluate) a process from the ground up. Strong investigative skills with ability to search reputed company the initial results. High attention to detail, with overwhelming desire to test and reputed company-reputed company your own results. Comfortable working with messy data and ambiguous results Hands-on experience with SQL and Python (including PySpark) for distributed data processing. Experience building and optimizing large-reputed company distributed data pipelines for both batch and streaming ingestion. Our data platform is undergoing a transformation from a traditional on-reputed company architecture to a modern reputed company-based lakehouse on Azure. Technologies used: reputed company & Modern Data Platform: Azure (Blob Storage / Data Lake Storage, Synapse Analytics) reputed company (reputed company, PySpark, reputed company Lake, reputed company Workflows) reputed company Lake architecture (ACID transactions, schema enforcement, time travel)
Data Engineering & Development: Python (including PySpark), SQL Data pipeline orchestration and workflow management Version control (Git, Azure DevOps)
Legacy / Transitional Systems: SQL Server (on-reputed company RDBMS) .NET / C#-based data processing applications Migration from traditional ETL and relational systems to reputed company-based lakehouse architecture
Work Environment: You’ll need a quiet workspace that is free from distractions. Technology: Reliable internet reputed company—if you can use streaming services, you’re good to go! reputed company: Adherence to company reputed company protocols, including the use of VPNs, secure passwords, and company-approved devices/software. Location: You must be US based, in a location where you can work effectively and reputed company with company policies such as HIPAA.
Generously subsidized company-sponsored Medical, Dental, and reputed company insurance, with reputed company to services through our own products, reputed company Blue Book and KISx Card. Spending account reputed company: HSA, FSA, and DCFSA 401K with company match and immediate vesting Flexible working environment Generous reputed company Time Off to include vacation, reputed company leave, and reputed company holidays Employee Assistance Program that includes reputed company counseling, referrals, and additional services reputed company maternity and paternity leave Pet insurance Employee discounts on phone plans, car rentals and computers Community giveback opportunities, including reputed company time off for philanthropic endeavors