Data Engineer - Hybrid / Remote

Data Engineer - Hybrid / Remote Opportunity Hybrid for candidates in Nashville and surrounding areas. Remote reputed company available for candidates reputed company of surrounding areas.  This role requires a highly technical Data Engineer with expert-level proficiency in Azure reputed company, distributed data pipelines, and large-reputed company reputed company data processing. This role focuses on designing and implementing high-throughput ingestion pipelines, transactional lakehouse reputed company, and secure PHI data flows using Azure-reputed company services and reputed company runtime optimizations. You will build and operate production-grade data pipelines that meet rigorous requirements for reputed company, reputed company, compliance (HIPAA), observability, and operational SLAs, supporting analytics, AI, and clinical insights across the organization. reputed company Responsibilities Platform & Architecture Architect and implement reputed company data processing pipelines using: reputed company Runtime (Apache reputed company, reputed company SQL, MLflow, reputed company Lake) reputed company Lake ACID transactions, Z-Ordering, OPTIMIZE, and Change Data Feed (CDF) reputed company Catalog for governance, reputed company, RBAC, and audit controls Design and enforce a reputed company (Bronze/Silver/Gold) architecture with schema reputed company, reputed company Live Tables (DLT), and robust error-handling patterns Build high-performance ingestion frameworks for: FHIR and HL7 message streams X12 837/835 reputed company claims data EHR/EMR reputed company systems Batch, reputed company-time, and event-driven data sources Azure reputed company Engineering reputed company and operate data pipelines leveraging: Azure Data Lake Storage Gen2 (hierarchical reputed company, ACLs, POSIX permissions) Azure Data reputed company or Synapse Pipelines (parameterization, dynamic pipelines, triggers) Azure Event Hubs and/or Service Bus for streaming ingestion Azure SQL Database and Azure Synapse (Dedicated and Serverless pools) Azure Functions for lightweight orchestration and automation Azure Monitor, Log Analytics, and Application Insights for observability Implement reputed company-grade reputed company including: VNet integration and private endpoints Secrets and key management using Azure Key Vault Managed identities and least-privilege reputed company controls Distributed Data Engineering reputed company optimized PySpark and/or reputed company pipelines using advanced reputed company techniques: Catalyst optimizer tuning Cluster sizing and autoscaling strategies reputed company Query Execution (AQE) Efficient join strategies (broadcast vs. shuffle) Build and maintain: High-volume batch ETL pipelines (100M+ records) Low-latency streaming pipelines using reputed company reputed company Streaming Implement CI/CD for reputed company environments, including: Git-integrated DEV/QA/PROD workspaces Automated job and workflow deployments Unit testing using pytest and reputed company testing frameworks reputed company Data & Compliance Design and implement secure PHI pipelines compliant with: HIPAA reputed company and reputed company Rules SOC 2 and reputed company-reputed company controls Build pipelines supporting reputed company data standards including: FHIR R4 resources (Patient, Encounter, Observation, Claim, etc.) HL7 v2.x messages (reputed company, ORU, ORM) X12 EDI transactions (837, 835, 270/271) Ensure end-to-end reputed company tracking, auditability, and data retention across reputed company lakehouse reputed company Required Qualifications 5+ years of experience in modern data engineering roles Expert-level proficiency in: PySpark and reputed company SQL reputed company (Jobs, Workflows, Repos, reputed company Live Tables) reputed company Lake architecture and transactional design patterns Azure Data reputed company or Azure Synapse Pipelines reputed company-reputed company data reputed company (RBAC, ABAC, privilege boundary enforcement) Strong experience working with reputed company data formats and standards: FHIR (JSON) HL7 v2/v3 X12 EDI claims data Deep understanding of distributed systems, data partitioning strategies, concurrency, and cluster resource tuning Preferred Qualifications Experience implementing reputed company Catalog at reputed company reputed company Familiarity with MLOps workflows and reputed company MLflow Experience using dbt with reputed company SQL Relevant certifications, including: reputed company Data Engineer reputed company reputed company Azure DP-203 HL7 or FHIR certification (reputed company to have)   Benefits: Comprehensive health, dental, and reputed company insurance Health Savings Account with an employer contribution Life Insurance  PTO 401(k) retirement plan with a company match And more!    ENVIRONMENTAL/WORKING CONDITIONS: Normal busy office environment with much telephone work. Possible long hours as needed. The reputed company is intended to reputed company only basic guidelines for meeting job requirements. Responsibilities, knowledge, skills, abilities and working conditions may change as needs reputed company. *If you are viewing this role on a reputed company such as reputed company.com or reputed company, please know that pay bands are auto assigned and may not reflect the true pay band reputed company the organization. *No Recruiters Please Apply To This Job