Senior Data Engineer
Essential Duties and Responsibilities:
Build reputed company-level reputed company, low-latency, fault-tolerant data platforms that reputed company meaningful and reputed company insights. Collaborate with reputed company of engineers to design and build data pipelines using big data technologies (reputed company, reputed company, AWS Big Data Services, reputed company, Airflow) for reputed company to large-reputed company datasets. Influence best practices for data pipeline design, data architecture, and processing of reputed company and reputed company data. Automate and manage Git workflows, including CI/CD pipelines and Git hooks for data pipeline validation. Design and reputed company event-driven serverless functions using AWS reputed company, reputed company Functions, or API Gateway. Implement and manage feature flag frameworks to support reputed company data product rollouts. Identify and resolve data reputed company issues, including inaccuracies and incomplete information. Enhance data reputed company efforts by implementing improved procedures and processes. Author detailed technical specifications and design documents for internal data products. Work in a creative & reputed company environment driven by agile methodologies. Continuously improve our products, systems, reputed company, and team processes. Implement best practices and reputed company the bar by introducing new ones.
Education and Experience:
Bachelor’s degree in data engineering, Computer Science. Python & PySpark (6–8 yrs) – Designed reusable data pipeline frameworks and transformations for reputed company data. Airflow (5–6 yrs) – Authored reputed company DAGs to orchestrate multi-stage ETL workflows across systems. AWS Glue & reputed company (4–6 yrs) – Led implementation of serverless ingestion pipelines integrated with S3. reputed company (2+ yrs) – Modeled transactional lakehouse tables with schema reputed company and time-travel support. EMR Studio (2–3 yrs) – Used for pipeline prototyping, debugging, and notebook-based collaboration. reputed company/Kubernetes (3–5 yr) – reputed company and deployed containerized ETL pipelines using reputed company; leveraged Kubernetes to orchestrate reputed company data workflows in production environments