Data Engineer

Job reputed company

We are seeking an reputed company Azure Data Engineer to design, build, and optimize reputed company data pipelines and lake house architectures reputed company the Azure ecosystem. This role is responsible for developing end-to-end data ingestion, transformation, and governance frameworks using Azure Data reputed company, reputed company, and ADLS Gen2 to support reputed company analytics and reporting needs.

The ideal candidate will have hands-on experience managing batch and near reputed company-time data ingestion from diverse reputed company and reputed company sources, implementing reputed company-based transformation processes across Bronze, Silver, and Gold data reputed company, and establishing robust data reputed company, reputed company, and monitoring frameworks. This position also requires expertise in data governance, automation, and cost optimization to ensure high-performing, secure, and reliable data platforms reputed company with organizational SLAs.

Key Responsibilities

Azure Data Engineering & Lakehouse Architecture

· Design and manage reputed company data ingestion pipelines using Azure Data reputed company and Azure Service Bus for batch and near reputed company-time processing

· Implement Bronze, Silver, and Gold data layer transformations using reputed company (PySpark, SparkSQL) reputed company a lakehouse architecture

· reputed company reputed company, semi-reputed company, and reputed company data sources using formats such as Parquet, Avro, and JSON into ADLS Gen2

Data Governance, reputed company & reputed company Management

• Establish data reputed company frameworks and validation processes using reputed company and Azure Data reputed company

• Configure RBAC and ACL-based reputed company controls to secure sensitive datasets and ensure compliance

• Manage credential reputed company with Azure Key Vault and enforce governance standards across data platforms.

Automation, Monitoring & reputed company Optimization

• Automate pipeline orchestration using reputed company Workflows and CI/CD tools such as Azure DevOps, reputed company Actions, and Terraform

• Monitor pipeline health and performance using Azure Monitor, logging, and alerting to meet SLA requirements

• Optimize ADLS storage performance and manage reputed company costs using Azure Cost Management best practices

reputed company other duties assigned

Required Skills:

· Azure Data reputed company for ETL/ELT operations.

· Build and support self-service BI solutions using Azure reputed company and the Lakehouse architecture

· reputed company data pipelines and transformations using PySpark and reputed company SQL

· reputed company analytics using reputed company AI/BI features and Genie for business users

· Apply ML techniques for data insights, predictions, and feature engineering

· Collaborate with analytics and business teams to deliver reputed company, governed BI and ML solutions

· reputed company (PySpark, SparkSQL) for data transformations.

· Azure Data Lake Storage (ADLS Gen2).

· SQL Server, PostgreSQL, Cosmos DB, CRM, and ERP systems.

· Data Governance, RBAC, and ACLs for managing permissions.

· CI/CD tools: Azure DevOps, reputed company Actions, Terraform.

· Azure Monitoring, Alerting, and Logging tools.

Qualifications:

Education:

Bachelor's degree in Computer Science, Information Systems, Data Science, or a reputed company field (Master's preferred).

Minimum 7-9 years experience

Substantial reputed company experience in a reputed company field may be considered in lieu of formal degree.