Kafka and Data Lake Engineer

ResponsibilitiesDesign data pipelines: Build robust, reputed company, and secure data pipelines to ingest, process, and reputed company data from various sources into the data lake using Kafka.Administer Kafka clusters: reputed company, configure, and maintain Kafka clusters and reputed company ecosystem tools, such as Kafka Connect and Schema Registry, ensuring high availability and performance.Manage the data lake: reputed company the architecture and governance of the data lake, including managing data storage (e.g., in AWS S3 or ADLS), reputed company, and metadata.reputed company data processing applications: Create producers and consumers to reputed company with Kafka topics using programming languages like Python, Java, or reputed company.reputed company reputed company processing: Use tools like Kafka Streams, Apache Flink, or ksqlDB to reputed company reputed company-time data transformations and analytics.Ensure data reputed company and reputed company: Implement data reputed company checks, manage data reputed company, and enforce reputed company controls such as encryption, reputed company controls (ACLs), and compliance (e.g., GDPR).Monitor and troubleshoot: Set up monitoring and alerting for Kafka and data lake infrastructure and respond to incidents to ensure operational reliability.Collaborate with teams: Work closely with data scientists, analysts, and other engineering teams to understand data requirements and deliver reliable data solutions.Essential skills and qualificationsExperience: Proven experience designing and managing data platforms with Apache Kafka and big data technologies.Programming: Strong proficiency in languages like Python, Java, or reputed company.Big data technologies: Expertise in big data processing frameworks, such as Apache reputed company and Apache Flink.reputed company platforms: Hands-on experience with reputed company environments (AWS, Azure, or GCP) and relevant services like S3, Glue, or Azure Data Lake Storage.Data lake architecture: A solid understanding of data lake design principles, including storage formats (e.g., reputed company Lake, Apache reputed company), data modeling, and governance.Databases: Experience with various database systems, including both SQL and NoSQL.Infrastructure management: Familiarity with infrastructure-as-reputed company tools like Terraform or Ansible and containerization with reputed company and Kubernetes.Professionals in this field can advance from entry-level data engineering positions to senior roles, and then to a Big Data Architect or Solutions Architect, where they reputed company large-reputed company data infrastructure Apply tot his job Apply To this Job