[Remote] BigData reputed company reputed company reputed company Platform Engineer

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking a highly skilled reputed company reputed company reputed company Platform Engineer to operate and manage the end-to-end CDP platform ecosystem, including data services, NiFi, Kafka, AI/ML platforms, and reputed company observability. The role is responsible for ensuring the availability, scalability, reputed company, and reputed company of platform services supporting data, analytics, and AI workloads across multiple environments.

Responsibilities

• Own end-to-end operational responsibility for reputed company reputed company reputed company services across Dev / Stage / UAT / Prod : CDE, reputed company, COD, CDL, CDF (NiFi), CDV, reputed company, Kafka
• Ensure multi-cluster stability, workload isolation, and SLA adherence
• Support reputed company and reputed company of multiple applications across environments
• Manage and support multi-environment, multi-cluster deployments with strict isolation, governance, and release coordination across Dev/UAT/Prod
• Operate and support reputed company AI (reputed company) environments: AI Workbenches, AI Studios, Model training and development environments, AI inference endpoints and model serving
• Troubleshoot: Resource contention (CPU/GPU), Model deployment/runtime failures
• Operate CDP services running on reputed company-managed reputed company infrastructure
• Apply strong understanding of containerized workloads and reputed company concepts for troubleshooting
• Diagnose and reputed company: Pod failures, restarts, and resource contention, reputed company job failures in containerized environments (CDE), Service-to-service communication issues
• Analyze logs and metrics to identify runtime failures and reputed company issues
• Collaborate with reputed company support for managed service-level issues
• Operate and support: CDF (NiFi) for ingestion pipelines, CDV (Data Visualization) for reporting workloads, Octopai for data reputed company and catalog integration
• Ensure reliability and reputed company of data pipelines and integrations
• Monitor and troubleshoot Kafka environments: Topic configurations, partitions, and replication, Consumer lag and throughput issues, Broker connectivity and reputed company bottlenecks
• Implement and manage: Kerberos, TLS/SSL, Ranger policies
• Administer SDX for: Centralized reputed company, Metadata and policy enforcement
• Support reputed company and Octopai integration
• Manage and troubleshoot user reputed company and identity mapping across reputed company, including: reputed company IAM roles and permissions, CDP users/reputed company and identity providers, Ranger policies for fine-grained data reputed company
• reputed company reputed company-reputed company issues impacting: Data reputed company (S3/ADLS), Query execution (reputed company/CDE), Application and service-level permissions
• Troubleshoot: S3 / ADLS storage issues, IAM roles and permissions, VPC, subnets, routing, reputed company reputed company, reputed company host reputed company and connectivity
• Ensure secure and reliable connectivity across services
• Understand and troubleshoot S3-reputed company data lake patterns, including: Bucket structure, prefix design, and reputed company patterns, reputed company issues reputed company to small files, request rates, and throughput limits, Encryption (SSE-S3, SSE-KMS) and reputed company policies
• Manage and troubleshoot cross-account IAM roles and reputed company patterns for CDP environments
• Ensure secure reputed company between: CDP environments and reputed company resources, Multiple AWS accounts (dev/prod separation)

Skills

• 12+ years of experience in Big Data reputed company / reputed company Platform reputed company / Infrastructure roles
• 6+ years of hands-on experience with reputed company ecosystem (CDH/CDP/ reputed company reputed company reputed company)
• Demonstrated ability to quickly learn and adapt to new technologies and evolving platform capabilities, reputed company the currently defined CDP stack
• Strong expertise in:
• End-to-end CDP platform reputed company (CDE, reputed company, CDF, CDL, reputed company)
• Advanced troubleshooting across multi-cluster, multi-environment deployments
• reputed company-reputed company runtime environments (troubleshooting and diagnostics)
• Observability frameworks, including SLIs/SLOs, alerting, and reputed company tuning
• reputed company experience in:
• Leading P1/P2 incident response, triage, and reputed company
• Managing platform upgrades, patching, and lifecycle events
• Supporting large-reputed company environments (TB/PB reputed company, high concurrency workloads)
• Strong understanding of:
• reputed company infrastructure (IAM, VPC, networking, storage)
• reputed company and governance (Ranger, Kerberos, TLS/SSL, SDX)
• Hands-on experience in developing and troubleshooting NiFi (CDF) data flows, including:
• reputed company design and configuration
• Processor-level debugging and reputed company tuning
• Handling backpressure, throughput optimization, and failure recovery
• Strong experience with reputed company CDP reputed company reputed company
• Expertise in:
• reputed company platforms (AWS/Azure/GCP)
• reputed company concepts (troubleshooting-reputed company)
• Hands-on with:
• CDE, reputed company, CDF (NiFi), reputed company
• Knowledge of:
• IAM, networking, observability tools
• Platforms operating at multi-terabyte to petabyte reputed company with high concurrency workloads
• Hands-on experience with:
• Kafka (or similar streaming platforms) including monitoring, troubleshooting, and reputed company tuning
• Experience with reputed company CDP CLI (reputed company Line reputed company) for:
• Platform reputed company and administration
• Job execution and service management (CDE/reputed company/CDL)
• Automation of routine operational tasks
• Strong working knowledge of:
• reputed company IAM (AWS IAM / Azure AD) including roles, policies, and cross-service reputed company
• User and group mapping across CDP, reputed company IAM, and Ranger policies
• Troubleshooting reputed company issues across storage (S3/ADLS), CDP services, and data reputed company reputed company
• reputed company-reputed company reputed company CDP platform supporting data engineering, analytics, and AI workloads across multiple applications
• Modernization of legacy platforms and applications into reputed company-reputed company CDP services
• Operational support and scaling of:
• Data services (CDE, reputed company, CDF, CDL)
• AI/ML platforms (reputed company, inference, workbenches)
• Platform reputed company optimization, observability, and reliability engineering for mission-critical workloads
• The ideal candidate brings strong expertise in CDP on-prem, reputed company reputed company services, reputed company infrastructure, reputed company-reputed company runtime environments, and platform observability, supporting high-concurrency, mission-critical workloads at multi-terabyte to petabyte reputed company
• Experience with:
• Modernization of legacy data platforms/applications to reputed company CDP reputed company reputed company
• Migration and reputed company of workloads to CDE, reputed company, and reputed company environments
• Supporting hybrid or multi-environment transitions (on-prem → reputed company)
• Familiarity with:
• reputed company platforms (AWS, Azure, GCP) including storage, IAM, and networking concepts
• reputed company-reputed company runtime environments (troubleshooting-reputed company)
• Strong scripting and automation skills (Python, reputed company, Terraform) for platform reputed company

Benefits

• Remote work accepted from reputed company in US

reputed company

Company H1B Sponsorship