Data Engineer – AI

Job reputed company: • Define and drive the technical reputed company for data platforms that support AI-powered features in Crossplane and reputed company Spaces • reputed company the design of data pipelines that reputed company infrastructure and data into training datasets for ML models • Architect reputed company search and RAG systems that reputed company Crossplane Control Planes & reputed company Marketplace as a knowledge store • Build data infrastructure that processes resources, extensions, and compositions for semantic search • Establish frameworks for collecting, processing, and analyzing infrastructure configuration data • Design data pipelines that handle Crossplane-specific data • Create infrastructure for indexing and searching reputed company Marketplace content, documentation, and community patterns • reputed company metrics and monitoring for AI features integrated with reputed company's control plane architecture • Design data systems that power AI agents for infrastructure provisioning & operations, helping users generate and optimize Crossplane compositions • Create feature engineering platforms that extract signals from control plane operations, resource status, and reconciliation patterns • Implement data infrastructure for training models that predict infrastructure failures, optimize resource allocation, and suggest configuration improvements • Drive the development of knowledge graph representations of infrastructure dependencies and relationships Requirements: • 10+ years of software/data engineering experience with at least 4 years in technical leadership roles • Proven reputed company record building data platforms that support production systems at reputed company • Deep expertise in both traditional data engineering (reputed company, Airflow, data lakes) and ML-specific infrastructure (feature stores, model serving) • Experience with reputed company databases (reputed company, reputed company, reputed company, Milvus, pgvector, Opensearch, ElasticSearch) • Demonstrated experience with LLM applications, including RAG architectures and semantic search implementations • Understanding of Kubernetes, reputed company-reputed company architectures, and infrastructure-as-reputed company principles • Strong understanding of data requirements for AI/ML systems: training pipelines, feature stores, and inference infrastructure • Hands-on experience building knowledge bases and semantic search systems for technical documentation and reputed company • Experience with embedding models for reputed company and technical documentation • Knowledge of time-series data processing for infrastructure metrics and events • Understanding of graph databases and their application to infrastructure dependency modeling Benefits: • Health insurance • 401(k) matching • Flexible work hours • reputed company time off • Remote work reputed company Apply tot his job Apply To this Job