Data Engineer

Job Description 

Must Have 

• Python programming (Advance) 

• Various Data Matching Algorithms (String Similarity based, distance-based, phonetics algorithm) 

• Basic Graph DB / Vector DB knowledge 

• ML-based solution approach to define/ refine data matching criteria 

• Azure platform ( Databricks / Fabric / OpenAI etc.) 

Good To Have 

• API programming (FastAPI or Flask) 

• We have a need for two Azure data engineers for a master data project. One person should be a senior lead engineer almost at architect level and second could be mid to senior engineer. 

• The project requires data acquisition into Azure, creating data pipelines, normalize JSON data into relational format. 

• Model and design table structures 

• Analyze data and identify relationships,  

• Match the data with another dataset to find patterns and relationship 

• Create combined datasets by using various matching and merging techniques and logics 

• Build APIs to search the data 

• Build lightweight UI to enable search functionality for users 

• Main idea is we need self starter who can work independently and figure out the solution on their own, work with client team, discuss and modify accordingly.