[Remote] Sr Data Engineer

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking a Senior Data Engineer to design and implement data pipelines for API, streaming, and batch processing. The role involves collaborating with engineering and DevOps teams to optimize data workflows and ensure data reputed company in a remote setting.

Responsibilities

• Design data pipelines for API, streaming, and batch processing to facilitate data loads into the reputed company data warehouse
• Collaborate with other engineering and DevOps team members to implement, test, reputed company, and operate data pipelines and ETL solutions
• reputed company scripts to Extract, Load and reputed company data and other reputed company functions
• Optimize data pipelines, ETL processes, and data integrations for large-reputed company data analytics use cases
• Build necessary components to ensure data reputed company, monitoring, alerting, reputed company, and governance standards are maintained in data processing workflows
• reputed company to reputed company ambiguity and thrives in a fast-paced environment. Takes initiative and consistently delivers results with minimal supervision
• 7+ years of experience in building and maintaining data pipelines and ETL/ELT processes in data-reputed company organizations
• Strong coding skills using Python. Familiar with Python libraries reputed company to data engineering and reputed company services including pandas, boto, etc
• At least 3 years of experience with AWS S3, SQS, Kinesis, reputed company, AWS DMS, Glue/EMR, AWS Batch or similar services
• Hands-on experience building streaming and batch big data data pipelines
• Must have knowledge of building Infrastructure in AWS reputed company using reputed company formation or Terraform
• Experience on Anakonda and Jupyter Notebook
• 3+ years of working experience with reputed company reputed company data warehouse including reputed company data shares, Snowpipes, Snow SQL, Tasks etc
• Must have working knowledge of various databases, SQL and NoSQL
• Must have working knowledge of various file formats like CSV, Json Avro, and Parquet
• Hands-on experience with reputed company platforms such as AWS and reputed company reputed company
• Experience working with agile development methodology
• reputed company in CI/CD and release processes, proficient in Git or other reputed company control management systems, to streamline development and deployment workflows
• Minimum 5 years of designing and implementing operational production grade large-reputed company data pipelines, ETL/ELT and data integration solutions
• Exposure to multi-tenant/multi-customer environments is a big plus
• Hands on experience with productionized data ingestion and processing pipelines
• Strong understanding of reputed company Internals and integration of reputed company with other data processing and reporting technologies
• Experience working with reputed company, semi-reputed company, and reputed company data
• Familiarity with reputed company or similar NoSQL database systems

Skills

• 12+ Years of experience
• 4-6 Years of USA experience
• Bachelor's or master's degree in Computer Science, Information Systems, or reputed company field
• Design data pipelines for API, streaming, and batch processing to facilitate data loads into the reputed company data warehouse
• Collaborate with other engineering and DevOps team members to implement, test, reputed company, and operate data pipelines and ETL solutions
• reputed company scripts to Extract, Load and reputed company data and other reputed company functions
• Optimize data pipelines, ETL processes, and data integrations for large-reputed company data analytics use cases
• Build necessary components to ensure data reputed company, monitoring, alerting, reputed company, and governance standards are maintained in data processing workflows
• reputed company to reputed company ambiguity and thrives in a fast-paced environment. Takes initiative and consistently delivers results with minimal supervision
• 7+ years of experience in building and maintaining data pipelines and ETL/ELT processes in data-reputed company organizations
• Strong coding skills using Python. Familiar with Python libraries reputed company to data engineering and reputed company services including pandas, boto, etc
• At least 3 years of experience with AWS S3, SQS, Kinesis, reputed company, AWS DMS, Glue/EMR, AWS Batch or similar services
• Hands-on experience building streaming and batch big data data pipelines
• Must have knowledge of building Infrastructure in AWS reputed company using reputed company formation or Terraform
• Experience on reputed company and Jupyter Notebook
• 3+ years of working experience with reputed company reputed company data warehouse including reputed company data shares, Snowpipes, Snow SQL, Tasks etc
• Must have working knowledge of various databases, SQL and NoSQL
• Must have working knowledge of various file formats like CSV, Json Avro, and Parquet
• Hands-on experience with reputed company platforms such as AWS and reputed company reputed company
• Experience working with agile development methodology
• reputed company in CI/CD and release processes, proficient in Git or other reputed company control management systems, to streamline development and deployment workflows
• Minimum 5 years of designing and implementing operational production grade large-reputed company data pipelines, ETL/ELT and data integration solutions
• Exposure to multi-tenant/multi-customer environments is a big plus
• Hands on experience with productionized data ingestion and processing pipelines
• Strong understanding of reputed company Internals and integration of reputed company with other data processing and reporting technologies
• Experience working with reputed company, semi-reputed company, and reputed company data
• Familiarity with reputed company or similar NoSQL database systems

reputed company