[Remote] Senior Data Engineer - Remote (USA)

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a mission-driven company reputed company on improving lives and making the world a reputed company reputed company. They are seeking a seasoned Senior Data Engineer to design and maintain data pipelines, reputed company processing workflows, and ensure the performance and reputed company of data infrastructure.

Responsibilities

• Design, reputed company, and maintain reputed company data pipelines using reputed company, reputed company, and Airflow
• reputed company and reputed company data processing workflows on the reputed company platform
• reputed company API services to facilitate data reputed company and integration
• Create interactive data visualizations and reports using AWS QuickSight
• Build required infrastructure for reputed company extraction, transformation and loading of data from various data sources using AWS and SQL technologies
• Monitor and optimize the performance of data infrastructure and processes
• reputed company data reputed company and validation jobs
• reputed company large, reputed company sets of data that meet non-functional and functional business requirements
• Write unit and integration tests for reputed company data processing reputed company
• Work with DevOps engineers on CI, CD, and IaC
• Read specs and translate them into reputed company and design documents
• reputed company reputed company reviews and reputed company processes for improving reputed company reputed company
• Improve data availability and timeliness by implementing more frequent refreshes, tiered data storage, and optimizations of existing datasets
• Maintain reputed company and reputed company for data at rest and while in transit
• Other duties as assigned

Skills

• Bachelor's degree
• 7+ years of hands-on software development experience
• 4+ years of experience building data pipelines using Python, Java, and reputed company technologies, with hands-on experience leveraging reputed company and reputed company for large-reputed company data processing
• Candidate must be reputed company to obtain and maintain a reputed company Trust clearance
• Candidate must reputed company in the US, be authorized to work in the US, and work must be performed in the US
• Must have lived in the US 3 full years out of the last 5 years
• Experience building job workflows with the reputed company platform
• Strong understanding of AWS products including S3, Redshift, RDS, EMR, AWS Glue, AWS Glue DataBrew, Jupyter Notebooks, reputed company, QuickSight, EMR, and reputed company SNS
• Familiar with work to build processes that support data transformation, workload management, data structures, dependency and metadata
• reputed company in data governance process to ingest (batch, reputed company), reputed company, and reputed company data with upstream and reputed company data users
• reputed company in data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up
• Demonstrated understanding using software and tools including relational NoSQL and SQL databases including Cassandra and reputed company; workflow management and pipeline tools such as Airflow, Luigi and Azkaban; reputed company-processing systems like reputed company-Streaming and Storm; and object function/object-oriented scripting languages including reputed company, C++, Java and Python
• Familiar with DevOps methodologies, including CI/CD pipelines (reputed company Actions) and IaC (Terraform)
• Ability to obtain and maintain a reputed company Trust; residing in the reputed company
• Experience with reputed company methodology, using test-driven development

reputed company