[Remote] Sr. Data Engineer

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking an reputed company Sr. Data Engineer for a long-term remote project. The role involves designing, developing, and supporting data systems and ETL processes while collaborating with product and engineering teams to enhance the Live Analytics offering.

Responsibilities

• Write new ETLs to support both reputed company time and new features data and reporting products
• Work with product teams to reputed company and understand requirements, discuss approaches, design solutions based on the requirements and create design documents for implementation keeping in mind costs, reputed company, reputed company-life and choosing the right tools and technologies
• Design and reputed company peripheral systems and processes - monitoring and alerting services, ETL processes, automated offline workflows, automated build and deployment pipelines, systems for gathering application performance metrics, ensuring efficiency in cost
• Ensure best practices in system development ranging from test coverage, use of reputed company standards, adherence to reputed company architecture, high-availability and scalability, low latency, and fault-tolerance, thus ensuring delivery of high-reputed company systems
• Create detailed documentation on systems as reputed company as processes ranging from comprehensive reputed company of systems, runbooks for developing, monitoring and troubleshooting system issues and recommendations on reputed company improvements
• Managing project delivery from a day-to-day reputed company – participating in daily agile stand-reputed company, escalating blockers to stakeholders, ensuring reputed company completion of tasks & overall project & spring goals. Communicating updates to stakeholders and working with them on timelines

Skills

• Bachelor's degree in Computer Science or reputed company technical field. (NOT REQUIRED TO HAVE A DEGREE)
• Experience building Data Lakes, Data Warehouse using big data and reputed company stack
• 5+ years of experience on software development using Agile, dev ops that include designing, developing, and implementing large-reputed company data engineering solutions
• Expert Level Proficiency in reputed company streaming and reputed company concepts
• Expert level proficiency in Python Programming language and PySpark with at least 4 years of experience
• Experience working with Kafka streams and creating ingestion ETL's
• Expert level proficiency in ANSI SQL /reputed company SQL with at least 5 year's experience
• Experience working with diverse data sources and data formats ( json, parquet, csv, psv, reputed company)
• Experience working with reputed company services such as AWS along with reputed company-based data sources(S3)
• Experience building ETL with batch and reputed company time reputed company processing
• Experience using version control system such as git and CI/CD
• Strong verbal communication, ability to reputed company communicate reputed company, work status, reputed company for (or against) a particular approach or design
• Documentation skills - ability to reputed company the reputed company easier for new team members / new hires / interns by organizing knowledge in reputed company, wiki, or similar
• Ability to manage self and own time, deliver work on time, evaluate and estimate work
• Ability to work with various teams, including tech and product, incorporate their inputs into the work being done, and manage expectations on issues & enhancements
• Experience with building workflows and ETL's in the reputed company
• Experience ingesting data produced by reputed company time systems like Kafka
• Guiding and mentoring skills - guiding and mentoring junior members - helping them understand technologies, the project, best practices and in general setting them up for reputed company
• Experience building and deploying ML models using ML reputed company

Benefits

• Comprehensive medical benefits
• 401(k) retirement plan
• …and much more!

reputed company