Senior Big Data Engineer – Distributed Data Processing, Hadoop & Spark Platform Development

Join arenaflex as a Senior Big Data Engineer

Are you ready to be part of a transformative technology team that's reshaping how enterprises harness the power of data? At arenaflex, we're seeking a talented and passionate Senior Big Data Engineer to help us engineer, code, and ship programming that forms an essential part of our clients' digital lives. This is a remarkable opportunity to work alongside skilled architects in an open, supportive, and inclusive environment where your voice is valued, and you have the autonomy to make decisions about which technologies to use to solve challenging problems.

At arenaflex, we offer a range of opportunities to work with the latest and most innovative technologies. We encourage our engineers to contribute to the broader engineering community through open source initiatives. Understanding the importance of keeping your skills fresh and relevant, we provide dedicated time and resources for you to invest in your professional development. Find your place in technology and become part of #TeamArenaFlex.

About the Role

As a Senior Big Data Engineer at arenaflex, you will be a member of a product engineering or delivery and integration team reporting to an Engineering Director. You will play a critical role in designing, developing, and maintaining our big data infrastructure and analytics platforms. This position offers you the chance to lead a small team of talented developers and inspire them to achieve their full potential while working on cutting-edge technologies that process massive volumes of data.

You will be part of a fast-paced, entrepreneurial team that enables batch/real-time analytical solutions using groundbreaking technologies including Hadoop/Big Data, Data Science/Machine Learning, Python, SQL, UNIX, and Shell Scripting. Your work will deliver innovative solutions across multiple lines of business, making a significant impact on arenaflex's technological advancement and customer experience.

What You Will Do in This Role

As a key member of our engineering team, you will be responsible for the following:

• Software Development: Utilize intermediate to advanced level software development skills to contribute to programming efforts through consistent improvement practices, including tools, common components, and comprehensive documentation.
• Code Quality: Participate in code reviews and automated testing to ensure the highest quality standards. Troubleshoot critical software components and identify code defects for remediation.
• Deployment & Operations: Enable the deployment, support, and monitoring of software across test, integration, and production environments. Automate deployments in test or production environments and scale applications automatically based on demand projections.
• Technical Leadership: Lead a small team of developers and motivate them to achieve their highest potential. Influence team members with innovative changes and enhancements by challenging the norm and demonstrating thoughtful risk-taking.
• Agile Team Collaboration: Serve as a core member of an Agile team, driving customer story analysis and elaboration, design and development of software applications, testing, and build automation tools.
• Platform Management: Manage a specific platform/product or be part of a dynamic resource pool assigned to projects based on demand and business need.
• Innovation: Identify opportunities to adopt innovative technologies and embody an entrepreneurial mindset by looking beyond the obvious for continuous improvement opportunities.
• Code Development: Focus on writing code, API specifications, conducting code reviews, and testing in continuous runs or performing verification of concepts and automation tools.
• Data Pipeline Development: Design and develop advanced data pipelines for both batch and real-time data processing using Big Data technologies.
• Technical Documentation: Develop and document technical and functional specifications while analyzing software and system processing flows.

Essential Qualifications

To be successful in this role, you should have:

• A Bachelor's degree in Engineering, Computer Science, or a related field; OR a Master's degree in Computer Applications or equivalent practical experience.
• 3+ years of software development experience with demonstrated leadership of developers and Scrum teams.
• 3+ years of hands-on experience working with MapReduce, Hive, and Spark (core, SQL, and PySpark).
• Strong Data warehousing concepts and practical application.
• Knowledge of financial reporting environments is a plus.
• Mastery of distributed environments and their architectural considerations.
• Hands-on programming experience using Core Java or Python/Scala.
• Expertise in Hadoop and Spark Architecture and its working principles.
• Hands-on experience writing and understanding complex SQL (Hive/PySpark DataFrames), optimizing joins while processing large amounts of data.
• Experience in UNIX shell scripting for automation and data processing.
• Ability to design and develop advanced data pipelines for batch and real-time data processing.
• Demonstrated experience in analysis, design, development, testing, and implementation of enterprise applications.
• Exhibited ability to develop and document technical and functional specifications while analyzing software and system processing flows.
• Excellent technical and analytical skills with a problem-solving mindset.
• Strong communication skills to effectively collaborate with cross-functional teams.
• Exceptional project management abilities to handle multiple priorities.

Preferred Qualifications

While not required, the following qualifications would give you an edge:

• Knowledge of cloud platforms like GCP (Google Cloud Platform) or AWS (Amazon Web Services).
• Experience building Microservices and scalable solutions.
• 3+ years of experience designing and building solutions using Kafka streams or messaging queues.
• Experience with GitHub/Bitbucket and using CI/CD pipelines for automated deployments.
• Familiarity with NoSQL databases such as HBase, Couchbase, or MongoDB.

Why Join arenaflex?

At arenaflex, we believe that our greatest asset is our people. We foster an environment where innovation thrives, and every team member has the opportunity to make a meaningful impact. Here's what you can expect when you join our team:

• Cutting-Edge Technology: Work with the latest Big Data technologies and contribute to open source communities.
• Professional Development: We invest in your growth with dedicated time and resources for learning new skills, attending conferences, and pursuing certifications.
• Collaborative Culture: Work in an inclusive environment where your voice matters, and teamwork is celebrated.
• Work-Life Balance: Enjoy flexible working arrangements and a supportive atmosphere that values your well-being.
• Career Growth: Clear pathways for advancement and the chance to take on increasing responsibilities as you develop in your career.
• Competitive Compensation: We offer competitive salaries and comprehensive benefits packages.

Compensation & Benefits

We recognize that talent deserves to be rewarded. arenaflex offers a competitive compensation package that includes:

• Attractive hourly rate of $28 per hour (commensurate with experience and qualifications).
• Comprehensive health, dental, and vision insurance.
• 401(k) retirement plan with company matching.
• Paid time off and holidays.
• Annual performance bonuses and incentives.
• Professional development reimbursement.
• Employee assistance programs and wellness resources.

Work Environment

At arenaflex, we embrace a flexible work approach. While this position is based in Phoenix, USA, we support hybrid and remote work arrangements that allow our team members to balance their professional and personal lives effectively. Our offices are designed to foster collaboration and innovation, with state-of-the-art technology to support your work.

You'll be joining a diverse team of professionals who are passionate about technology and committed to excellence. We believe in continuous improvement and encourage creative problem-solving. Your ideas will be heard, and your contributions will be recognized.

How to Apply

If you're ready to take the next step in your career and make an impact at arenaflex, we encourage you to apply. The proposal of employment with arenaflex is conditioned upon the successful completion of a background verification check, subject to applicable laws and regulations.

Don't miss this exciting opportunity to be part of a team that's pushing the boundaries of what's possible in data engineering. Apply now and find your place in technology at arenaflex!

We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.