Senior Data Engineer | USA | Remote

Description • Join Grafana Labs, a globally recognized open-source powerhouse, as a Senior Data Engineer and play a pivotal role in shaping our data infrastructure. With over 20 million users worldwide, Grafana is at the forefront of observability, monitoring everything from critical infrastructure to scientific endeavors. We are a remote-first organization, fostering a collaborative and innovation-driven culture where transparency, autonomy, and trust are paramount. This is a unique opportunity to contribute to a company with a strong open-source legacy and a passion for making a meaningful impact. • As a Senior Data Engineer, you will be instrumental in maintaining and evolving the frameworks and systems responsible for acquiring, validating, cleansing, and loading data into and out of our analytics platforms. The systems you build and manage will directly empower our business partners, enabling them to more accurately and reliably track and forecast key metrics such as sales, revenue, and usage/consumption. Your work will be critical in providing the data foundation for informed business decisions. • A significant aspect of this role involves leading the development and implementation of machine learning pipelines. As we mature our data capabilities, you will be at the forefront of productionalizing internal predictive models, transforming raw data into actionable insights that drive strategic initiatives and enhance our product offerings. • This position offers broad engagement across various departments within the company. You will collaborate closely with finance, revenue and customer experience operations, analytics teams, and analytics engineering. Your expertise will be sought after to integrate with and enhance our existing data stack, which includes cutting-edge tools and technologies such as Google Cloud Storage (GCS), BigQuery, dbt, dlt, Prefect, Python, Fivetran, Rudderstack, Hightouch, and OpenMetadata. • Your core responsibilities will include building and maintaining robust, production-quality data pipelines. This involves establishing reliable data flows both into and out of BigQuery from various operational systems, ensuring data integrity and accessibility. • You will be responsible for implementing comprehensive data quality and freshness checks, along with setting up monitoring processes. This proactive approach will guarantee the accuracy, consistency, and timeliness of our data, which is crucial for reliable analytics and reporting. • A key part of your role will be to maintain and actively contribute to our data ingestion framework. This includes leveraging and potentially extending various purpose-built data load tools (dlt) connectors to efficiently bring data into our ecosystem. • You will create and maintain thorough, up-to-date documentation for all data engineering processes, systems, and workflows. Clear and accessible documentation is vital for knowledge sharing, onboarding new team members, and ensuring the long-term maintainability of our data infrastructure. • Maintaining observability and robust monitoring of our internal data pipelines is essential. You will ensure that our data flows are transparent, performant, and that any issues are identified and addressed promptly. • Troubleshooting and resolving data pipeline issues will be a regular part of your duties, ensuring minimal disruption and maximum downstream data availability for our business partners. • You will contribute significantly to our dbt (data build tool) systems. This includes ensuring that source and staging layers adhere to our established standards, are optimized for efficiency and cost-effectiveness, and are highly available. • You will actively participate in the investigation and implementation of event-driven data movement and transformation processes, exploring modern architectural patterns to enhance data processing capabilities. • Furthermore, you will be involved in exploring and implementing advanced analytic data storage and table formats, such as Apache Iceberg, to optimize performance and scalability. • This role is ideal for a motivated self-starter who is eager to make a tangible impact and is not afraid to tackle complex, large-scale data challenges. Your ability to explain technical concepts clearly to non-technical audiences and foster essential cross-team relationships will be key to your success. • This is a remote opportunity, and we are seeking candidates located within USA time zones only. Requirements • Strong software development skills, with proficiency in at least one of the following languages: Python, Java, Scala, or Go. • High proficiency in SQL for complex data querying and manipulation. • Proven experience building and maintaining data ingestion pipelines using a workflow orchestration system such as Prefect, Dagster, or Airflow. • Working knowledge of dbt or similar data transformation tools. • Excellent communication skills, with the ability to articulate technical concepts to diverse audiences and build strong cross-functional relationships. ️ Benefits • 100% Remote, Global Culture: Work from anywhere within USA time zones in a company that values collaboration and diversity. • Stock Options (RSUs): Receive Restricted Stock Units, granting you ownership and a stake in Grafana Labs' success. • Generous Paid Time Off: Enjoy 30 days of annual leave, with 3 designated company shutdown days to ensure you can truly disconnect and recharge. • Career Growth Opportunities: Access defined pathways for professional development and career advancement within a scaling organization. Apply tot his job Apply To this Job