Senior Software Security Engineer, Detection Engineering

About Grafana Labs

Grafana Labs is a remote-first, open-source powerhouse, serving over 20 million users globally with its visualization tool, Grafana. Our dashboards are ubiquitous, seen from NASA to Wimbledon. We also empower over 3,000 companies, including industry giants like Bloomberg and JPMorgan Chase, to manage their observability strategies using the Grafana LGTM Stack, featuring scalable metrics (Grafana Mimir), logs (Grafana Loki), and traces (Grafana Tempo).

We are rapidly expanding, committed to our open-source heritage, global collaborative culture, and a passion for impactful work. Our team thrives in an innovation-driven environment built on transparency, autonomy, and trust.

Even if you don't meet every single requirement, we encourage you to apply for this potentially career-defining opportunity.

This is a remote position based in Germany.

The Opportunity for a Senior Software Security Engineer, Detection Engineering

As a Senior Software Security Engineer on the Detection & Response Engineering team, you will be instrumental in developing advanced security tools and processes for our cutting-edge observability platform. Your work will focus on identifying and neutralizing sophisticated threats targeting our platform, employees, and customers.

You will engage with all aspects of the technology stack, performing advanced development, detection research, and response automation. Your contributions and learnings will also be shared with the broader security community. You will collaborate with other security engineers, developers, and customer-facing teams to resolve complex security and detection challenges.

Key Responsibilities

• Collaboratively design, build, and maintain internal detection systems using Go, TypeScript, Python, and the Grafana observability stack, processing millions of security data points daily.
• Research and develop sophisticated detection (as code) capabilities and rules to cover risks across product and corporate systems, contributing these detections to the OSS community where appropriate.
• Work with product teams and other stakeholders to ensure effective telemetry across all existing and future products.
• Lead the development of response tooling to streamline and automate response activities, alongside writing and maintaining runbooks for non-automatable scenarios.
• Following a SOCless model, guide cross-functional teams in integrating telemetry, detections, and response procedures into their operational processes.
• Design security and operations metrics to track success and demonstrate the security value of our work.
• Lead the response to security alerts, potential incidents, and customer security issues, participating in security incident on-call rotations.

We prioritize developer productivity, offering modern AI coding assistants (your choice, within security guidelines) with a company-funded budget to facilitate quick iteration without friction. We advocate for pragmatic AI-assisted development—for faster prototyping, test generation, refactors, documentation, and incident follow-ups—always alongside rigorous code review and quality standards. You will also have access to frontier models such as GPT-Codex 5/3, Claude Opus 4.6, and Gemini 3 Pro.

What Makes You a Great Fit

• Significant experience (4+ years in a software engineering-oriented role) with at least one programming language, primarily Go, TypeScript (React), and Python. A code screen will be part of the evaluation.
• Experience with core security concepts and their application to modern application architectures, understanding threat models for cloud systems, defense strategies, and attack detection.
• Experience with common security operations or detection engineering concepts and practices, including detection rule formats like Sigma, YARA, or Rotom.
• Significant experience with public clouds, Kubernetes container ecosystems, and securing applications within them, encompassing eBPF, cloud IAM, service meshes, or container hardening.
• A motivated self-starter with considerable curiosity and a bias towards action, demonstrating a passion for learning, security, and advancing security across the company and industry.
• An adept communicator, proficient in in-person, asynchronous communication, and technical documentation.

Bonus Points For

• Working knowledge of Grafana Labs OSS projects and products, with experience using observability tooling (metrics, logs, traces, profiles) to solve security problems.
• Battle-tested ideas on novel approaches to security and detection challenges for hybrid cloud and OSS companies like Grafana Labs.
• Experience collaborating with OSS communities.
• Significant experience securing large-scale distributed systems running on Kubernetes in public clouds.

Compensation & Rewards

In Germany, the base compensation for this role ranges from EUR 94,208 to EUR 113,050. Actual compensation may vary based on level, experience, and skillset. All roles include Restricted Stock Units (RSUs), aligning every team member with Grafana Labs' success through shared ownership.

Why You’ll Thrive At Grafana Labs

• 100% Remote, Global Culture: We unite talent worldwide, fostering collaboration and shared purpose as a remote-only company.
• Scaling Organization: Engage in meaningful work within a high-growth, dynamic environment.
• Transparent Communication: Benefit from open decision-making and regular company-wide updates.
• Innovation-Driven: Enjoy the autonomy and support to deliver great work and explore new ideas.
• Open Source Roots: Our community-driven values shape our work.
• Empowered Teams: A high-trust, low-ego culture that prioritizes outcomes.
• Career Growth Pathways: Clearly defined opportunities for career development.
• Approachable Leadership: Transparent and engaged executives.
• Passionate People: Join a team of intelligent, supportive individuals dedicated to their work.
• In-Person Onboarding: Start strong with fellow 'Grafanistas' to immerse yourself in our mission and methods.
• Balance is Key: Benefit from a global annual leave policy of 30 days, including 3 dedicated Grafana Shutdown Days for complete disconnection (local legislation compliant).

Key skills/competency

• Security Engineering
• Detection Engineering
• Cloud Security
• Kubernetes
• Go/Python/TypeScript
• Incident Response
• Threat Modeling
• Observability
• Detection Rules (Sigma, YARA)
• Distributed Systems