or apply directly on OnBoard's site. We never take the application ourselves.
Is this posting real?
- This role has been open
- 19 days OnBoard's roles stay open a median of 23 days
- Reposted
- No
- Salary listed
- No 0% of OnBoard's roles list one
- Ghost-job risk at OnBoard
- low 1 stale, 1 reposted of 11 open
- Hiring momentum
- 18 roles opened in the last 90 days ↑ up vs. the prior 90 days
- Last confirmed on the employer's board
- 2026-09-17
Measured from postings appearing on and disappearing from OnBoard's own greenhouse board since 2026-08-03. Full hiring picture for OnBoard.
About this role
The Senior AI Engineer for Observability at OnBoard will focus on building and improving AI-powered product systems, ensuring their reliability and quality in production. This role involves designing evaluation pipelines, monitoring AI performance, and collaborating with product engineering teams to enhance AI features using technologies like LLMs and RAG. The engineer will also be responsible for establishing observability standards and ensuring compliance with data handling requirements.
- benefits
- 5/5
- freshness
- 4/5
- career value
- 5/5
- role clarity
- 5/5
- pay transparency
- 0/5
Scored from the posting itself — how clearly the role is described, how much it says about pay and benefits, and how recently it was listed. Not a judgement of OnBoard as an employer.
What you need
- 5+ years of software engineering experience building production systems
- Hands-on experience building or operating LLM-powered features, RAG systems, AI workflows, or similar AI applications
- Strong engineering ability in Python, C#/.NET, or both
- Practical understanding of prompts, embeddings, vector search, retrieval quality, orchestration patterns, and LLM application architecture
- Experience designing evaluations, quality metrics, or regression frameworks for AI or software systems
- Strong observability fundamentals: tracing, logging, metrics, alerting, and production debugging
Nice to have
- Experience with LLMOps or AI observability tools such as Arize, Langfuse, LangSmith, W&B, Humanloop, or Helicone
- Experience with OpenTelemetry, Azure Monitor, Application Insights, or similar observability platforms
- Experience with Azure AI Search, Pinecone, Qdrant, Weaviate, pgvector, or other vector search platforms
- Experience with Semantic Kernel, LangChain, LlamaIndex, AutoGen, or related frameworks
- Experience with LLM-as-judge evaluation, RAG evaluation, semantic similarity metrics, hallucination detection, groundedness scoring, or human-feedback workflows
What you get
- Fully remote work with company provided equipment (laptop, software, etc.)
- Comprehensive, high-quality medical/prescription drug plan options, as well as dental and vision plan offerings
- Employer contribution to Health Savings Account (HSA) if you participate in a High Deductible Healthcare Plan
- Medical Flexible Spending Accounts available
- Dependent Care Flexible Spending Accounts available
- Basic life insurance in the amount of $50,000 or 1 X’s your salary (whichever is higher)
Worth weighing
- The role emphasizes hands-on engineering and building AI systems, which may not suit candidates looking for a purely observational role.
- The position requires a strong focus on quality and observability, which could lead to high expectations and pressure to deliver reliable AI features.
- No specific salary range is provided, which may require candidates to inquire during the interview process.
Summarised from OnBoard's posting. Read the full original.
Listed by OnBoard on their greenhouse job board, last confirmed open on 2026-09-17. PitchMeAI is not the employer.
More roles at OnBoard
- Principal Software Engineer - AIUnited States
- Sales Development RepresentativeUnited States
- Sales Development RepresentativeIndianapolis, Indiana, United States
- Director, BrandUnited States
- Senior Manager, Marketing OperationsUnited States
- Customer Support Specialist ICanada
- Site Reliability EngineerUnited States
- Marketing Operations SpecialistUnited States