Senior LLMOps Engineer

Senior LLMOps Engineer

CINC Systems is a leading provider of accounting and management software for the community association management industry. Their platform serves tens of thousands of associations and millions of homes within a highly regulated, data-sensitive, multi-tenant SaaS environment. The company is actively integrating AI-native capabilities directly into the core of its platform, aiming for reliable, scalable systems that deliver tangible customer value. The Senior LLMOps Engineer plays a crucial role in ensuring AI is production-ready, observable, secure, and cost-effective.

About The Role

The Senior LLMOps Engineer acts as a hands-on technical leader, responsible for the operation, scaling, and governance of large language model capabilities across the CINC platform. This role primarily focuses on the systems and practices bridging AI engineering and production operations, encompassing orchestration, evaluation, observability, safety, and cost control. This is a production engineering role, not a research position, suited for individuals who recognize that AI systems demand the same rigorous fundamentals, clear feedback loops, and disciplined operations as any other critical software system.

Key Responsibilities

• Design and operate LLM orchestration and runtime systems to support reliable, low-latency AI workflows.
• Build and maintain evaluation pipelines to measure the quality, regressions, and business impact of LLM-driven features.
• Implement comprehensive observability for AI systems, including tracing, metrics, and feedback loops at the prompt, agent, and workflow levels.
• Establish cost management strategies for LLM usage, incorporating budgeting, rate limiting, caching, and optimization.
• Partner closely with AI and product engineers to safely and incrementally productionize AI features.
• Define and enforce robust guardrails for security, privacy, and data handling within AI workflows.
• Support experimentation with new models and tools while rigorously ensuring production stability.
• Improve incident readiness and response protocols for AI-related failures and degradations.
• Influence crucial build versus buy decisions for LLM tooling and platforms.
• Mentor engineers and help cultivate best practices for operating AI systems at scale.

Qualifications

Technical Experience

• 8+ years of experience in software engineering, platform engineering, or DevOps roles.
• Hands-on experience operating LLM-powered systems in production environments.
• Familiarity with various LLM providers and orchestration frameworks.
• Strong understanding of distributed systems, APIs, and cloud-native architectures.
• Experience designing observability and evaluation systems for complex workflows.
• Practical knowledge of cost and performance optimization in cloud environments.
• Experience working with event-driven architectures and asynchronous workflows.

Leadership and Collaboration

• Proven ability to lead through influence rather than direct authority.
• Highly structured thinker with strong problem-solving capabilities.
• Clear communicator adept at explaining AI operational trade-offs to both technical and non-technical stakeholders.
• Comfortable collaborating across diverse teams in a dynamic, evolving environment.

Mindset and Values

• Possesses a builder mindset, prioritizing reliability and tangible outcomes.
• Believes that AI enhances engineering fundamentals rather than replacing them.
• Exhibits a learning-first attitude, continuously staying current with evolving AI tools and practices.
• Pragmatic and composed under pressure, particularly during incidents.
• Customer-aware, understanding the real-world impact of AI system behavior.

What Success Looks Like

• LLM-powered features consistently demonstrate reliability, observability, and cost-effectiveness in production.
• Engineers are empowered to ship AI-enabled capabilities with confidence and clear guardrails.
• AI quality and performance issues are detected early and resolved swiftly.
• The organization develops strong operational discipline surrounding AI systems.
• The Senior LLMOps Engineer is recognized as a trusted expert and partner across the engineering department.

Key skills/competency

• LLMOps
• Large Language Models
• Production Engineering
• Distributed Systems
• Cloud-Native Architecture
• Observability
• Evaluation Pipelines
• Cost Optimization
• DevOps
• API Design