PitchMeAI Jobs
Flodesk

Senior Applied AI Engineer, Agent Quality & Evaluations

Flodesk · Remote

Posted about a month ago · $150,000 - $250,000

or apply directly on Flodesk's site. We never take the application ourselves.

Is this posting real?

This role has been open
31 days
Flodesk's roles stay open a median of 47 days
Reposted
No
Salary listed
Yes
44% of Flodesk's roles list one
Ghost-job risk at Flodesk
high
5 stale, 0 reposted of 9 open
Hiring momentum
10 roles opened in the last 90 days
↑ up vs. the prior 90 days
Last confirmed on the employer's board
2026-10-24

Measured from postings appearing on and disappearing from Flodesk's own greenhouse board since 2026-08-03. Full hiring picture for Flodesk.

About this role

In this role, you will manage and improve Flodesk's AI systems that assist small business owners in creating effective marketing campaigns. You will oversee the development of prompts, agent instructions, and evaluation criteria, ensuring the AI behaves as intended and meets quality standards. This position requires hands-on work that integrates engineering, product management, and natural language processing to enhance user experience and system performance.

Our read on this posting4.2out of 5
benefits
3/5
freshness
5/5
career value
4/5
role clarity
4/5
pay transparency
5/5

Scored from the posting itself — how clearly the role is described, how much it says about pay and benefits, and how recently it was listed. Not a judgement of Flodesk as an employer.

What you need

  • A track record of building or meaningfully improving production LLM or agentic products, not just prototypes
  • Strong software engineering skills in Python, TypeScript or a similar language
  • Experience with LLM evaluation, observability and experimental design
  • A strong product sense to translate member feedback into quality definitions and system changes
  • Clear communication skills across technical and nontechnical disciplines

Nice to have

  • Built content-generation, creative or marketing tools
  • Built multi-turn agents that use tools and preserve user intent across edits
  • Used Langfuse, Braintrust or comparable evaluation and observability platforms
  • Worked on products for small businesses, creators or marketers

What you get

  • $150,000 - $250,000 base salary, depending on your location and experience
  • Fully paid health insurance for individual coverage
  • 16 weeks paid parental leave for non-birthing parents; 22 weeks paid maternity leave for birthing parents
  • Unlimited flexible time off
  • 401(k) match (US employees only)
  • $1,000 annual stipend for learning and development

Worth weighing

  • Remote-first company with a globally distributed team
  • Role involves hands-on systems work, not just writing prompts in isolation
  • Focus on building a scalable evaluation infrastructure
  • No specific mention of the tech stack beyond Python and TypeScript

Summarised from Flodesk's posting. Read the full original.

Listed by Flodesk on their greenhouse job board, last confirmed open on 2026-10-24. PitchMeAI is not the employer.

More roles at Flodesk

All 9 open roles at Flodesk →