Member of Technical Staff, Data Platform

About Contextual AI

We're revolutionizing how AI Agents work by solving AI's most critical challenge: context. The right context at the right time unlocks the accuracy and production scale that enterprises leveraging AI require. Our enterprise AI development platform sits at the intersection of breakthrough AI research and practical developer needs. Our end-to-end platform allows AI developers to easily and accurately ingest and query documents from enterprise data sources and easily embed retrieval results into their business workflows.

Contextual AI was founded by the pioneers of Retrieval-Augmented Generation (RAG), the foundational technique behind the context layer, connecting foundation models to current and relevant information. Backed by the industry's most forward-thinking venture capitalists, we're not just participating in the enterprise AI revolution, we're defining it. Join us in building a future where AI doesn't just answer questions, it transforms businesses.

Job Overview

The Data Platform team, within the Contextual AI platform organization, powers product development, applied research, and customers' data-intensive workloads. The team designs, builds, and operates foundational data services at Contextual AI. This is a greenfield opportunity to shape the technical direction of the data engineering team at Contextual AI.

What you'll do:

• Design and implement scalable services, APIs, and databases to support the processing and ingestion of petabytes of information daily.
• Build and improve state-of-the-art multimodal LLMs to maximize document understanding performance.
• Design and implement comprehensive evaluation pipelines for E2E agentic RAG workflows.
• Architect and build streaming infrastructure, data orchestration systems, vector databases.
• Collaborate with ML researchers to understand state-of-the-art (SOTA) requirements for RAG systems, translating them into service specifications.
• Work directly with product managers and application engineers to understand customer requirements for end-to-end RAG systems, and translate them into technical solutions.
• Ensure seamless integration with machine learning models and pipelines, enabling efficient model deployment and management.
• Mentor and guide junior team members, promoting knowledge sharing and professional growth.

What we're seeking:

• Education: At least a Bachelor's degree in Computer Science, Software Engineering, or related field.
• Experience: Kubernetes services, distributed queuing systems, and streaming infrastructure. Proven ability to diagnose distributed vector databases and design systems for low-latency retrieval of image, text, audio, and video vectors.
• Machine Learning: Familiarity with machine learning concepts and frameworks, including dense information retrieval, document understanding/parsing models, and vision language model.
• Problem-Solving: Strong problem-solving skills and the ability to work effectively in a fast-paced, collaborative environment.
• Communication: Excellent communication and interpersonal skills, with the ability to work closely with cross-functional teams.

Key skills/competency

• Data Platform
• Scalable Services
• APIs & Databases
• Streaming Infrastructure
• Data Orchestration
• Vector Databases
• Multimodal LLMs
• Retrieval-Augmented Generation (RAG)
• Kubernetes
• Machine Learning