Multi-Agent Job Tracking with LiteLLM: Building a Private Local Python Pipeline

Using a single prompt for your entire job search workflow yields poor results. When you ask one LLM call to read a job posting, compare it to your resume, calculate a fit score, and draft an outreach note, the model tends to skip requirements and fabricate experience.

A subagent setup solves this by breaking the workflow into discrete, single-purpose steps. Using LiteLLM—an open-source Python library that provides a unified interface across local and cloud models—you can orchestrate three specialized subagents locally with no recurring platform fees.

The Three-Agent Separation of Concerns

To maintain accuracy, separate ingestion from evaluation and drafting:

With this division, Agent 3 cannot fabricate experience because it only receives verified facts approved by Agent 2.

Implementing the Pipeline with Python and LiteLLM

Create a virtual environment and install the library with pip install litellm. Ensure your local model runner, such as Ollama, is running.

Create a script named job_agents.py with this structure:

import json
from litellm import completion

MODEL = 'ollama/mistral'

def run_extractor(job_text):
    prompt = f'Extract hard skills and minimum experience from this job text. Output JSON only with keys skills and experience: {job_text}'
    response = completion(model=MODEL, messages=[{'role': 'user', 'content': prompt}])
    return response.choices[0].message.content

def run_auditor(requirements, resume_text):
    prompt = f'Compare these requirements: {requirements} against this resume: {resume_text}. Output JSON with match_score (0-100) and verified_skills.'
    response = completion(model=MODEL, messages=[{'role': 'user', 'content': prompt}])
    return response.choices[0].message.content

def run_drafter(verified_data):
    prompt = f'Draft a 3-sentence recruiter note using ONLY these verified skills: {verified_data}. Do not invent details.'
    response = completion(model=MODEL, messages=[{'role': 'user', 'content': prompt}])
    return response.choices[0].message.content

This structure ensures that each function runs independently and logs its intermediate output before passing data forward.

Executing the Pipeline and Managing State

Run the pipeline by passing a job description through the chain sequentially. If the Auditor agent returns a match score below your target threshold (for instance, under 70%), the script stops before invoking the Drafter.

This check prevents you from drafting outreach messages for roles where your profile does not meet core requirements.

Frequently Asked Questions

Why use LiteLLM instead of calling model APIs directly?

LiteLLM standardizes the input and output format across hundreds of providers. You can test your pipeline locally with Ollama and switch to a cloud provider later by changing a single string parameter.

How do I stop the Extractor agent from returning invalid JSON?

Include system constraints specifying format: json where supported, or use a negative constraint instructing the model to exclude conversational markdown formatting.

Can this pipeline run automatically in the background?

Yes. You can schedule the Python script to run using operating system tools like cron on Linux and macOS, or Task Scheduler on Windows.

Key Takeaways

Related Reading