Building a Lightweight RSS Job Sourcing Subagent Without Paid APIs
Ads
Refreshing job boards manually wastes time and creates application fatigue. Cluttered email alerts and paid aggregators often delay listings by hours. Setting up a local RSS sourcing script gives you direct control over new job listings.
Many job boards and career pages provide open RSS or Atom feeds. By combining a small script with a filtering prompt, you can triage listings locally without API subscription costs.
Finding public feed endpoints on major portals
You do not need to scrape dynamic web pages to find listings. Many hiring platforms maintain feeds that update as soon as jobs are published.
Sites like RemoteOK, We Work Remotely, and niche technical boards often append .rss or /feed to category URLs. Some enterprise applicant tracking systems also provide public feeds for specific department searches.
Collect three to five feed URLs that match your specialty, such as backend systems, infrastructure, or machine learning. Save the links in a local plain-text file.
Designing the local subagent architecture
A simple sourcing pipeline needs two parts: an ingestion script and a scoring filter. You do not need heavy agent frameworks.
A Python script using the feedparser library reads the feeds, removes duplicates using a local text file or SQLite table, and extracts the title, company, description, and link.
Instead of passing raw HTML to an external API, the script outputs unread listings into a local Markdown digest for your language model to score.
The evaluation and scoring prompt
Use this prompt to score daily listing digests against your career criteria.
System Prompt: You are a technical career advisor. Evaluate incoming job listings against a strict set of candidate constraints. Be ruthless in eliminating roles that do not match technical focus, compensation floors, or location boundaries.
User Prompt: Here are my non-negotiable criteria:
- Role Focus: Distributed systems, Go, Kubernetes, Linux internals.
- Excluded: Frontend development, management tracks, unpaid trial tasks.
- Location: Fully remote, US-based.
Evaluate the following raw listings: [PASTE RAW RSS DIGEST HERE].
For each listing, return a JSON object with: 'title', 'company', 'fit_score' (1 to 5), and 'match_reason' (maximum one sentence). Only display roles with a fit_score of 4 or 5.
Running the pipeline locally
Create a directory on your machine and install the feed library: pip install feedparser. Write a script that iterates through your URLs, extracts entries from the past 24 hours, and appends them to daily_leads.txt.
Run the script each morning using cron or a manual terminal command. Paste the resulting text into your evaluation prompt to filter the listings in seconds.
Frequently asked questions
Is parsing public RSS feeds against job board terms of service?
Public RSS and Atom feeds are built for automated reading. They do not bypass authentication walls or generate heavy server load.
Can I run this without installing Python?
Yes. You can use desktop feed readers like NetNewsWire or Feedly, then paste the unread daily entries directly into your model prompt for scoring.
How do I prevent duplicate listings across multiple feeds?
Store each entry's entry.id or URL in a local text file or database, and skip items that have already been recorded.