Which data sources are supported by the rfp-ingest skill?

The skill currently supports SAM.gov (REST API), Maryland eMMA (Web scraping), and RFPMart (both REST API and manual CSV uploads).

How does the skill handle duplicate RFP records?

It employs a three-tier deduplication strategy: exact ID/Source matching, fuzzy title similarity within deadline windows, and URL canonicalization.

Can I filter RFPs for specific industries like IT?

Yes, the skill includes built-in mapping for IT-relevant prefixes such as SW (Software), ITES (IT Services), and NET (Networking) to prioritize relevant leads.

Does this skill support automated data updates?

Yes, it includes implementation patterns for Convex cron jobs that can be configured to poll APIs like SAM.gov at regular intervals (e.g., every 6 hours).

How does it detect if a contract requires a security clearance?

The ingestion logic includes an eligibility pre-filtering engine that uses regex patterns to flag disqualifiers or requirements like 'security clearance' or 'US citizenship' automatically.

RFP Data Ingestion Engine

Name: RFP Data Ingestion Engine
Author: Atemndobs

byAtemndobs

0•

ウェブスクレイピングとデータ収集

Automates the collection, normalization, and deduplication of Request for Proposal (RFP) opportunities from government and private data sources.

The rfp-ingest skill provides a robust framework for centralizing RFP discovery by aggregating data from SAM.gov, Maryland eMMA, and RFPMart into a unified canonical schema. It manages the complexities of API rate limiting, web scraping, and manual CSV parsing while implementing sophisticated deduplication logic and eligibility pre-filtering. Designed for developers building procurement platforms, it includes production-ready Convex actions, schema definitions, and automated cron job configurations to ensure a reliable and searchable database of contract opportunities.

主な機能

01Multi-source aggregation from SAM.gov, eMMA, and RFPMart

02Canonical schema normalization for consistent data processing

03Automated eligibility detection for security clearances and set-asides

04Fuzzy-match deduplication and URL canonicalization

05Production-ready Convex backend integration and scheduled crons

060 GitHub stars

ユースケース

01Building a centralized monitoring dashboard for government contract leads

02Filtering massive RFP feeds for specific IT-relevant NAICS codes

03Automating the synchronization of manual CSV email alerts into a database

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add atemndobs/nebula-rfp rfp-ingest

For use in Claude.ai and ChatGPT

Download Skill