What sources does Newsstack use for news ingestion?

Newsstack continuously ingests articles from a variety of major RSS feeds (e.g., AP News, NPR, NY Times, BBC) and integrates with the GDELT DOC API for comprehensive, real-time news aggregation.

How does Newsstack ensure data quality and topic grouping?

It employs a robust three-layer deduplication process (URL hash, SimHash, and vector cosine similarity) to remove duplicates. Articles are then clustered into related stories using HDBSCAN over their embeddings for organized insights.

What kind of data and tools does Newsstack expose to AI agents?

Newsstack offers structured news data, including article content, entities, and clusters. AI agents can access intelligence tools like latest headlines, semantic search, regional news, LLM-generated topic briefings, and trending topics via its API.

Newsstack

Name: Newsstack
Author: lonelycode

0•

Aggregates articles from diverse sources, deduplicates, clusters related stories, and exposes structured news data to AI agents.

Newsstack functions as an MCP server for comprehensive news intelligence, collecting articles from RSS feeds and the GDELT API. It processes this data through sophisticated steps including multi-layered deduplication (URL hash, SimHash, vector cosine), article embedding, named entity recognition (NER), and HDBSCAN clustering to identify related stories. All machine learning inference runs locally, ensuring privacy and eliminating the need for external API keys. The system then provides structured news data and analytical tools to AI agents through a streamable HTTP API, enabling functions like fetching headlines, semantic search, regional news, topic briefings, and trending topic identification.

主な機能

01Local ML inference for embeddings, named entity recognition (NER), and LLM-based summarization.

02Exposes structured news data and intelligence tools via a streamable HTTP API for AI agents.

03HDBSCAN clustering of articles based on embeddings to group related stories.

04Multi-layered deduplication using URL hash, SimHash, and vector cosine similarity.

05Aggregates news from RSS feeds and GDELT API with scheduled ingestion.

060 GitHub stars

ユースケース

01Building custom news monitoring dashboards and alerting systems.

02Generating automated intelligence briefings on specific topics or regions for research and reporting.

03Powering AI agents with real-time, structured news intelligence for analysis and decision-making.