What happens to the papers that are filtered out?

While the 'core_set.csv' contains the top-ranked papers, a full deduped list is maintained in 'papers_dedup.jsonl' so no data is permanently lost during the ranking process.

Is the ranking process random or deterministic?

The process is deterministic. Rerunning the skill with the same input files will yield stable, repeatable results, which is essential for academic transparency.

Can I adjust the number of papers included in the final core set?

Yes, you can use the --core-size flag in the command line or define it within your queries.md file to set a specific target size for your output.

How does this skill identify duplicate research papers?

It uses a normalization process on the title and publication year to group similar entries, ensuring that only the version with the most complete metadata is retained.

Research Paper Deduplicator & Ranker

Name: Research Paper Deduplicator & Ranker
Author: WILLOSCAR

byWILLOSCAR

•

Ciencia de Datos y ML

Streamlines research paper collections by deduplicating entries and ranking them to create high-quality core datasets for academic analysis.

Acerca de

The Dedupe-Rank skill is a deterministic curation tool designed to transform broad, noisy research retrieval results into a manageable 'core set' of papers. It utilizes sophisticated normalization of titles and years to eliminate duplicates while preserving the richest available metadata. By ranking entries based on relevance, recency, and specific architectural signals—such as pinning known classics or ensuring a quota of prior surveys—it provides a stable, repeatable foundation for building taxonomies, literature reviews, and research outlines.

Características Principales

Automated identification and pinning of classic papers and prior surveys
Configurable core set sizing for flexible research depth and iteration speed
Deterministic title and year-based deduplication to ensure data integrity
Smart ranking based on relevance, recency, and specific research signals
96 GitHub stars
Generates structured CSV core sets with stable IDs and direct PDF links

Casos de Uso

Converting raw search results into a refined core set for systematic literature reviews
Deduplicating research datasets compiled from multiple academic databases
Selecting high-impact papers to serve as the foundation for AI-generated outlines

Acerca de

Características Principales

Automated identification and pinning of classic papers and prior surveys
Configurable core set sizing for flexible research depth and iteration speed
Deterministic title and year-based deduplication to ensure data integrity
Smart ranking based on relevance, recency, and specific research signals
96 GitHub stars
Generates structured CSV core sets with stable IDs and direct PDF links

Casos de Uso

Converting raw search results into a refined core set for systematic literature reviews
Deduplicating research datasets compiled from multiple academic databases
Selecting high-impact papers to serve as the foundation for AI-generated outlines