How does it handle text longer than the standard 512-token limit?

It includes built-in strategies for long text, such as 'head and tail' truncation and support for the Longformer model which can handle up to 4096 tokens.

Which transformer models are supported by this skill?

The skill supports a variety of popular models including BERT, RoBERTa, DistilBERT, GPT-2, and Longformer, suitable for tasks ranging from classification to generation.

What kind of text preprocessing is available?

It provides comprehensive cleaning options including HTML/URL removal, special character stripping, stopword removal, lemmatization, and spell correction.

Does it support model deployment?

Yes, the skill includes commands to export your fine-tuned models to production-ready formats like ONNX along with inference scripts.

Can I use my own datasets for training?

Yes, the pipeline is designed to work with your local data, providing structured increments for training, validation, and testing sets within the SpecWeave directory.

NLP Pipeline Builder

Name: NLP Pipeline Builder
Author: anton-abyzov

byanton-abyzov

•

데이터 과학 및 ML

Builds production-ready natural language processing pipelines using state-of-the-art transformer models and SpecWeave automation.

The NLP Pipeline Builder is a specialized skill for Claude Code that automates the creation and deployment of end-to-end machine learning workflows for text. It streamlines complex tasks such as text preprocessing, tokenization, and the fine-tuning of transformer models like BERT, RoBERTa, and GPT. By integrating directly with the SpecWeave framework, it ensures that every ML iteration—from sentiment analysis to named entity recognition—is backed by structured increments, comprehensive testing, and deployment-ready exports like ONNX.

주요 기능

01Seamless integration with transformer models like BERT, RoBERTa, and GPT-2

02Advanced handling for long documents using Longformer and truncation strategies

0313 GitHub stars

04Automated text preprocessing including cleaning, lemmatization, and spell correction

05End-to-end support for text classification, NER, and sentiment analysis

06Built-in commands for model evaluation and production deployment

사용 사례

01Automating entity extraction from unstructured legal or medical documentation

02Fine-tuning domain-specific language models for specialized text generation tasks

03Developing high-accuracy sentiment analysis tools for social media or product reviews

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add anton-abyzov/specweave nlp-pipeline-builder

For use in Claude.ai and ChatGPT

Download Skill