Stream and transform large datasets into Elasticsearch using high-performance, memory-efficient processing.
The Elasticsearch File Ingest skill empowers Claude to manage high-performance data imports from various formats like NDJSON, CSV, Parquet, and Arrow IPC. Built for efficiency, it uses stream-based processing to handle massive datasets without memory bottlenecks, offering advanced features like custom JavaScript transformations for data enrichment and automatic schema inference. This tool is essential for developers and data engineers who need to quickly seed indices or import batch data into Elasticsearch 8.x or 9.x environments while maintaining control over mappings and pipelines.
主要功能
01Automatic mapping and ingest pipeline inference from source files
02Custom JavaScript transformation functions for real-time data enrichment
03Stream-based ingestion for low memory footprint on large files
04Multi-format support for NDJSON, CSV, Parquet, and Arrow IPC
05High throughput processing reaching 50k+ documents per second
06452 GitHub stars
使用场景
01Loading large CSV or Parquet datasets into Elasticsearch for analysis
02Stream-processing system logs with custom JavaScript filtering logic
03Batch importing historical data while applying complex field transformations
