Which GPU is best for a team of 20 developers?

For medium production teams, 2x RTX 4090 or A6000 GPUs are recommended to handle models in the 34B-70B range with reasonable concurrency and throughput.

What network bandwidth is required for local AI nodes?

The skill recommends 10GbE for standard corporate network access and up to 100GbE for internal model loading and storage-to-GPU traffic to minimize loading times.

How do I calculate the VRAM needed for a specific model?

Use the formula: (Parameters × Bytes per Parameter) + Context Window Overhead. For example, a 70B model in INT4 needs approximately 38GB of VRAM plus 2-8GB for the context window.

Can this skill help with cost-benefit analysis for on-premise AI?

Yes, it includes detailed TCO templates and cloud API cost comparisons to help you determine the breakeven point between local infrastructure and cloud providers like OpenAI or Anthropic.

AI Hardware Sizing & Infrastructure

Name: AI Hardware Sizing & Infrastructure
Author: majiayu000

bymajiayu000

•

数据科学与机器学习

Calculates and specifies optimal hardware requirements for local Large Language Model (LLM) deployments and AI workloads.

This skill provides specialized expertise in planning and sizing local AI infrastructure, covering everything from GPU selection to full server configurations for various team sizes. It enables developers and IT architects to perform detailed capacity planning, throughput calculations, and TCO modeling to ensure local model deployments are production-ready, cost-effective, and scaled appropriately for specific model sizes like Qwen, GLM, or Llama variants. It is particularly useful for organizations moving away from cloud APIs toward self-hosted, data-sovereign AI solutions.

主要功能

01GPU VRAM requirement calculations for FP16, INT8, and INT4/AWQ quantization

02Comprehensive storage architecture and high-performance network bandwidth planning

03Throughput estimation and capacity planning formulas based on developer usage patterns

04Total Cost of Ownership (TCO) modeling and local-vs-cloud cost comparison analysis

0594 GitHub stars

06Standardized server configuration templates for small, medium, and enterprise teams

使用场景

01Designing a dedicated on-premise AI server for a development team of 50 users

02Building a business case for capital investment in AI hardware versus recurring cloud API costs

03Calculating if existing hardware can support hosting a 70B parameter model with 32k context

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add majiayu000/claude-skill-registry local-ai-infrastructure

For use in Claude.ai and ChatGPT

Download Skill