Spark Executor

About

Spark Executor is a Model Context Protocol (MCP) server engineered to execute PySpark code. Built with Clean Code principles, it offers a modular and organized structure, configured through environment variables and a JSON interface based on the MCP protocol. The server executes PySpark code through spark-submit, and includes both unit and integration tests to ensure reliability. It requires a pre-configured PySpark and Spark environment and supports Go 1.23 or higher.

Key Features

• Offers a modular and organized structure.
• Executes PySpark code via spark-submit.
• Configurable via environment variables.
• Uses a JSON interface based on the MCP protocol.
• Implements Clean Code principles for maintainability.
• 0 GitHub stars

Use Cases

• Integrating Spark processing into MCP-based systems.
• Providing a scalable and managed environment for PySpark execution.
• Executing PySpark jobs remotely.