Can I use these patterns for both sync and async operations?

Yes, the patterns include logic to check for the __await__ attribute and provide a fallback to synchronous execution when necessary.

What is the main benefit of using pyo3_async_runtimes with this skill?

It enables async Python callbacks that are significantly faster than traditional spawn_blocking for rapid operations, offering up to 28x speed improvements.

Is this skill only for HTML to Markdown conversion?

While derived from the high-performance Kreuzberg document engine, these PyO3 patterns are applicable to any project bridging Rust and Python for performance.

Does this skill cover GIL management best practices?

Yes, it recommends using Python::attach() over with_gil() and provides patterns for correctly releasing the GIL before awaiting async futures in Rust.

How does this skill help prevent deadlocks in PyO3?

It provides specific patterns for using block_in_place versus spawn_blocking, particularly warning against spawn_blocking on PostProcessors which is a common cause of GIL deadlocks.

PyO3 Performance Patterns

Name: PyO3 Performance Patterns
Author: kreuzberg-dev

bykreuzberg-dev

•

480

•

Developer Tools

Optimizes Rust and Python integration using PyO3 to ensure high-performance async execution and deadlock-free GIL management.

This skill provides specialized guidance for developers using PyO3 to bridge Rust and Python, focusing on performance optimization and concurrency safety. It helps Claude implement efficient async Python callbacks using pyo3_async_runtimes, manage Global Interpreter Lock (GIL) release/attach cycles correctly, and choose between spawn_blocking and block_in_place to prevent critical deadlocks in post-processing tasks. By applying these patterns, developers can achieve significantly faster execution for operations within document intelligence and data conversion pipelines.

Key Features

01Deadlock prevention strategies for PostProcessors and Validators

02Efficient GIL management using Python::attach() patterns

03Async Python callback optimization via pyo3_async_runtimes

04Performance-driven choice between spawn_blocking and block_in_place

05480 GitHub stars

06Conditional async handling with fallback to spawn_blocking

Use Cases

01Integrating Rust-based HTML-to-Markdown converters with Python async frameworks

02Building high-performance Python libraries with a Rust core

03Implementing document processing pipelines that require concurrent data extraction

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add kreuzberg-dev/html-to-markdown pyo3-performance-patterns

For use in Claude.ai and ChatGPT

Download Skill