duck.utils.threading.threadpool

Module Contents

Classes

ThreadPoolManager

Thread pool manager with task type protection.

Functions

get_or_create_thread_manager

Retrieve or create the ThreadPoolManager instance bound to the current thread (or inherited from its parent thread).

Data

REGISTRY

API

exception duck.utils.threading.threadpool.ManagerNotFound

Bases: Exception

Raised if manager cannot be resolved and user strictly wants to get the manager and user doesn’t allow creating it if it doesn’t exist.

Initialization

Initialize self. See help(type(self)) for accurate signature.

duck.utils.threading.threadpool.REGISTRY

None

class duck.utils.threading.threadpool.ThreadPoolManager(creator_thread: Optional[threading.Thread] = None, _id: Optional[Union[str, int]] = None)

Thread pool manager with task type protection.

Use start() to initialize a centralized threadpool for sync tasks. Restrict submitted tasks by their task_type, preventing inappropriate jobs in critical worker pools.

Initialization

Initialize the threadpool.

Parameters:

• creator_thread – This is the thread responsible for this manager.’

• _id – A custom Unique Identifier for the manager.

__instances

[]

This is the list of created instances.

__repr__()

__str__()

_worker_init()

Method called when worker thread is initialized.

classmethod all_instances() → List[duck.utils.threading.threadpool.ThreadPoolManager]

Returns a list of all created instances so far.

get_pool() → concurrent.futures.ThreadPoolExecutor

Returns the running threadpool.

Returns:

Running thread pool.

Return type:

concurrent.futures.ThreadPoolExecutor

Raises:

RuntimeError – If the threadpool is not running.

classmethod registry() → Dict[int, Dict[Any, duck.utils.threading.threadpool.ThreadPoolManager]]

Returns the registry for created instances. Useful for tracking.

start(max_workers: int, task_type: Optional[str] = None, daemon: bool = False, thread_name_prefix: Optional[str] = None)

Starts the threadpool, ready to accept tasks.

Parameters:

• max_workers – Maximum threads for pool.

• task_type – Only allows tasks with this type to be submitted. Useful for protecting pools handling critical jobs (e.g., request_handling only).

• daemon – Whether pool worker threads should be daemon threads.

• thread_name_prefix – Thread naming prefix (optional).

Raises:

RuntimeError – If thread pool already available and initialized.

stop(wait: bool = True)

Shutdowns the threadpool.

Parameters:

wait – Whether to wait for running tasks to finish.

submit_task(task: Callable, task_type: Optional[str] = None, log_exception: bool = True) → concurrent.futures.Future

Submit a task to the threadpool.

Parameters:

• task – Callable to execute.

• task_type – Type/flag of this task. If manager was initialized with a specific allowed task_type, this must match or raise UnknownTaskError.

• log_exception – Whether to log exception immediately after it happens. Set this to False, if you are using future.result() or future.exception() later in your code.

Raises:

• UnknownTaskError – If task_type mismatches the pool’s allowed type.

• RuntimeError – If the thread pool is None/not running.

Returns:

Future for the executing task.

Return type:

concurrent.futures.Future

exception duck.utils.threading.threadpool.UnknownTaskError(task_type, pool_task_type)

Bases: Exception

Raised when attempting to submit a task of a disallowed or unknown type.

This error indicates a task type was provided (or omitted) that does not match the pool’s configured protection. Typical use is to prevent accidental or inappropriate task submission to specialized or critical pools. If you need to run a different type of task, consider subclassing or reconfiguring the pool.

Initialization

Initialize self. See help(type(self)) for accurate signature.

duck.utils.threading.threadpool.get_or_create_thread_manager(id: Optional[Any] = None, force_create: bool = False, strictly_get: bool = False) → ThreadPoolManager

Retrieve or create the ThreadPoolManager instance bound to the current thread (or inherited from its parent thread).

This function attaches the manager to the current thread’s identity so that all child and sub-child threads automatically share the same manager unless they explicitly call this function with force_create=True to create an isolated one.

This is especially useful in systems where each worker thread requires its own dedicated thread-pool manager (e.g., to isolate request-scoped or component-scoped work) while still allowing thread trees to share a consistent state.

Parameters:

• id –

  Optional namespace/identifier for the manager.

  • Using the same id returns the same manager instance for this thread lineage.

  • Using a different id allows multiple managers to coexist per thread.

  Example:

  default_mgr = get_or_create_thread_manager()  # default
  job_mgr = get_or_create_thread_manager(id="job-executor")  # separate manager
  

• force_create – If True, bypasses lineage resolution and creates a fresh manager bound to the current thread. This is required inside worker threads where isolated loops are needed.

• strictly_get – Whether to strictly get the loop manager or raise an exception if manager not found.

Raises:

• ManagerNotFound – Raised if manager not found and strictly_get=True.

• TypeError – If arguments strictly_get and force_create are both True.

Notes:

• When using worker threads, this function must be called inside each worker thread, not in the main thread. Calling it in the main thread will cause the manager to propagate to all descendant threads, resulting in one shared instance.

• The manager resolves using a thread lineage lookup: if the current thread has no manager for the given id, its parent thread is checked recursively until the root thread is reached.

• If you already have created manager, use strictly_get argument to strictly get your created manager or raise an exception if manager not found without creating new one.

Returns:

The resolved or newly created manager instance.

Return type:

ThreadPoolManager

Centralized Thread Manager With Hierarchical Context Propagation

This module provides a structured system for managing thread-scoped ThreadPoolManager instances with optional task-type protection. It enables worker threads, request handlers, or subsystem threads to each maintain their own dedicated thread-pool manager, preventing concurrency issues caused by cross-thread state leakage.

Unlike a global thread pool, managers resolved through this module follow a thread lineage model: child threads inherit their parent’s manager unless they explicitly create or request a new one. This creates predictable, isolated execution environments ideal for component-based rendering, request lifecycles, and subsystems that must not share threadpools.

Core Features

1. Thread-Scoped Managers Each thread may own its own ThreadPoolManager instance. This manager is automatically reused by all descendant threads unless a new instance is explicitly created. This eliminates the need for fragile global registries.

2. Hierarchical Resolution (Thread Lineage Lookup) Manager resolution behaves like a “thread context”:

   • Start from the current thread.

   • If no manager exists for the specified ID, walk upward through the parent thread chain.

   • If none exists in the lineage, create a new manager and attach it to the current thread.

   This makes manager acquisition both deterministic and flexible.

3. Multiple Manager Namespaces Using the id parameter, threads can maintain several different logical managers simultaneously (e.g., CPU-bound pool, IO pool, component-render pool). Each namespace is isolated without requiring separate registry structures.

4. Task-Type Protection Each manager can enforce a single allowed task_type for submitted tasks. This prevents accidental or unsafe mixing of workloads—for example, ensuring that: - time-sensitive tasks run only in designated pools, - component rendering tasks never leak into background worker pools, - slow jobs cannot clog request-handling threads.

5. Daemon and Non-Daemon Worker Control Managers can start thread pools with daemon threads (for non-blocking shutdown) or non-daemon threads (for guaranteed completion of jobs before exit).

6. Safe Integration With Worker Thread Models When used inside worker threads, calling get_or_create_thread_manager() ensures that the thread and all of its descendants operate under the same manager instance. This prevents component registry mismatches and mixed execution contexts that occur when unrelated threads share global pools.

Motivation

Frameworks that use component rendering, fine-grained request lifecycles, or thread-local state (such as dynamic UI components or live-updating view systems) often need isolated execution domains. Using traditional global ThreadPoolExecutor instances can lead to:

• state bleeding across requests,

• mixed component registries,

• concurrency bugs when a task intended for one context is executed in another,

• performance degradation when slow tasks block high-priority ones.

This module solves these problems by binding pool managers to thread identity and propagating them through thread hierarchies, ensuring that each task belongs to the correct execution environment.

Typical Usage

def worker_entrypoint():
    # Initialize or reuse the thread's manager
    manager = get_or_create_thread_manager(id="render")

    # Start a dedicated thread pool for rendering tasks
    manager.start(max_workers=4, task_type="component_render")

    future = manager.submit_task(
        some_callable,
        task_type="component_render",
    )
    return future.result()

## Best Practices

- Always call `get_or_create_thread_manager()` *inside* worker threads, never
  in the main thread (unless global propagation is desired).
- Choose meaningful manager IDs to organize workloads (e.g., "io", "cpu",
  "request", "component-render").
- Use task-type protection to prevent misuse of specialized pools.
- Avoid accessing the internal `REGISTRY` directly; use the resolver instead.

This module is designed to be robust, extensible, and suitable for advanced
server architectures and dynamic component systems such as those in the Duck
Lively Component System.