Skip to content
Ice Docs

Thread Pool Design Considerations

2 min read

2 min read

2 min read

2 min read

2 min read

By default, the client and server thread pools of a communicator have a single thread, so Ice dispatches one request at a time and processes one reply at a time. To let your application dispatch requests or process replies concurrently, you increase the maximum size of these thread pools. This page describes what each choice means for your application.

A thread pool with a maximum size of one thread behaves as follows:

  • The pool dispatches one request at a time, which lets you avoid (or postpone) dealing with thread-safety issues in your servants.
  • The pool processes one reply at a time. The AMI callbacks, which run on the same thread, also run one at a time.
  • The pool dispatches one request at a time, which lets you avoid (or postpone) dealing with thread-safety issues in servants with synchronous dispatch methods.
  • The pool processes one reply at a time.
  • The pool dispatches one request at a time.
  • The pool processes one reply at a time.
  • The pool reads one request at a time. The dispatches themselves run concurrently, each in its own task; Ice.Connection.name.MaxDispatches limits the concurrent dispatches on a connection.
  • The pool processes one reply at a time.

A thread pool with several threads dispatches requests and runs AMI callbacks concurrently, so your servants and callbacks must be thread-safe. The right maximum size depends on your workload: a compute-bound server gains nothing from more threads than cores, while a server whose dispatches block, on I/O for example, benefits from more threads than cores. Test with a realistic workload to find this size.

The Python interpreter executes one thread at a time. With a multi-threaded pool, a dispatch that blocks (on I/O, for example) lets another dispatch run, but two dispatches never run in parallel. Your servants must still be thread-safe, since the interpreter can switch from one dispatch to another at any point.

Each dispatch runs in its own task, executed by Swift concurrency's global concurrent executor whatever the size of the Ice thread pool, so dispatches run concurrently and your servants must be safe for concurrent calls. A multi-threaded pool only reads requests and replies in parallel.