Asynchronous Method Dispatch (AMD) in C++
The number of simultaneous synchronous requests a server is capable of supporting is determined by the number of threads in the server's thread pool. If all of the threads are busy dispatching long-running operations, then no threads are available to process new requests and therefore clients may experience an unacceptable lack of responsiveness.
Asynchronous Method Dispatch (AMD), the server-side equivalent of AMI, addresses this scalability issue. Using AMD, a server can receive a request but then suspend its processing in order to release the dispatch thread as soon as possible. When processing resumes and the results are available, the server sends a response explicitly using a callback object provided by the Ice run time.
AMD is transparent to the client, that is, there is no way for a client to distinguish a request that, in the server, is processed synchronously from a request that is processed asynchronously.
In practical terms, an AMD operation typically queues the request data (i.e., the callback object and operation arguments) for later processing by an application thread (or thread pool). In this way, the server minimizes the use of dispatch threads and becomes capable of efficiently supporting thousands of simultaneous clients.
An alternate use case for AMD is an operation that requires further processing after completing the client's request. In order to minimize the client's delay, the operation returns the results while still in the dispatch thread, and then continues using the dispatch thread for additional work.
Enabling AMD
To enable asynchronous dispatch, you must add an ["amd"] metadata directive to your Slice definitions. The directive applies at the interface and the operation level. If you specify ["amd"] at the interface level, all operations in that interface use asynchronous dispatch; if you specify ["amd"] for an individual operation, only that operation uses asynchronous dispatch. In either case, the metadata directive replaces synchronous dispatch, that is, a particular operation implementation must use synchronous or asynchronous dispatch and cannot use both.
Consider the following Slice definitions:
["amd"] interface I
{
bool isValid();
float computeRate();
}
interface J
{
["amd"] void startProcess();
int endProcess();
}In this example, both operations of interface I use asynchronous dispatch, whereas, for interface J, startProcess uses asynchronous dispatch and endProcess uses synchronous dispatch.
AMD Mapping
The ["amd"] metadata changes the name of the skeleton’s pure virtual function to <operation-name>Async. This function returns void and accepts the operation's in-parameters, followed by two callback parameters provided by the Ice runtime.
For example, suppose we have defined the following operation:
interface Example
{
["amd"] string op(short s, out long l);
}Operation op is mapped as follows:
virtual void opAsync(
std::int16_t s,
std::function<void(std::string, std::int64_t)> response,
std::function<void(std::exception_ptr)> exception,
const Ice::Current& current) = 0;AMD Exceptions
There are two processing contexts in which the logical implementation of an AMD operation may need to report an exception: the dispatch thread (the thread that receives the request), and the response thread (the thread that sends the response).
These are not necessarily two different threads: it is legal to send the response from the dispatch thread.
The implementation of the Async function in your servant class can throw an exception synchronously: it’s equivalent to calling the exception callback with this exception.
Chaining AMI and AMD Invocations
Since the asynchronous proxy callback API and the asynchronous dispatch API are similar, it is possible to implement an asynchronous dispatch by sending an asynchronous request to a proxy.
Continuing our example from the previous section, suppose our servant also holds a proxy to another object of the same type:
class ExampleServant : public Example
{
public:
ExampleServant(ExamplePrx prx) : _other{std::move(prx)}
{
}
void opAsync(
std::int16_t s,
std::function<void(std::string, std::int64_t)> response,
std::function<void(std::exception_ptr)> exception,
const Ice::Current&) override
{
// Ice-supplied AMD response and exception callbacks are passed
// as AMI callbacks.
_other.opAsync(s, std::move(response), std::move(exception));
}
private:
const ExamplePrx _other;
};
Oneway Proxy
If your AMD implementation uses a oneway proxy, remember that the AMI response callback is not called: you need to call the AMD response from the AMI sent callback.