Data Encoding for Optional Values

Slice definitions can use optional values as parameters and as fields. To make optional values truly useful, the receiver needs the ability to gracefully ignore an optional value that it does not recognize. Although the Ice encoding omits type information and therefore is not self-describing, the encoding for optional values must include enough information for a receiver to determine how many bytes an optional value occupies so that it can skip to the next value in the stream. Despite this requirement, the encoding rules minimize the overhead associated with optional values, as you will see below.

Overview of the Optional Value Encoding

The encoding for optional parameters and fields follows these general rules:

• Each optional value has a corresponding integer tag that uniquely identifies it.
• Optional values always appear after any required values.
• Optional values must be sorted by their tags. (Unlike required values, order of declaration does not affect optional values.)
• An optional value is encoded only if the sender has supplied a value.

Optional fields of a class or exception appear in the slice after any required fields and before the indirection table, if present. The slice flags must indicate the presence of optional fields, which are included in the byte count for the slice. If a slice contains optional fields, the byte value 255 must be written after the last optional field; this marker denotes the end of the optional fields and is also included in the byte count for the slice.

The encoding does not use an explicit end marker for optional parameters; the end of the encapsulation also marks the end of any optional parameters.

An optional value is encoded as the tuple <type, tag, value>, where type is the optional type that tells the receiver how to determine the number of bytes occupied by the value. The value itself is marshaled using the standard Ice encoding rules for its Slice type.

Encoding for Optional Types and Tags

The first byte of an encoded optional value includes the optional type, and may also include the tag. The optional type occupies the first three bits of this byte, as described in the table below:

The next five bits of the leading byte contain the tag, but only if the tag value is less than 30. Otherwise, the next five bits contain the value 30 as a marker to indicate that the tag value is encoded as a size starting with the next byte. As you can see, using tag values in the range 0 to 29 produces the most compact encoding.

Variable-size types whose encoded size cannot be determined in advance use the FSize optional type, where "FSize" denotes a leading fixed-length (32-bit) size. For these types, the sender reserves four bytes to hold the size, encodes the value as usual, then replaces the four bytes with the actual encoded size. This strategy avoids the need to shift the encoded data in the buffer, at the expense of potentially consuming more bytes than necessary to encode the size.

Fixed-size types use the VSize optional type, where "VSize" denotes a leading variable-length size. The sender can determine the encoded size of these types in advance, and therefore encodes it as a size followed by the value as usual.

Strings also use the VSize optional type but do not require an additional size because the string encoding already includes a leading size. The same is true for sequences of elements of size 1, such as a sequence of bool, a sequence of byte or a sequence of a struct with a single bool or byte field.

The optional type Class represents a class reference or inline instance. Note that a receiver must decode an inline instance even if it does not recognize the tag value because the instance may be referenced by other parameters.

The following table describes the encoding of Slice types:

Examples of Optional Value Encoding

The examples presented below demonstrate the encoding for optional values with typical use cases.

Optional Parameters Example

The following Slice operation makes use of optional input and output parameters:

bool op1(byte b,
         optional(2) string name,
         short sh,
         optional(1) long count,
         out double d,
         out optional(300) Object* p);

Suppose the parameters have the values shown in the table below:

The parameters of the outgoing request are encoded in an encapsulation with all required parameters first, in order of declaration, followed by the optional parameters sorted by tag:

Using tag values less than 30 allows the encoding to combine the optional type and the tag into the same byte using the expression (Tag << 3) + Type.

Now consider the contents of the reply message:

The body of the reply message contains the out parameter d, the return value, and the optional parameter p. The tag value 300 is too large to combine with the optional type, therefore it appears immediately following the optional type encoded as a size. A proxy value uses the FSize optional type, meaning a 32-bit integer precedes the encoded value to specify its size.

The server here supplies a nil value for the optional proxy parameter. The client must not interpret this to mean that the optional parameter is unset; rather, the parameter is set, it just happens to be set to a nil value. If the server had supplied no value for the parameter, it would not appear in the encoding at all.

As an example, if the client does not recognize the optional proxy parameter, it can skip the parameter as follows:

1. Extract the byte containing the optional type
2. Examine the first three bits to determine the type
3. Examine the five remaining bits to determine the tag's status
4. A value of 30 means the tag is encoded separately
5. Extract the tag as a size
6. The optional type FSize means a 32-bit integer holds the value's size; extract the integer
7. Skip ahead the specified number of bytes
8. If we have reached the end of the encapsulation, there are no more optional parameters remaining

Optional Fields Example

The only difference between the encoding for optional parameters and the encoding for optional fields is the latter uses a marker byte to signify the end of optional parameters in a slice. Consider the following definitions:

struct Color
{
    short red;
    short green;
    short blue;
}

class Shape
{
    optional(1) string label;
}

class Rectangle extends Shape
{
    int width;
    int height;
    optional(10) Color fill;
    optional(9) Color border;
    optional(11) float scale;
}

Suppose we are encoding an instance of Rectangle with the field values shown in the table below:

Using the sliced format, the instance data looks as follows:

Notice the use of an end marker (byte value 255) denoting the end of the optional fields in each slice.

See Also

• Basic Data Encoding
• Data Encoding for Classes