In a recent article, I presented rules for evolving JSON Schemas in a backward, forward, and fully compatible manner. The backward compatibility rules for Protocol Buffers (Protobuf) are much simpler, and most of them are outlined in the section “Updating A Message Type” in the Protobuf Language Guide.
The Confluent Schema Registry uses these rules to check for compatibility when evolving a Protobuf schema. However, unlike some other tools, the Confluent Schema Registry does not require that a removed field be marked as reserved when evolving a schema in a backward compatible manner. The reserved keyword can be used to prevent incompatible changes in future versions of a schema. Instead of requiring the reserved keyword, the Confluent Schema Registry achieves compatibility across multiple versions by allowing compatibility checks to be performed transitively. The reserved keyword is most useful for those tools that do not perform transitive compatibility checks. Using it when removing fields is still considered a best practice, however.
For the Protobuf oneof construct, there are four additional compatibility issues that are mentioned in the Protobuf Language Guide. Since the text of the language guide is a bit terse, for the rest of this article I’ll discuss each of these four issues further. First, let me reproduce the text below.
- Be careful when adding or removing oneof fields. If checking the value of a oneof returns
None/NOT_SET, it could mean that the oneof has not been set or it has been set to a field in a different version of the oneof. There is no way to tell the difference, since there’s no way to know if an unknown field on the wire is a member of the oneof. - Move fields into or out of a oneof: You may lose some of your information (some fields will be cleared) after the message is serialized and parsed. However, you can safely move a single field into a new oneof and may be able to move multiple fields if it is known that only one is ever set.
- Delete a oneof field and add it back: This may clear your currently set oneof field after the message is serialized and parsed.
- Split or merge oneof: This has similar issues to moving regular fields.
Each of the above issues can cause information to be lost. For that reason, these are considered backward compatibility issues.
Adding or removing oneof fields
Let’s look at an example of when a oneof field is removed. Let’s say we have the following Protobuf message with fields f1 and f2 in a oneof.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
string f2 = 2;
}
}
Later we decide to remove the field f2.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
}
}
If an old binary with the first version of SampleMessage creates a message and sets f2 in test_oneof, then a new binary with the second version of SampleMessage will see test_oneof as unset. The value of f2 will be contained in the unknown fields of the message, but there’s no way to tell which, if any, of the unknown fields were previously in test_oneof. The information that is lost is whether the oneof was really set or not.
This behavior may be unexpected when comparing it to how Protobuf handles removing a value from an enum. Any unrecognized enum values encountered on the wire are retained by the binary, and a field using the enum will still appear as set (to an unrecognized value) if the binary encounters a value that is no longer enumerated in the enum.
The information loss from removing a oneof field can cascade and cause other information to be lost, depending on the scenario. For example, if the new binary reads the message with f2 set by the old binary and modifies the same message by setting f1, then when the old binary reads the modified message, the old binary may see f2 as still set! In this scenario, f2 can still appear as set because setting f1 in the new binary does not clear the unknown field for f2, since for the new binary, f2 is not associated with test_oneof. When reading from the wire, the old binary may read f2 after f1, and then clear f1. This assumes that f2 is deserialized after f1. The value of f1 will also be lost for the old binary.
Furthermore, when reading the modified message, a binary in a different programming language may read f1 after f2, thus clearing the value of f2. In this case, the value for f2 will be lost. The actual serialization order is implementation-specific and subject to change. Most implementations serialize fields in ascending order of their field numbers, but this is not guaranteed, especially in the presence of unknown fields. Also, inconsistencies in how various programming languages handle unknown fields prevents a canonical serialization order from being defined. The Protobuf Encoding documentation states the following:
- By default, repeated invocations of serialization methods on the same protocol buffer message instance may not return the same byte output; i.e. the default serialization is not deterministic.
- Deterministic serialization only guarantees the same byte output for a particular binary. The byte output may change across different versions of the binary.
So even a later version of the binary in the same programming language may see different results.
For these reasons, removing a field from a oneof is considered a backward incompatible change. Likewise, adding a field to a oneof is considered a forward incompatible change (since a forward compatibility check is just a backward compatibility check with the schemas switched).
Moving fields into or out of a oneof
Consider the following Protobuf message with f1 in a oneof and f2 and f3 outside of the oneof.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
}
string f2 = 2;
string f3 = 3;
}
Later we decide to move both f2 and f3 into test_oneof.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
string f2 = 2;
string f3 = 3;
}
}
If an old binary with the first version of SampleMessage creates a message and sets f1, f2, and f3, then when a new binary with the second version of SampleMessage reads the message from the wire, it will clear two of the fields (depending on serialization order) and leave only one field, say f3, with its value. Thus the values of the other two fields will be lost.
As mentioned, the order in which fields are serialized is implementation-specific. For the fields in the oneof, only the value of the last field read from the wire is retained. Therefore, an additional problem is that a different field in the oneof may appear as set when using a different implementation.
For these reasons, moving fields into a oneof is considered a backward incompatible change, unless the oneof is new and only a single field is moved into it. Moving fields out of a oneof is also a backward incompatible change, since this has the effect of removing the fields from the oneof.
Deleting a oneof and adding it back
In this scenario there are three versions involved. The first version has f1 and f2 in a oneof.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
string f2 = 2;
}
}
In the second version, we remove the field f2 from the oneof.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
}
}
Later we add the field f2 back to the oneof.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
string f2 = 2;
}
}
If an old binary with the first version of SampleMessage creates a message and sets f2 in test_oneof, then next a binary with the second version reads the message and sets f1 in test_oneof, and finally a later binary with the third version reads the modified message, it may see f2 as set, and the value of f1 as lost. This is a similar scenario to that described above involving only two binaries when removing a field from the oneof. Here the modified message is being read by a third later binary, rather than by the first original binary in the scenario involving only two binaries.
Splitting or merging a oneof
Consider a Protobuf message with two oneof constructs.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
}
oneof test_oneof {
string f2 = 2;
}
}
Later we decide to merge the oneof constructs.
message SampleMessage {
oneof test_oneof {
string f1 = 1;
string f2 = 2;
}
}
If an old binary with the first version of SampleMessage creates a message and sets both f1 and f2, then when a new binary with the second version of SampleMessage reads the message, it will see only one of the fields as set, with the value of the other field being lost. Again, which field is set and which is lost depends on the implementation. This issue is similar to that described above involving moving existing fields into a oneof.
Summary
In Protobuf, evolving schemas with oneof constructs can be problematic for a couple of reasons:
- Information may be lost, such as the values of fields that were moved into a oneof, and whether a oneof was set or not. The information loss can cascade and result in further loss, such as when an unknown field reappears in a oneof, causing the previous value of the oneof to be cleared.
- A schema change can unfortunately cause multiple fields which have been set to be pulled into the same oneof. If this happens, all of the fields except one will be cleared. Since the order in which fields are serialized to the wire is implementation-specific and subject to change, especially in the presence of unknown fields, the field that is retained may be different between implementations.
For these reasons, Confluent Schema Registry implements two backward compatibility checks for the oneof construct:
- Removing a field from a oneof is a backward incompatible change.
- Moving existing fields into a oneof is a backward incompatible change, unless it is a new oneof with a single field.