Skip to content

Question: Forward compatibility between encoder code and schema version #154

Open

Description

@estaban

I am using SBE to pass data between 2 services: service A encodes the message and service B decodes it. I have 3 code packages:

  • Package 1 (used by service A and B): contains the codec code generated by the SBE tool
  • Package 2 (used by service A): the client code, that encodes the message
  • Package 3 (used by service B): the server code, that decodes the message

At the moment, I have version 0 of the message has no body. I am adding a new variable length field in version 1 of the message.

Let's assume I want to update service B to use the new message version. I update the decoding code in package 3 to deserialize the new field and push a change in both packages 1 and 3.

While I pushed the change in package 1, but have not yet update the encoder code in package 2, then package 2 will use message version 1, but will not encode the new field. This leads the receiver to fail decoding the message when trying to decode the length of the variable length field.

Is there a best practice to avoid this situation other than package versioning? Should I add a build-time check asserting that the message version used in package 2 is 0, so that I don't end up with package 1 ahead of package 2?

Activity

  1. changed the title [-]Ensure encode code and schema version are in sync[/-] [+]Question: Ensure the encoder code is in sync with the schema version[/+] on Aug 23, 2022
  2. changed the title [-]Question: Ensure the encoder code is in sync with the schema version[/-] [+]Question: Forward compatibility between encoder code and schema version[/+] on Aug 23, 2022
  3. donmendelson commented on Oct 4, 2022

    @donmendelson
    Member

    The SBE v1.0 specification states:

    The <messageSchema> root element contains a version number attribute. By default, version is zero, the initial version of a message schema. Each time a message schema is changed, the version number is incremented.

    Version applies to the schema as a whole, not to individual elements. Version is sent in the message header so the consumer can determine which version of the message schema was used to encode the message.

    In short, you cannot mix versions within a schema. However, it is generally valid to use more than one SBE schema over a session. The receiver would be able to read the schema, template, and version numbers in each message header and therefore, know which decoder to use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions