Object Serialization Framework

The CommCare code base uses a hand-rolled object serialization framework to ensure compatibility with J2ME (we've since updated to Java 7). This serialization framework has a few details and gotchas that developers need to keep in mind.

Object serialization is a way to persist objects across executions of a program by turning a (Java) object into a series of bytes that can be stored and used to reconstitute the object later.

The ProtocolFactory class

The ProtocolFactory class is responsible for keeping track of all the serializable classes, creating new instances of those classes for deserialization, and mapping classes to a unique hash used for knowing which class to inflate. At the Android level serializable classes (those that implement Externalizable) are gathered at startup by analyzing the dex file and are then registered with the ProtocolFactory instance. Tests and the CLI use the LiveProtocolFactory, which lazily registers serializable classes at serialization time, hence only allowing for classes to be deserialized if they have been serialized during program execution once before.

The Externalizable interface

In order for a class to be serializable it must implement readExternal and writeExternal of the Externalizable interface. The writeExternal class method writes the object's state and writes it to an output stream to serialize the object. The readExternal class method is called on a newly created class instance to instantiate the instance with the state stored in the provided input stream.

For example, if we have an class that represents some boolean data the de/serialization code would be such:

public class BooleanData implements Externalizable {

    private boolean data;

    @SupressWarning("unused")
    public BooleanData() {
        // for serialization
    }

    @Override
    public void readExternal(DataInputStream in, PrototypeFactory pf)
            throws IOException, DeserializationException {
        data = in.readBoolean();
    }

    @Override
    public void writeExternal(DataOutputStream out) throws IOException {
        out.writeBoolean(data);
    }
    
    ...
}

Notes about the code above: When deserializing an object the PrototypeFactory must create a new object instance. To achieve this the class must have an empty public constructor.

When do class modifications requires data migrations?

If you change code for a serializable class you need to check if that change will require a data migration.

For instance, in the Android implementation of CommCare we store serialized Case objects as byte blobs in a database. If you decide to add a new field to the class that you want to persist across serializations, then you will need to modify the deserialization code. This means that when you try to deserialize objects already in the database, which were serialized using the old scheme, things will break. Hence you need to add custom migration code that reads the objects out of the DB using the old scheme and writes them back in using the new scheme. An example of this can be found here

Classes that don't need to be migrated:

• Suite, Menu, Entry: these classes implement Externalizable but the instead of storing instances in a DB they are recreated by parsing the xml at startup.
• FormDef: When an app is installed form xml is parsed and turned into FormDef objects that are then serialized and stored in a database. When forms are opened, these objects are deserialized from the database, but if an error crops up, the code falls back to parsing the form xml to load the form. Hence you don't need to worry about performing migrations of FormDef objects, changing the model will only incur a one-time cost for users of a couple seconds.