Skip to content

v2_0_cpp_adaptor

Jump to bottom
Takatoshi Kondo edited this page Aug 26, 2023 · 24 revisions

adaptors

When you want to pack to msgpack::object from a various type object, you need an adaptor. Converting to msgpack::object from a various type object and vice versa, it requires an adaptor too.

See conversion.

predefined adaptors

msgpack-c provides predefined adaptors for C++ primitive types and standard libraries.

C++ type msgpack::object type note
bool bool
char* str
std::deque array
char positive/negative integer
signed ints *1 positive/negative integer
unsigned ints *2 positive integer
float float32 since 2.1.0
double float64
T[] array since 2.0.0
char[] str since 2.0.0
unsigned char[] bin since 2.0.0
std::list array
std::map array
std::pair array
std::set array
std::string str
std::wstring array of positive integer
std::vector array
std::vector<char> bin
std::vector<unsigned char> bin since 1.2.0
msgpack type msgpack::object type note
nil_t nil
tuple array
array_ref *3 array since 1.2.0
raw_ref *3 bin
v4raw_ref *3 str since 1.2.0
ext ext *4
ext_ref *3 ext *4
variant any type since 1.2.0, boost is required
C++11 type msgpack::object type note
std::array array
std::array<char> bin
std::array<unsigned char> bin since 1.2.0
std::forward_list array
std::tuple array
std::unordered_map array
std::unordered_set array
std::unique_ptr depends on template type since 1.2.0
std::shared_ptr depends on template type since 1.2.0
boost type msgpack::object type note
string_ref str since 1.2.0
string_view str since 2.1.0
optional depends on optional type since 1.2.0
fusion sequence array since 1.2.0

When you use the adaptors for boost containers, you need to define MSGPACK_USE_BOOST.

*1 signed ints signed char, signed short, signed int, signed long, signed long long

*2 unsigned ints unsigned char, unsigned short, unsigned int, signed long, signed long long

*3 xxx_ref types enforce packing types without copies.

*4 ext with maximum size limitation On 64bit environment, there is no limitation. But on 32bit environment, ext32 with maximum size is not supported. See https://github.com/msgpack/msgpack-c/pull/175#issue-51915694

msgpack::object type is defined as follows: https://github.com/msgpack/msgpack/blob/master/spec.md

These adaptors are defined in the following directory: https://github.com/msgpack/msgpack-c/tree/master/include/msgpack/adaptor

defining custom adaptors

classes

intrusive approach

When you want to adapt your class to msgpack, use MSGPACK_DEFINE macro.

macro msgpack::object type note
MSGPACK_DEFINE array or map *3
MSGPACK_DEFINE_ARRAY array since 1.2.0
MSGPACK_DEFINE_MAP map since 1.2.0

*3 When you define MSGPACK_USE_DEFINE_MAP, MSGPACK_DEFINE is adapted to array, otherwise adapted to map.

#include <msgpack.hpp>

struct your_class {
    int a;
    std::string b;
    MSGPACK_DEFINE(a, b);
};

// ...

Let's say a == 42, b == "ABC", the object of your_class is serialized as follows:

// array
[42,"ABC"]
// map (MSGPACK_USE_DEFINE_MAP is defined)
{"a":42,"b":"ABC"}

I use JSON to help understanding. But actual condition is msgpack format.

The macro MSGPACK_DEFINE provides packing, converting to msgpack object with zone, and converting to your_class from msgpack::object functionalities. your_class is packed/converted as msgpack array.

https://github.com/msgpack/msgpack-c/blob/cpp_master/example/cpp03/class_intrusive.cpp

When you want to adapt your class with its base classes ,use MSGPACK_BASE macro.

#include <msgpack.hpp>
struct base1 {
   int a;
   MSGPACK_DEFINE(a);
};
struct base2 {
   int a;
   MSGPACK_DEFINE(a);
};
struct your_class : base1, base2 {
    int a;
    std::string b;
    // You can choose any order. It is represented to the msgpack array elements order.
    MSGPACK_DEFINE(b, MSGPACK_BASE(base2), a, MSGPACK_BASE(base1));
};

// ...
macro msgpack::object type
MSGPACK_BASE array or map *4
MSGPACK_BASE_ARRAY array
MSGPACK_BASE_MAP map

*4 When you define MSGPACK_USE_DEFINE_MAP, MSGPACK_BASE is adapted to array, otherwise adapted to map.

You must use MSGPACK_BASE with MSGPACK_DEFINE, MSGPACK_BASE_ARRAY with MSGPACK_DEFINE_ARRAY, or MSGPACK_BASE_MAP with MSGPACK_DEFINE_MAP. When you use MSGPACK_BASE_MAP, the key is the STR of the base class name.

Let's say base1::a == 1, base2::a == 2, your_class::a == 42, and your_class::b == "ABC", the object of your_class is serialized as follows:

// array
["ABC",[2],42,[1]]
// map (MSGPACK_USE_DEFINE_MAP is defined)
{"b":"ABC","base2":{"a":2},"a":42,"base1":{"a":1}}

I use JSON to help understanding. But actual condition is msgpack format.

When you use an array based adaptor, e.g.)MSGPACK_DEFINE_ARRAY, the order of member variables and base classes is important. The index of the msgpack::object array is corresponding to the member variables order. When you use a map based adaptor, e.g.)MSGPACK_DEFINE_MAP, the names of member variables and base classes is important. The keys of the msgpack::object map is corresponding to member variables names and the base classes names.

since 2.1.0

You can also customize the name of the map key using MSGPACK_NVP. For example, the key of the value b is #b. You can use any strings even if it can't use as C++ variable name. It helps inter programming language data transportation.

#include <msgpack.hpp>

struct your_class {
    int a;
    std::string b;
    MSGPACK_DEFINE_MAP(a, MSGPACK_NVP("#b", b));
};
{"a":42,"#b":"ABC"}

non-intrusive approach

When you don't want to modify your class, you can use non-intrusive approach. msgpack-c provides the following four functor class templates. Those templates are in the namespace msgpack::adaptor.

convert

Converting from msgpack::object to T.

template <typename T>
struct convert {
    msgpack::object const& operator()(msgpack::object const&, T&) const;
};
pack

Packing from T into msgpack::packer.

template <typename T>
struct pack {
    template <typename Stream>
    msgpack::packer<Stream>& operator()(msgpack::packer<Stream>&, T const&) const;
};
object

Set msgpack::object by T.

template <typename T>
struct object {
    void operator()(msgpack::object&, T const&) const;
};
object_with_zone
  • Set msgpack::object::with_zone by T.
template <typename T>
struct object_with_zone {
    void operator()(msgpack::object::with_zone&, T const&) const;
};

In order to add packing/conversion supports for your custom types, you need to specialize the templates.

Here are templates that are specialized to 'your_type'.

template <>
struct convert<your_type> {
    msgpack::object const& operator()(msgpack::object const&, your_type&) const {
        // your implementation
    }
};
template <>
struct pack<your_type> {
    template <typename Stream>
    msgpack::packer<Stream>& operator()(msgpack::packer<Stream>&, your_type const&) const {
        // your implementation
    }
};
template <>
struct object<your_type> {
    void operator()(msgpack::object&, your_type const&) const {
        // your implementation
    }
};
template <>
struct object_with_zone<your_type> {
    void operator()(msgpack::object::with_zone&, your_type const&) const {
        // your implementation
    }
};

You don't need to specialize all the templates. When you want to support pack only, then you can specialze only the pack template.

Example:

#include <msgpack.hpp>

class my_class {
public:
    my_class() {} // When you want to convert from msgpack::object to my_class
                  // using msgpack::object::as fucntion template,
                  // my_class should be default constructible.
    my_class(std::string const& name, int age):name_(name), age_(age) {}

    // my_class should provide getters for the data members you want to pack.
    std::string const& get_name() const { return name_; }
    int get_age() const { return age_; }
private:
    std::string name_;
    int age_;
};

namespace msgpack {
MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS) {
namespace adaptor {

// Place class template specialization here
template<>
struct convert<my_class> {
    msgpack::object const& operator()(msgpack::object const& o, my_class& v) const {
        if (o.type != msgpack::type::ARRAY) throw msgpack::type_error();
        if (o.via.array.size != 2) throw msgpack::type_error();
        v = my_class(
            o.via.array.ptr[0].as<std::string>(),
            o.via.array.ptr[1].as<int>());
        return o;
    }
};

template<>
struct pack<my_class> {
    template <typename Stream>
    packer<Stream>& operator()(msgpack::packer<Stream>& o, my_class const& v) const {
        // packing member variables as an array.
        o.pack_array(2);
        o.pack(v.get_name());
        o.pack(v.get_age());
        return o;
    }
};

template <>
struct object_with_zone<my_class> {
    void operator()(msgpack::object::with_zone& o, my_class const& v) const {
        o.type = type::ARRAY;
        o.via.array.size = 2;
        o.via.array.ptr = static_cast<msgpack::object*>(
            o.zone.allocate_align(sizeof(msgpack::object) * o.via.array.size));
        o.via.array.ptr[0] = msgpack::object(v.get_name(), o.zone);
        o.via.array.ptr[1] = msgpack::object(v.get_age(), o.zone);
    }
};

} // namespace adaptor
} // MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS)
} // namespace msgpack

https://github.com/msgpack/msgpack-c/blob/cpp_master/example/cpp03/class_non_intrusive.cpp

When you use msgpack::object::as member function template in convert class template specilization, , temporary objects are created.

template<>
struct convert<my_class> {
    msgpack::object const& operator()(msgpack::object const& o, my_class& v) const {
        if (o.type != msgpack::type::ARRAY) throw msgpack::type_error();
        if (o.via.array.size != 2) throw msgpack::type_error();
        v = my_class(
            o.via.array.ptr[0].as<std::string>(), // temporary object is created here
            o.via.array.ptr[1].as<int>());
        return o;
    }
};

If you can get the reference of the member variables in converting target class, you can remove the temporary object creation. For example, if member variables of my_class are public, you can apply operator>> to them as follows:

class my_class {
public:
    /* ... */
    std::string name_;
    int age_;
};

template<>
struct convert<my_class> {
    msgpack::object const& operator()(msgpack::object const& o, my_class& v) const {
        if (o.type != msgpack::type::ARRAY) throw msgpack::type_error();
        if (o.via.array.size != 2) throw msgpack::type_error();
        o.via.array.ptr[0] >> v.name_; // no temporary object creation
        o.via.array.ptr[1] >> v.age_;
        return o;
    }
};
accessing private members

non-intrusive approach can't access the target class's private members because of non-intrusive. In order to access private members from adaptor class template specializations, you need to define friend class template declaration in the target class (intrusively) as follows:

class my_class {
    // ...
    template <typename T, typename Enabler>
    friend struct msgpack::MSGPACK_DEFAULT_API_NS::adaptor::pack;

    template <typename T, typename Enabler>
    friend struct msgpack::MSGPACK_DEFAULT_API_NS::adaptor::convert;

    template <typename T, typename Enabler>
    friend struct msgpack::MSGPACK_DEFAULT_API_NS::adaptor::object;

    template <typename T, typename Enabler>
    friend struct msgpack::MSGPACK_DEFAULT_API_NS::adaptor::object_with_zone;

    template <typename T, typename Enabler>
    friend struct msgpack::MSGPACK_DEFAULT_API_NS::adaptor::as;

    // ...
};

Before the version 4.0.3, you need to declare for object adaptor as follows:

    template <typename T, typename Enabler, typename Enabler2>
    friend struct msgpack::MSGPACK_DEFAULT_API_NS::adaptor::object;

You need to declare only the class template specializations that you defined.

non default constructible class support (C++11 only, since 1.2.0)

You might want to convert to a class that doesn't have default constructor from a msgpack::object. In order to do that, you can use 'as' class template specialization.

Here is a non default constructible class:

struct no_def_con {
    no_def_con() = delete;
    no_def_con(int i):i(i) {}
    int i;
    MSGPACK_DEFINE(i);
};

Define 'as' class template specialization as follows:

namespace msgpack {
MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS) {
namespace adaptor {
template <>
struct as<no_def_con> {
    no_def_con operator()(msgpack::object const& o) const {
        if (o.type != msgpack::type::ARRAY) throw msgpack::type_error();
        if (o.via.array.size != 1) throw msgpack::type_error();
        return no_def_con(o.via.array.ptr[0].as<int>());
    }
};
} // adaptor
} // MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS)
} // msgpack

Then, you can convert to your class form msgpack::object as follows:

   msgpack::object o = /*...*/
   no_def_con ndc = o.as<no_def_con>();

NOTE: MSVC2015 doesn't support C++11 feature completely. This feature isn't supported on MSVC2015 due to lack of Expression SFINAE. See https://github.com/msgpack/msgpack-c/issues/343#issuecomment-131654386.

enums

When you want to adapt enum or enum class to msgpack, use MSGPACK_ADD_ENUM macro.

#include <msgpack.hpp>

enum your_enum {
    elem1,
    elem2
};

MSGPACK_ADD_ENUM(your_enum);

enum class your_enum_class {
    elem1,
    elem2
};

MSGPACK_ADD_ENUM(your_enum_class);

// ...

You need to use MSGPACK_ADD_ENUM in the global namespace.

The macro MSGPACK_DEFINE provides packing, converting to msgpack object with or without zone, and converting to your_enum from msgpack::object functionalities. your_enum is packed/converted as msgpack positive/negative integer.

See an example.

Clone this wiki locally