Skip to content

v2_0_cpp_packer

Jump to bottom
Kenvi Zhu edited this page Sep 27, 2021 · 2 revisions

msgpack::packer

Supported types

msgpack::packer packs any data to msgpack format. Currently, the following formats are supported:

https://github.com/msgpack/msgpack-c/tree/cpp_master/include/msgpack/adaptor

In addition, you can pack msgpack::object.

You can add your adaptor class template specialization to support packing objects that have the type you want to pack.

Overload declaration:

#include <msgpack.hpp>

namespace msgpack {
MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS) {
namespace adaptor {

template <>
struct pack<the_type_you_want_to_pack> {
    template <typename Stream>
    msgpack::packer<Stream>& operator()(msgpack::packer<Stream>& o, the_type_you_want_to_pack const& v) const {
        // packing implementation.
        // o.pack_???(v.???);
        return o;
    }
};
} // namespace adaptor
} // MSGPACK_API_VERSION_NAMESPACE(MSGPACK_DEFAULT_API_NS)
} // namespace msgpack

You can call packing member functions of packer at your packing implementation. See pack manually.

See also adaptor

Buffer

You can use any classes that have the following member function:

write(const char*, std::size_t);

In the C++ standard library, std::stringstream has it.

std::tuple<int, bool, std::string> src(1, true, "example");
std::stringstream buffer;
msgpack::pack(buffer, src); // calls std::stringstream::write internally.

msgpack provides some buffers:

sbuffer

sbuffer is simple buffer. It uses malloc, free, and realloc internally. When msgpack::pack is called, src is copied to buffer. realloc is faster than new, copy, and delete operation.

vrefbuffer

vrefbuffer is a kind of reference buffer. When msgpack::pack is called, src might be referenced by buffer. So you can avoid copies, but you need to keep src objects until buffer's liftime will be finished.

vrefbuffer has a threshold value named m_ref_size. If the size of serialized src object is greater or equal to threashold value, the object is referenced.

vrefbuffer.hpp

    void write(const char* buf, size_t len)
    {
        if(len < m_ref_size) {
            append_copy(buf, len);
        } else {
            append_ref(buf, len);
        }
    }
#ifndef MSGPACK_VREFBUFFER_REF_SIZE
#define MSGPACK_VREFBUFFER_REF_SIZE 32
#endif

#ifndef MSGPACK_VREFBUFFER_CHUNK_SIZE
#define MSGPACK_VREFBUFFER_CHUNK_SIZE 8192
#endif

namespace detail {
    // int64, uint64, double
    std::size_t const packer_max_buffer_size = 9;
} // detail

class vrefbuffer {
public:
    // ref_size is stored to m_ref_size
    vrefbuffer(size_t ref_size = MSGPACK_VREFBUFFER_REF_SIZE,
               size_t chunk_size = MSGPACK_VREFBUFFER_CHUNK_SIZE)
        :m_ref_size(std::max(ref_size, detail::packer_max_buffer_size + 1)),
         m_chunk_size(chunk_size)
    {
        ...
    }
};

You can pass the threshold size on the constructor, but if you give the ref_size that is less than 10, vrefbuffer::ref_size, actual threshold value whould be 10. Because, the maximum size of msgpack format fixed object is 9 and their buffers are allocated on the stack as follows:

pack.hpp

template <typename Stream>
inline packer<Stream>& packer<Stream>::pack_fix_int64(int64_t d)
{
    char buf[9];
    buf[0] = static_cast<char>(0xd3); _msgpack_store64(&buf[1], d);
    append_buffer(buf, 9);
    return *this;
    // buf's lifetime is finished, so can't use vrefbuffer::append_ref()
}

zbuffer

The contents of zbuffer is commpressed using zlib.

fbuffer

The contents of fbuffer is stored to file using c-style FILE*.

pack

When you want to pack your data, call pack(). The first argument is a buffer, and the second argument is the value you want to pack. You can pass any supported types.

template <typename Stream, typename T>
inline void pack(Stream& s, const T& v);

pack manually

You can also pack manually. See the following example:

msgpack::sbuffer sbuf;
msgpack::packer<msgpack::sbuffer> packer(sbuf);
char c[] = { 1, 2, 3, 4, 5, 6 };
packer.pack_bin(sizeof(c));         // pack header and size
packer.pack_bin_body(c, sizeof(c)); // pack payload

msgpack::packer provides the following manual packing functions. These functions are usually used when you implement custom type support by operator<< (packer& o, const the_type_you_want_to_pack& v). Of course, you can use the unpacking functions directly in your application.

class packer {
public:
    packer<Stream>& pack_uint8(uint8_t d);
    packer<Stream>& pack_uint16(uint16_t d);
    packer<Stream>& pack_uint32(uint32_t d);
    packer<Stream>& pack_uint64(uint64_t d);
    packer<Stream>& pack_int8(int8_t d);
    packer<Stream>& pack_int16(int16_t d);
    packer<Stream>& pack_int32(int32_t d);
    packer<Stream>& pack_int64(int64_t d);

    packer<Stream>& pack_fix_uint8(uint8_t d);
    packer<Stream>& pack_fix_uint16(uint16_t d);
    packer<Stream>& pack_fix_uint32(uint32_t d);
    packer<Stream>& pack_fix_uint64(uint64_t d);
    packer<Stream>& pack_fix_int8(int8_t d);
    packer<Stream>& pack_fix_int16(int16_t d);
    packer<Stream>& pack_fix_int32(int32_t d);
    packer<Stream>& pack_fix_int64(int64_t d);

    packer<Stream>& pack_char(char d);
    packer<Stream>& pack_signed_char(signed char d);
    packer<Stream>& pack_short(short d);
    packer<Stream>& pack_int(int d);
    packer<Stream>& pack_long(long d);
    packer<Stream>& pack_long_long(long long d);
    packer<Stream>& pack_unsigned_char(unsigned char d);
    packer<Stream>& pack_unsigned_short(unsigned short d);
    packer<Stream>& pack_unsigned_int(unsigned int d);
    packer<Stream>& pack_unsigned_long(unsigned long d);
    packer<Stream>& pack_unsigned_long_long(unsigned long long d);

    packer<Stream>& pack_float(float d);
    packer<Stream>& pack_double(double d);

    packer<Stream>& pack_nil();
    packer<Stream>& pack_true();
    packer<Stream>& pack_false();

    packer<Stream>& pack_array(size_t n);

    packer<Stream>& pack_map(size_t n);

    packer<Stream>& pack_str(size_t l);
    packer<Stream>& pack_str_body(const char* b, size_t l);

    packer<Stream>& pack_bin(size_t l);
    packer<Stream>& pack_bin_body(const char* b, size_t l);

    packer<Stream>& pack_ext(size_t l, int8_t type);
    packer<Stream>& pack_ext_body(const char* b, size_t l);
};

packing classes

msgpack provides a convenient macro to adapt your class to pack function.

class my_class {
public:
    std::string value;
    int i;
    MSGPACK_DEFINE(value, i); // write the member variables that you want to pack
};

If the class has MSGPACK_DEFINE() macro, you can pack the class, you can convert the class to msgpack::object, and you can convert msgpack::object to the class.

packing enums

msgpack provides a convenient macro to adapt enum variables to pack function. When the class you want to pack has the enum member variables, you need to tell msgpack which is the enum type you want to pack.

#include <msgpack.hpp>

class TestEnumMemberClass
{
public:
  TestEnumMemberClass()
    : t1(STATE_A), t2(STATE_B), t3(STATE_C) {}

  enum TestEnumType {
    STATE_INVALID = 0,
    STATE_A = 1,
    STATE_B = 2,
    STATE_C = 3
  };
  TestEnumType t1;
  TestEnumType t2;
  TestEnumType t3;

  MSGPACK_DEFINE(t1, t2, t3);
};

// This macro should be written in the global namespace
MSGPACK_ADD_ENUM(TestEnumMemberClass::TestEnumType);

// ...
// pack enums

After MSGPACK_ADD_ENUM() definition, you can pack the enums.

See an example.

See also adaptor

Clone this wiki locally