Serialize And Deserialize streams of data using Python and MessagePack

Overview:

MessagePack offers serializing messages using a compact binary form.When used on similar usecases this gives clear advantage over text bases message formats like JSON.
Python data can be serialized using message pack as individual variables or as a stream of data.
This article show how to serialize and deserialize a stream of data using Python and MessagePack.

Data serialized using msgpack.Packer is collected in a bytes stream using Python io.BytesIO.
The data is deserialized using msgpack.Unpacker iteratively as a stream and printed on the console.

# This is a Python example for serialising data using Python BytesIO stream and MessagePack

import msgpack

from io import BytesIO

# A collection of strings need to be serialized

stringTokens = ["Beethovan","Mozart","Mahler","Brahms"]

content = BytesIO()

# Use packer to serialize or just msgpack.packb()

packer = msgpack.Packer()

for token in stringTokens:

serialized = packer.pack(token)

print("Serialized: {}".format(serialized))

content.write(serialized)

# Create a new stream

# in case when deserializing through a separate program

# as below

# content1 = BytesIO(content.getvalue())

# or

# just move to the beginning of the stream

content.seek(0)

# Start unpacking using Unpacker

# Unpacker is kind of real stream....compared to packer

unpacker = msgpack.Unpacker(content, raw=False)

for unpacked in unpacker:

print("Deserialized: {}".format(unpacked))

Serialized: b'\xa9Beethovan'

Serialized: b'\xa6Mozart'

Serialized: b'\xa6Mahler'

Serialized: b'\xa6Brahms'

Deserialized: Beethovan

Deserialized: Mozart

Deserialized: Mahler

Deserialized: Brahms