Dumping an ndarray object

Overview:

An ndarray object can be stored in several forms and at several destinations.
An ndarray object can be:
- converted to a list
- converted to Python bytes
- written to a file in either text or binary form
- converted to a string containing the pickle of the ndarray object
- pickled and written to a file
There are several methods provided by NumPy for an ndarray object to transform it into various forms listed above which include tolist(), tostring(), tobytes(), tofile(), dumps() and dump().

Converting an ndarray into a python list object:

An ndarray can be converted into a Python list object, using the tolist() method of the numpy.ndarray.
The nesting of the output list depends on the shape of the ndarray object.

Example:

import numpy as np

# Convert a 2d array into a list

array_2d = np.arange(9).reshape(3, 3)

print("2-dimensional array of type ndarray:")

print(array_2d)

print("2-dimensional array from an ndarray object converted to a list:")

print(array_2d.tolist())

Output:

2-dimensional array of type ndarray:

[[0 1 2]

[3 4 5]

[6 7 8]]

2-dimensional array from an ndarray object converted to a list:

[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

Converting an ndarray into bytes:

Both tostring() and tobytes() method of numpy.ndarray can be used for creating a byte array from string.
The tostring() and tobytes() methods return a python bytes object which is an immutable sequence of bytes.
The memory layout of the bytes returned by tostring() and tobytes() methods can be in continuously arranged ‘C’ style or continuously arranged ‘C’ Fortran style as per the values passed on to the parameter order.

Example:

import numpy as np

# Convert a 2d array into a string of bytes

array_2d = np.arange(9).reshape(3, 3)

print("2-dimensional array of type ndarray:")

print(array_2d)

print("2-dimensional array from an ndarray object converted to a string of bytes:")

print(array_2d.tostring())

Output:

2-dimensional array of type ndarray:

[[0 1 2]

[3 4 5]

[6 7 8]]

2-dimensional array from an ndarray object converted to a string of bytes:

b'\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\

x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00

\x00\x00\x06\x00\x00\x00\x00\x00\x00\x00\x07\x00\x00\x00\x00\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00'

Writing ndarray into a file:

An ndarray object can be written to a file using the method tofile().
The ndarray objects are written to files in two formats.
- Text format
- Binary Format
By default the tofile() method writes the ndarray object to a file in binary format.
To change to text format, the parameter sep has to be used.
The parameter sep tells the file is being written in text mode and what is the separator character being used.
The tofile() method does not take care of preserving endianness while writing the ndarray object to the file.

Reading an ndarray from a file:

The contents of an ndarray object written to a file using the method tofile() can be read into a ndarray object using the method numpy.fromfile().
While reading from a file which has the contents of and ndarray object, the separator character and the type of the ndarray elements need to be specified.

Example:

import numpy as np

# Convert a 2d array into a string of bytes

array_2d = np.arange(9).reshape(3, 3)

filePath1 = "./ArrayFile.txt"

filePath2 = "./ArrayFile.bnf"

print("2-dimensional array of type ndarray:")

print(array_2d)

print("Writing the ndarray object to a file in text format:")

array_2d.tofile(filePath1, sep="|")

print("Done writing in text format")

print("Write the ndarray object to a file in binary format:")

array_2d.tofile(filePath2)

print("Done writing in binary format")

print("Reading ndarray from file1:")

array1 = np.fromfile(filePath1, dtype = 'int', sep='|')

print("Type of array1 is %s"%(type(array1)))

print(array1)

print("Reading ndarray from file2:")

array2 = np.fromfile(filePath2, dtype = 'int')

print("Type of array2 is %s"%(type(array2)))

print(array2)

Output:

2-dimensional array of type ndarray:

[[0 1 2]

[3 4 5]

[6 7 8]]

Writing the ndarray object to a file in text format:

Done writing in text format

Write the ndarray object to a file in binary format:

Done writing in binary format

Reading ndarray from file1:

Type of array1 is <class 'numpy.ndarray'>

[0 1 2 3 4 5 6 7 8]

Reading ndarray from file2:

Type of array2 is <class 'numpy.ndarray'>

[0 1 2 3 4 5 6 7 8]

Pickling ndarray object into a string:

An numpy.ndarray object can be pickled into a python string object using the method dumps() of the numpy.ndarray.
The string which has the pickle can be converted again into an ndarray object using the loads() method of NumPy.

Example:

import numpy as np

# Create a 3-dimensional array

array3d = np.arange(8).reshape(2,2,2)

print("Original ndarray object:")

print("======================")

print(array3d)

# Pickle the 3-dimensional array

pickledNDArray = array3d.dumps()

print("Pickle of the ndarray:")

print("======================")

print(pickledNDArray)

# Load back the 3-dimensional array

array1 = np.loads(pickledNDArray)

print("Pickle loaded back:")

print("======================")

print(array1)

Output:

Original ndarray object:

======================

[[[0 1]

[2 3]]

[[4 5]

[6 7]]]

Pickle of the ndarray:

======================

b'\x80\x02cnumpy.core.multiarray\n_reconstruct\nq\x00cnumpy\nndarray\nq\x01K\x00\x85q\x02c_codecs\nencode\nq\x03X\x01\

x00\x00\x00bq\x04X\x06\x00\x00\x00latin1q\x05\x86q\x06Rq\x07\x87q\x08Rq\t(K\x01K\x02K\x02K\x02\x87q\ncnumpy\ndtype\nq\

x0bX\x02\x00\x00\x00i8q\x0cK\x00K\x01\x87q\rRq\x0e(K\x03X\x01\x00\x00\x00<q\x0fNNNJ\xff\xff\xff\xffJ\xff\xff\xff\xffK\

x00tq\x10b\x89h\x03X@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\

x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x06\x00\x0

0\x00\x00\x00\x00\x00\x07\x00\x00\x00\x00\x00\x00\x00q\x11h\x05\x86q\x12Rq\x13tq\x14b.'

Pickle loaded back:

======================

[[[0 1]

[2 3]]

[[4 5]

[6 7]]]

Pickling ndarray object into a file:

An ndarray object can be pickled into a file using the dump() method of nump.ndarray class.
The pickled ndarray object can be loaded from the file and a new ndarray object can be created using the load() method of NumPy.

Example:

import numpy as np

# pickle file path for the ndarray object

pickleFilePath = "./ArrayPickle.pkl"

# create an ndarray with a dimension of 3 x 3

array_2d = np.arange(9).reshape(3,3)

print(array_2d)

# write the pickle of the ndarray object to the disk file

array_2d.dump(pickleFilePath)

# read the pickle from the file

arrayFromPickle = np.load(pickleFilePath)

# print the type of the object

print("Type of the object retrieved from the pickle file:")

print(type(arrayFromPickle))

# print the contents of the ndarray

print("numpy.ndarray object retrieved from the pickle file:")

print(arrayFromPickle)

Output:

[[0 1 2]

[3 4 5]

[6 7 8]]

Type of the object retrieved from the pickle file:

numpy.ndarray object retrieved from the pickle file:

[[0 1 2]

[3 4 5]

[6 7 8]]