Multiprocessing.Queue

Overview:

  • The Queue class in Multiprocessing module of Python Standard Library provides a mechanism to pass data between a parent process and the descendent processes of it.
  • Multiprocessing.Queues.Queue uses pipes to send data between related * processes.

A Pipe is a message passing mechanism between processes in Unix-like operating systems.

  • For any read/write (get/put method calls) operation the Multiprocessing.Queues.Queue class deserialises/serializes data using Python's pickle module and sends through the underlying pipe.
  • Python's Multiprocessing.Queues.Queue also uses semaphores and locks to take care of the scenarios involving multiple reader/writer processes.

 

Example:

import multiprocessing

import time

 

# Producer/Writer

def procFunction0(messageQueue):

    for i in range(1,10):

        messageQueue.put("Child1:Message%d"%i)

        time.sleep(1)

 

# Consumer/Reader

def procFunction1(messageQueue):

    while messageQueue.empty() is False:

        print("From reader:%s"%messageQueue.get())

        time.sleep(1)

 

# Producer/Writer

def procFunction2(messageQueue):

    for i in range(1,10):

        messageQueue.put("Child3:Message%d"%i)

        time.sleep(1)

 

if __name__ == "__main__":

 

    multiprocessing.set_start_method("fork")

   

    messageQueue  = multiprocessing.Queue()

   

    # Create child processes

    childProcess0 = multiprocessing.Process(target=procFunction0, args=(messageQueue,))

    childProcess1 = multiprocessing.Process(target=procFunction1, args=(messageQueue,))

    childProcess2 = multiprocessing.Process(target=procFunction2, args=(messageQueue,))

 

    # Start all the child processes - Writer, Reader, Writer

    childProcess0.start()

    childProcess1.start()

    childProcess2.start()

 

    # Wait for child processes to finish

    childProcess0.join()

    childProcess1.join()

    childProcess2.join()

 

 

Output:

From reader:Child1:Message1

From reader:Child3:Message1

From reader:Child1:Message2

From reader:Child3:Message2

From reader:Child1:Message3

From reader:Child3:Message3

From reader:Child1:Message4

From reader:Child3:Message4

From reader:Child3:Message5

From reader:Child1:Message5

From reader:Child1:Message6

From reader:Child3:Message6

From reader:Child1:Message7

From reader:Child3:Message7

From reader:Child1:Message8

From reader:Child3:Message8

From reader:Child3:Message9

From reader:Child1:Message9

 


Copyright 2020 © pythontic.com