Python: Multiprocessing w/ params

Programming
,
1

Can anyone help me with this problem?

I am trying to run two Python functions, func1() and func2(), in parallel. To do this, I have used the accepted answer from Python: How can I run python functions in parallel? which uses the multiprocessing module.

When I use the example code with no parameters, the functions run in parallel:

from multiprocessing import Process

def func1():
   print('func1: starting')
   for i in range(10000000): pass
   print('func1: finishing')

def func2():
   print('func2: starting')
   for i in range(10000000): pass
   print('func2: finishing')

def runInParallel(*fns):
   proc = []
   for fn in fns:
      p = Process(target=fn)
      p.start()
      proc.append(p)
   for p in proc:
      p.join()

if __name__ == '__main__':
   runInParallel(func1, func2)

$ python multi-process.py
func1: starting
func2: starting
func2: finishing
func1: finishing

However, when I pass parameters to the functions, the functions do not run in parallel:

def func1(a): 
   ......

def func2(b):
   ......

if __name__ == '__main__':
   runInParallel(func1(1), func2(2))

$ python multi-process.py 
func1: starting
func1: finishing
func2: starting
func2: finishing

Why is this happening? How can I get the functions to run in parallel when passing parameters?

2

This is happening because the functions are no longer being passed as arguments to runInParallel(). Instead, func1(1) and func2(2) are being called immediately and their return values (which are None) are being passed to runInParallel().

To fix this, you can use functools.partial() to create new functions with the parameters already filled in. Here’s an example:

from functools import partial

def func1(a): 
   ......

def func2(b):
   ......

if __name__ == '__main__':
   p1 = Process(target=partial(func1, 1))
   p2 = Process(target=partial(func2, 2))
   p1.start()
   p2.start()
   p1.join()
   p2.join()

This code creates two new functions using partial(): partial(func1, 1) and partial(func2, 2). These functions have the a and b parameters already filled in with the values 1 and 2, respectively. The new functions are then passed to Process() and executed in parallel.