How to Parallelize a for Loop in Python

If you need to run a for loop in parallel, then you can do one of the following:

Option 1 – Using multiprocessing

import multiprocessing

def sumall(value):
    return sum(range(1, value + 1))

pool_obj = multiprocessing.Pool()

answer = pool_obj.map(sumall,range(0,5))
print(answer)

Option 2 – Using joblib module

from joblib import Parallel, delayed
import math

def sqrt_func(i, j):
    time.sleep(1)
    return math.sqrt(i**j)

Parallel(n_jobs=2)(delayed(sqrt_func)(i, j) for i in range(5) for j in range(2))

Option 3 – Using asyncio

import asyncio
import time

def background(f):
    def wrapped(*args, **kwargs):
        return asyncio.get_event_loop().run_in_executor(None, f, *args, **kwargs)
    return wrapped

@background
def your_function(argument):
    time.sleep(2)
    print('function finished for '+str(argument))

for i in range(10):
    your_function(i)

print('loop finished')