Preventing Wasteful Parallel Callbacks When Running Multiple Processes and Caching

Dash Python
17

Looks like the issue is the default start method on MacOS changed from fork to spawn in Python 3.8

https://docs.python.org/3/whatsnew/3.8.html#multiprocessing

Setting the start method back to fork is a quick work around e.g.

from jitcache import Cache
import time
import multiprocessing as mp

mp.set_start_method('fork')

cache = Cache()

@cache.memoize
def slow_fn(input_1, input_2, input_3=10):
    print("Slow Function Called")
    time.sleep(1)
    return input_1 * input_2 * input_3

print(slow_fn(10, 2))
2 Likes