I think you are pushing the boundaries of Dash.
The problem you are running into is scope and how can you keep track of the components the user has generated.
Without making use of global variables I don’t see how you can do this (except passing values with a hidden div).
Here is a thread where some explorers ran into this same problem.