Turns out that this was linked to this issue: [BUG] Duplicate callback outputs with identical inputs (and potentially different states) fail · Issue #2486 · plotly/dash · GitHub.
The issue was actually not with the outputs, but rather with the inputs. Dash generates the hash for the allow_duplicate outputs based on the input(s) (not the states) for a callback and based on what order the inputs are in. So, that was causing the issue. The workaround is to either switch the orders of the inputs wherever possible, or add a dummy input wherever necessary.