I have observed the same behaviour, both points render but only the label from one is rendered, even with marker(allowoverlap=True). This can also sometimes applies to labels of points that are very close together, not just completely on top of each other.
Adding some jitter to the raw data before passing to the map may be the only way to resolve this, unless the Scattermapbox code is changed to handle overlapping labels in a different way.
eg (psuedocode)
- find all points in your dataframe that have the same coordinate
- adjust any duplicates by adding some randomdx randomdy movement to them
Depending on how far you are willing to move them, this may or may not help. If the movement is very small, and the labels large (as in your screenshot), if the labels still overlap as they still may not be rendered. However, if you add hover tips and the small jitter, then at least you will be able to zoom in and hover over each individual point to get the details.