CSS for Markdown only

Yes, you can use

dcc.Markdown(...., dangerously_allow_html=True)

However, you should be aware that this can introduce security risks - For more information see:

If you are displaying trusted data, this shouldn’t be a problem, but if you are displaying data from user input or extracted from a large database, there is a potential to create an XSS vulnerability