Is it possible to create a custom grid over the mapbox maps like below in plotly
You can define the lines as mapbox layers. You need the lon and lat coords for the rectangle vertices. From these coords you define a meshgrid, and then the lines over the meshgrid, as layers of geojson source.
See this example https://community.plotly.com/t/circle-on-scattermapbox-with-coordinates-as-reference/36633/2 on how to draw a circle as a layer.