Legend affects Pie Chart size

I have a pie chart that has it’s size being affected by the legend. When the legend has relatively few entries, the chart is fine, but as I add entries the pie becomes smaller. Is this expected behavior, and is there a way to circumvent it?