Changing size, either sprite's or font size has no effect on draw calls.
Having different materials does increase draw calls, especially if you have them on different layers and sandwich things, like this:
Sprite on depth 0 (atlas A)
Sprite on depth 1 (atlas B)
Sprite on depth 2 (atlas A)
The above, while only using 2 materials, will require 3 draw calls for the GPU to draw them in order like that.
If you keep your Sprites/Default on higher depth values than everything else and on the same panel, they will only be using 1 draw call (assuming you've asked Unity to place them into the same atlas).