Not sure which bloom you're using, but a typical Bloom effect takes all color above the specified threshold and blurs it, then applies it on top of the original image.
Label is the only thing white in your screenshot, so likely the only thing that exceeds the set threshold.
If you're using Unity's old-school bloom effect, then it's even worse. That one does blooming based on the values of the alpha channel, and is only meant for solid objects (objects with no transparency), which happens to exclude UI as all UI elements have transparency.