Turn off trimming before adding that sprite to the atlas. The padding values I mentioned were for a non-trimmed sprite.
Before that I did not notice this feature

Attach (see edges):
1. Trim alpha disabled.
2. All borders = 1.
I feel like a complete fool

Alternatively, if I set Bottom Border = 0 (or Left And Right too), then again, do not get the correct result.