I guess I'm still confused.
Intially, I was trying to have one catch-all script, but as per Arek's suggestion, I moved it to scripts attached to each object/collider.
Yes, for the case of OnSelect(true) I can look at the object and know what was clicked - but in my case, I need to be able to know who was clicked in the case of OnSelect(false)
I say this because the scenario is this - I click a tower, I get an NGUI Button. I now click the button. What happens? An OnSelect(true) fires for the button, and an OnSelect(false) fire for the tower, which is great. BUT, in the OnSelect(false) for the tower, I need to ask - "Who was clicked? If it was not a button, then I want to go do action x (where x is remove the panel I drew previously)".
That's the part I'm stuck at. If there was a guarantee of the order the events fire (such as all OnSelect(false) fire after OnSelect(true)) then I could manage who was selected, but I don't think thats the case.