I feel your pain. I learned a long time ago to engage people in conversation and learn more about them before saying yes OR no.
Quite a long time ago, I almost lost the opportunity to make an incredible amount of money. At a local Quick-Check, a biker was ahead of me in line buying a soda. I was talking to a potential customer about making a custom piece of furniture for his collection and the biker butted into my conversation. He looked like a Hell's Angels reject, but I politely listened as this unshaven, scruffy, no-accout biker explained what he wanted for his new house. Our conversation continued in the parking lot where I was introduced to Elizabeth Taylor, saw 'Capitalist Tool' on Scruffie's bike, and put two and two together - and listened more closely after intoducing my client to Malcom Forbes

Malcom used to ride motorcycles in my area and often with Liz Taylor when she was in town, it turned out. I got both commi$$ion$, BTW.
Never say no to anyone too quickly