I never had that problem after a few times of "I already made plans but thank you"s.
About the smoking in a non-smoker's home: Never! I am a smoker in a family of non-smokers. I bring my own cigaretts and ashtray. I do not leave butts on the ground EVER. I don't like it at my home and I don't like it anywhere else. I don't care if it's bitter freezing cold,...I will not smoke in their home. If I am ever blessed with grandchildren, I will not smoke around them. I didn't smoke around my kids when they were small. I figure after high school they're old enough.