giz, I managed a book store and heard that line about a dozen times a day. The whole time they were running their mouths about reading all day, I was busy doing displays, cleaning shelves, windows, floors, bathrooms, whatever, and wondering if the fools were even watching.
I love to cook and always, always wanted to own a tea room. I just knew that it was more work that I needed to take on.
Whatever your decision, I'm sure that many of us will be around to encourage or to listen while you vent, when needed. Just give it lots of thought before committing yourself to anything.