I've experienced the other side of the coin, where the person with the restrictive diet is terribly demanding about what is served. My feeling is that, if someone invites you to their home, you behave graciously. Yes, it's very nice if the host tailors the meal to accommodate special diets, but I also think it's nice if the person with the restrictive diet brings a dish as well. Maybe I'm overly aware of putting people out, but if I was the only vegetarian attending a large party, I wouldn't expect the host to make concessions for me. I'd ask if I could bring something or would bring a dish that I could eat.....and that the other guests could try.
Especially in the case of vegetarians, it's pretty easy to eat and not need special dishes. I don't think most people serve all-meat meals. Plus, there are still many people who wouldn't have the first clue about what to fix as a main if it didn't involve meat or whatever else needed to be omitted.
I just know that the people who have been pains about the meal in my circle (complaining about they can't eat this and can't eat that, but then never offer to help with the meal or bring something) don't get invited back.
I guess my feeling is that, if you have a terribly restrictive diet, it's presumptuous to expect a party host to create a menu simply to accommodate your needs....and I know people who DO expect that. I'm talking people who are dairy-free, gluten-free and strict vegetarians. Like I said, most people wouldn't know what to even fix. Gosh, add in garlic and onions and I think most people would just put out a platter of crudites!