Aside from what lurkinggirl said in her extensive and excellent post, I also have to add the financial factor. Consider: what would the expenses be, for say, Joe Millionaire? It's basically just one fixed location, with a relatively small number of remote locations. So you have relatively low costs, and thus a network may be more willing to purchase such a dating reality show.
Contrast this with the two best-produced adventure reality shows: TAR and Combat Missions. TAR, in my honest opinion, probably costs more than any other reality show to produce. Consider the costs involved: a dozen pitstops, your camera crews (one for each team, then perhaps more at the route markers, roadblocks, and detours), and the logistics of flying everyone around the world. Or consider Combat Missions. Fine, it's basically around one location as well, so you don't have all your location costs. However, the amount of equipment and such that you need for the show is also rather large.
All that costs money. Like has been said before, these shows also suffer from rather inept marketing. Most marketers nowadays are, in my opinion, capable only of selling sex. They are incapable of selling adventure anymore. So, all in all, you are asking the networks to undertake a larger gamble, and the return is not nearly as assured. Naturally, since their primary goal is to make money, they are more reluctant to take that risk, resulting in less of these type of shows.