Old Abe was smart, too. He got it from Proverbs 17:28
28 Even a fool, when he holdeth his peace, is counted wise:
and he that shutteth his lips is esteemed a man of understanding.
I actually got into a debate about who said that quote once, so I don't just know all this off the top of my head. Apparently all the people its attributed to (Mark Twain, Abe Lincoln, George Washington, etc) got it from somewhere else. They aren't even sure if any of them really said it. It's just an old proverb. I just googled and got this tidbit from Ask Yahoo!
A page titled Mark Twain and the Mutating Quote attributed at least four variations of the same phrase to the eminently quotable Twain, explaining that it was a case of "split personality" that accounted for the variations, rather than a rash of misquotes.
Other pages suggested a number of other authors for the saying, including: Abraham Lincoln, George Eliot, Groucho Marx, Albert Einstein, and a mysterious figure named Silvan Engel.
In an attempt to solve this proverbial puzzle, we paid a visit to Bartleby.com, home to the online version of Familiar Quotations. Unfortunately, after searching on a number of possible keywords and potential authors, we couldn't find a single reference to this quote.
All the confusion and disagreement surrounding both the author and the wording of this saying led us to suspect that it may be a simple maxim, not attributable to any single person -- which is not to question its wisdom.