There's also skill and common sense. If you don't move the dirt correctly, it keeps falling back into the hole. If you don't check under the place you are tossing the dirt FIRST, you have to move all that dirt to look there.
There also probably some observational skills which can help--how the dirt is packed, whether or not you notice the earth you are digging has been turned recently, whether or not you think that with a certain amount of objects buried and a certain shape area, if there's a mathematical way to figure more likely spots. Whether or not you notice where people have dug already (if they fill it in convincingly you might not).
That said, luck can definitely get you something QUICKER, but I doubt it can completely deny you finding something if you are in shape, and able to think clearly.