Well, let's see:
1. From the gist of your tale, the cat doesn't seem to have any identification. I feel a responsible pet owner would identify their pet somehow, no?
2. If the neighbors are the owners, they haven't sought their cat--no lost pet flyers, no door-to-door search. I know that if one of my dogs was lost, I'd be out night and day, going door to door, and putting up flyers everywhere in the vicinity.
3. Those who can identify this cat don't exactly give his owners a ringing endorsement.
I hate to say it, but it doesn't sound like the owners particularly care about or care for this animal.
He sounds like a good cat. I'm sure your family would make better owners. I'm not an attorney, but if his owners don't care enough to look for him, I don't think it's your responsibility to.
And, personally, I'd feel better if you were its owner.
Good luck. Just my two cents.