Yeah, the police usually don't get involved in disagreements or money issues unless there is violence. They won't force someone to turn the baby over without a court order. Even with a court order, it's difficult to get enforcement. I've seen it.
Anyone with half a brain knows you are not making yourself look good before a judge when you keep the belongings of the mother's baby, causing her grief, just to get your loan repaid. A judge will hate that behavior. His mom should not have supported that. Not good.
Something is not right with Joe. I don't know what it is, but something is missing. It's just not there. He doesn't seem to have any interpersonal skills. He doesn't seem to have a reasonable bone in his body. He has zero maturity. Why? His parents appear to be above average in their attitude and example.
Why is he like that? The only thing I can come up with is that he is doing some kind of drug that is really messing him up. I hope not.