Adam is indeed the person who is gaslighting Ashley. He finally gets the ankle bracelet off, having lost enough weight to do so, and he heads out the window into the garden, where there's a tape recorder buried beside the statue of Sabrina. He turns on the recorder and sure enough it's the sound of a baby crying. The only problem with this is how did he plant it there in the first place if he isn't able to leave the ranch house with his ankle bracelet on?
Presumably someone must be working with him. They show him stumbling around by himself in and out of the house, so there has to really be something wrong with his eyesight.