In the earlier discussion with the girls, he had said there were things in his past that caused him to shut down his emotions, which is why he came across that way.
What Jerry said - which was extremely rude and uncalled for - probably brought up some past event and he exploded.
Wasn't one of the situation to do with an ex girlfriend too? maybe that is what happened there and it seems like he is really trying. I wish I could find that convo. Jerry is a jerk, in this and many other ways.