Correct me if I'm wrong, but I believe a returned PAT would be 2, because it would be treated as a 2 point conversion for the defense.
To get a single point, you need to score a safety on the PAT against the team that just scored a TD. This would require the football going the length of the football field (either it gets fumbled and batted away the length of the field which happened to Ole Miss a few years ago, or the defense returns the ball and fumbles near the endzone). Then the offense recovers the ball in the endzone, resulting in a PAT safety against the offense.
As far as I know, it's the only way to have a single point.