Father Patrick Smythe described how he had been invited to the complaint's workplace in Leeds years after he is said to have indecently assaulted him.
Smythe, 79, said he had been contacted as there were concerns about a the "presence of a malevolent nature" where he had been working.
The defendant told the court that he "conducted a simple blessing with holy water" at the workplace.
As Smythe gave evidence at Leeds Crown Court, his barrister Susannah Proctor asked: "Was (the complainant) part of the process?"
Smythe replied: "Yes he was there. He was happy enough that I had been."
The defendant described feeling "shocked and hurt" to find out that the same person had made a sexual assault allegation against him years later.
Smythe, Manor Square, Otley, pleads not guilty to six counts of indecent assault and one of attempted indecent assault.
The offences are alleged to have taken place around 40 years ago.
The charges relate to six boys who were in his care at the time of the alleged offences.
Three of the complainants claim Smythe sexually abused them during trips to the former Leeds International Pool.
Smythe told the court heard he could not remember ever taking boys to the International Pool.
He told the court that he had arranged for youngsters to visit other West Yorkshire pools including Kippax, Featherstone, Wakefield, Rothwell and Barnsley, but never Leeds.
He said: "I have spent three years trying to recall when I took a group of boys to Leeds International Pool and I cannot bring the scene to my mind.
"I do not have a memory of taking boys to Leeds International Pool."
Smythe is also accused of committing sex offences against boys in his care while on a retreat at a hostel in North Yorkshire.
He denied carrying out any offences at the hostel.
The prosecution claim one of the boys at the hostel boy complained, switched on the light and started shouting at Smythe after he was assaulted.
It is alleged Smythe began crying and pleaded with the boy and his friends not to tell anyone.
Jurors have been told the priest then offered them money, a meal and trips to the cinema and a swimming pool to buy their silence.
Smythe denied the complainant's claims, saying: "He gives a very dramatic and emotional account."
The defendant told the jury he had unintentionally placed his hand on the boy's groin area while he was sleeping beside him
Smythe said: "I found my hand on his crotch and thought 'oh dear' and I turned away onto my other side for the rest of the night."
Smythe said the boy became angry and threatened to go home and tell his father the next day.
The priest told the jury how he then spoke to the boy and calmed him down.
Smythe said: "I said 'I am sorry if you are upset but I had no intention of doing you any harm.
"'It was accidental but if you feel it is proper to tell your father I understand that. If you want to go home I understand that as well.'"
Smythe said the boy and his friends then decided to stay at the hostel for the rest of their trip.
The trial continues.