Father Patrick Smythe is also accused of committing sex offences against boys in his care while on a retreat at a hostel in North Yorkshire.
Leeds Crown Court heard the offences are alleged to have taken place around 40 years ago.
Smythe, 79, of Manor Square, Otley, pleads not guilty to six counts of indecent assault and one of attempted indecent assault.
Michael Morley, prosecuting, told the jury that the charges relate to six boys who were in his care at the time of the alleged offences.
The prosecutor said: "We say this defendant has a sexual interest in male children.
"Not only does he have a sexual interest in children but, we say, he has acted upon that interest and he has abused that position he had in relation to these children.
"We say he has sexually abused numerous children in his care.
"You will hear from six adult males who have come forward and told the police that this defendant sexually abused them or attempted to sexually abuse them."
Mr Morley continued: "He, like any other suspect, has been interviewed by police.
"He will deny, and has denied, that any of the abuse occurred."
Mr Morley said it is alleged that the first complainant was indecently assaulted in the shallow end of the International Pool.
The court heard Smythe is alleged to have put his hand inside the boy's swimming trunks.
The jury heard the complainant did not report the incident at the time but came forward many years later after watching a TV documentary about abuse within the Catholic church.
The prosecutor said: "That brought everything back to him."
Jurors heard the second complainant is also alleged to have been indecently assaulted at the same swimming pool.
Mr Morley said he was aged 12 at the time and had been receiving grief counselling from Smythe following the death of a relative.
Three other complainants are alleged to have been targeted during the night while they were sleeping during stays at the hostel.
Mr Morley said Smythe spoke to one of the boys after the alleged incident and told him to "forget about it."
The court heard another boy complained, switched on the light and started shouting at Smythe after he was assaulted.
The prosecutor said Smythe began crying and pleaded with the boy and his friends not to tell anyone.
Mr Morley said: "He came in and sat down and told the boys that he could 'count on one hand the number of times he had done this.'"
The court heard the priest then offered them money, a meal and trips to the cinema and a swimming pool to buy their silence.
Mr Morley added: "Although as a child he had taken the bribe and taken the 'shush money', this has bothered him all his life."
A fifth complaint is alleged to have been indecently assaulted in a van while Smythe was driving the vehicle.
Smythe is also alleged to have tried to assault him at the hostel after reaching into the boy's sleeping bag during the night.
The final complainant is alleged to have been indecently assaulted after being "trapped" against the side of the swimming pool in Leeds.
The court heard the boy broke down in tears in the changing room after the alleged attack and later told his father who then reported Smythe.
Mr Morley said: "One would have expected the police to have been involved.
"Unfortunately it appears that at the time this allegation was made, it was not handled in the way one would expect and it may be that it was just swept under the carpet."
The trial continues.