The children were walking to Rufford Park Primary School, Yeadon, at about 8.30am on Wednesday when they were followed.
The man is described by the school as in his 60s or 70s, wearing a red coat, brown jeans and a black polo shirt.
17 pictures you'll only understand if you're from Leeds
Leeds Olympian Nicola Adams and girlfriend Ella Baig reveal baby boy's name as they talk about parenthood
Resident claims Leeds pub music is 'like a bad night in Benidorm’ amid plans to extend opening hours
Two teenagers in hospital after crashed motorbike sets on fire on Leeds street
Caught on camera Leeds: West Yorkshire Police need to speak to these people urgently
The same man was seen lurking outside the school at about 10am the same day, before he walked down the ginnel to the side of the building.
The incident has been reported to the police and Leeds City Council.
In a letter to parents, Rufford Park Primary School said: "We have spoken to the children and given stranger danger reminders but not given the details of the incidents or the description of the man.
"We are sharing this information with you so that you can be vigilant too at the times of drop off and pick up."