A strike by Maniche Sani in added time ensured the Lions were on the wrong end of a cup upset as they exited the competition at the third qualifying round stage.
On the balance of play the hosts deserved to go through and they had the first chance just moments in, Cameron Johnson broke clear and saw his effort go just over the crossbar.
Guiseley had their best chance just after 10 minutes, when a good one-two between Kaine Felix and Lee Shaw saw the forward’s effort come back off the crossbar.
The rest of the half saw chances at a premium, with a real battle taking place in midfield.
Sani, who came on as a substitute just before half-time, almost netted for the hosts right at the start of the second half when he ran through on goal but put too much power on his shot which blazed over.
The away side were unable to create a clear-cut chance in the second half, although a ball over the top found Felix but his first touch let him down – the keeper gratefully received it and the opportunity was gone.
It looked like a replay would be the next step but Sani had other ideas, his driving effort found the back of the net and left the home fans celebrating with just seconds left.
After the defeat Guiseley joint-manager Russ O’Neill said: “It is obviously disappointing to go out after the run we had last season. They scored at the best time for them, we now have to focus on the league.”
Guiseley’s next game is against Altrincham at home on Saturday as they look to get back to winning ways.
Farsley Celtic also exited the FA Cup at the third qualifying round stage on Saturday when they were beaten 5-0 at home by fellow National League North side Southport.
The visitors took the lead through a Ben Atkinson own goal on 18 minutes and second-half goals from Jack Simpson, Russell Benjamin, Devarn Green and David Morgan sealed Celtic’s fate.
Elsewhere, York City survived an FA Cup scare to secure a 2-1 win at Northern Premier League hopefuls Buxton.
York fell behind to a goal from their former loan striker Diego De Girolamo when his 11th-minute effort squirmed past teenage goalkeeper Ryan Whitley.
Alex Brown then put through his own net early in the second period under pressure from Nathan Dyer, who went on to grab the winner with 10 minutes left on the clock.
The 18-year-old wing-back initially headed against the bar but was on hand to slam in a second chance after leading marksman Alex Kempster had picked up the loose ball from the rebound.