Rovers now need just three points from their last two games to secure a home final in the Championship Shield against Leigh Centurions.
Featherstone opened the scoring just before the 10-minute mark following a swift move which ended with Shaun Robinson strolling over unchallenged.
It didn’t take long for Dewsbury to hit back as Paul Sykes nipped through a gap and dived over the whitewash.
The fast-paced nature of the contest continued as Featherstone regained the lead moments later when Harry Newman reacted quickest to a loose pass.
The 18-year-old then burst 80 metres to dot down for his fifth try in the Championship Shield.
Dewsbury were back in front just after the half-hour mark. James Glover weaved through the opposition defence to finish out wide after an initial break from Josh Guzdek.
Rovers responded just two minutes later when Whylie squeezed over in the corner to put the hosts ahead for a third time.
John Davies extended the advantage with his 200th career point as he took a short pass and forced his way over from close range.
It got better for Featherstone moments later, on the last tackle Anthony Thackeray found Whylie in space who moved the ball back inside for Misi Taulapapa to plant down for his 100th try on British soil.
Dewsbury narrowed the difference three minutes after the second-half restart as Daniel Igbinedion latched onto Guzdek’s clever grubber kick. Fev’s Connor Farrell and Glover traded tries and then Farrell and Dale Morton did the same before Ridyard strode over on his return to seal the win for Rovers.
Featherstone: Hardman, Whylie, Taulapapa, Newman, Robinson, Thackeray, Wildie, Cooper, Carlile, Brooks, Farrell, Davies, Lockwood. Subs: Maskill, Knowles, Hardcastle, Ridyard.
Dewsbury: Guzdek, Potts, Glover, Ryder, Morton, Sykes, Delaney, Sheriffe, Ward, Trout, Walshaw, Knowles, Brown. Subs: Richardson, Everett, Teanby, Igbinedion.
Referee: N Bennett (RFL)