Kayne Ramsay's stoppage time winner saw Town claim victory on their return to action following a December interrupted by postponements.
Luke Armstrong had given Town the lead from close range, but the visitors hit back before the break though Otis Khan, before Harry Clifton's finish four minutes after the restart had turned the tie on it's head.
Armstrong was able to pull Town back on level terms though, claiming his third brace in three SkyBet League Two, before Ramsay's first Harrogate goal in the 91st minute sent the Town fans home smiling.
Simon Weaver made two changes from his last starting line-up 23 days ago, with Kyle Ferguson and Danny Grant replaced by Joe Mattock and Tyler Frost.
Town had made a strong start to the contest and grabbed the opening goal in the 12th minute as Armstrong continued his fine form in front of goal, tapping in Sam Folarin's low cross from a yard out.
Grimsby hit back with a well struck Kieran Green volley from the edge of the box, but Pete Jameson was able to gather at the second attempt.
Harrogate continued to probe with Tyler Frost dragging wide of the post, before Jaheim Headley broke free down the left and found Josh Coley, whose prodded finsh was blocked by a defender.
John McAtee went close to equalising in the 23rd minute when firing wide on his right foot, before the visitors did find an equaliser just before the half hour mark.
Brendan Kiernan cut the ball back to Michee Efete whose low cross was bundled in by Khan at the back post.
Another McAtee shot forced a save from Jameson in what was the last of the first half's goalmouth action.
Grimsby found themselves ahead four minutes after the restart when Kiernan's touch found McAtee, who carried the ball to Clifton who was able to round the advancing Jameson.
Town responded well to finding themselves behind with Folarin heading wide at the far post and Armstrong sending a powerful effort across the face of goal.
Armstrong and Folarin combined again in the 65th minute with the latter shooting wide, before a nice turn and pass from Josh Falkingham found Armstrong, who's cross was well claimed by Max Crocombe.
Town got a deserved equaliser with just under 15 minutes of time remaining when good work from substitute Jack Muldoon set Armstrong free to finish on his left foot, claiming a third brace in three games.
Just as it looked like the tie wqas heading for a draw, Ramsay carried the ball into the Grimsby area and squeezed his left footed finish past Crocombe at the near post to give Town fans a late Christmas present!
Harrogate Town: Jameson, Mattock, Falkingham, Burrell, Thomson, Coley (Austerfield 90), Folarin (Wright 79), Headley, Frost (Muldoon 65), Ramsay, Armstrong
Subs not used: Oxley, Welch-Hayes, Horbury, Ilesanmi
Goals: Armstrong 12, 76, Ramsay 90+1
Booked: Burrell, Ramsay
Grimsby: Crocombe, Efete, Green, Waterfall, Holohan, McAtee, Khan, Clifton, Amos, Kiernan (Orsi 83), Maher
Subs not used: Glennon, Hunt, Morris, Simmonds, Smith, Scannell
Goals: Khan 29, Clifton 49
Booked: Crocombe
Referee: Bobby Madden
Attendance: 2,744
Team Line-ups
TownJameson
Mattock
Falkingham
Burrell
Thomson
Coley (Austerfield 90)
Folarin (Wright 79)
Headley
Frost (Muldoon 65)
Ramsay
Armstrong
Grimsby TownCrocombe
Efete
Green
Waterfall
Holohan
McAtee
Khan
Clifton
Amos
Kiernan (Orsi 83)
Maher
OfficialsB Madden
Match SponsorDoodle HR