Following an early dismissal for Forest’s Riley Harbottle, Trevor Peake’s young Foxes, third in the Premier League U18 South Division, led through Simran Thandi’s smart header after 54 minutes.
However, the hosts levelled eight minutes later as Yassine En-Neyah capitalised on a heavy touch from City goalkeeper Rhys Davies to lay it on for Brennan Johnson to slide it home from close range.
Once the tie had gone to extra-time, Luke Thomas, who made his Development Squad debut during a 6-1 success over Tottenham Hotspur this month, rattled the woodwork with seconds to spare.
But the shootout ended disappointingly. While Connor Tee, Kairo Edwards-John, Tyrese Shade and Thomas converted from 12 yards, Calvin Ughelumba and Khanya Leshabela were denied.
Speaking to LCFC.com, Peake said: "If you summarise the whole game, if we couldn't take advantage of having 11 men over 100 minutes, we probably didn't deserve to go through.
"We had the better chances and got into their final third more than them, which you would expect, but it's something we've struggled with. They got into shape, they shuffled across and we found it hard.
"Full credit to Forest, they fully deserved to go through. They put that tactic in place and they stood tall for a very long time. It's still been a wonderful season for us, we've done tremendously well.
"The league form gave us hope that we could have got further in the Youth Cup so it's very disappointing. There are nine of the 11 who probably won't get this chance again as they're in their second years.
"Overall, we've got a great chance in the league and it's going to be an exciting end to the season, it's just a little frustrating not to be in the Fifth Round draw of this competition."
Next up, City have the opportunity to recover from this setback away at Reading in the league on Saturday afternoon (12pm kick-off).
Nottingham Forest: Soloha, Richardson (Ram 86), Preston, Smith, Stewart, Harbottle (c), Appiah (Bondswell 57), Charlesworth, McCormick (Mighten 97), En-Neyah, Johnson
Subs not used: Shelvey, Barnes
Goal: Johnson 62
Sent off: Harbottle 21
City: Davies (Stolarcyzk 119), Thandi, Ughelumba, Gill, Heaven, Hušek, Thomas, Leshabela (c), Shade, Tee, Edwards-John
Subs not used: Ikpakwu, James, Meite, Myring
Goal: Thandi 54
Attendance: 652
Referee: Anthony Tankard
Images: Dan Westwell Photography