Goals from Jon Stead and Dan Jones had the hosts two to the good inside 20 minutes, before Brendan Kiernan added a third in the second half. Thanawat Suengchitthawon did find the net late on for the Foxes, who went home empty handed.
City were playing their second match in the competition after last month's 2-1 victory over Hull City and made two changes from the side that lost to Derby County in Premier League 2 on Friday.
Will Russ and Harvey Godsmark-Ford came in, the latter for his first start of the season, with Mitchell Clark and Ben Nelson dropping out of the side.
The youthful Foxes started in a 4-4-2 formation with Ali Reghba and Russ partnered up front, while Harrogate made eight changes from their weekend defeat to Bolton Wanderers in Sky Bet League 2.
That didn't stop the hosts starting on the front foot, with Kiernan forcing Jacob Stolarczyk into the first save of the game, as he comfortably held his low effort. City's stopper then produced a smart diving save to keep out Lloyd Kerry's volley from the edge of the box.
Kiernan continued to cause problems, cutting inside onto his right and firing in another low attempt that deflected out, before Stead headed narrowly wide from the following corner.
The veteran striker would soon convert, flicking in Kiernan's cross from close range after making a near post run.
The home side pressed for another and could have grabbed it soon after, when Tom Walker's corner found Kevin Lokko, whose header came back off the post.
Jones and Lokko then saw efforts cleared off the line, before Jones poked home to double the advantage when Leicester had failed to clear their lines.
The League 2 side were showing their attacking strength as Jack Emmett got to the byline and fired a cross into the middle, but no one was there to apply the finish, ahead of Kiernan seeing another effort held by Stolarczyk.
The Foxes gradually grew into the game and almost halved the deficit when Tyrese Shade capitalised on a defensive error to get through on goal, only for Harrogate 'keeper James Belshaw to divert his effort wide.
Back at the other end, Kiernan remained a threat and tried to curl it into the top corner just before the interval, but couldn't find the target.
Stolarczyk was called back into action immediately after the break, denying Stead his second of the evening with a close range save at the near post.
Just before the hour mark, it was three. Walker's shot was well saved by Stolarczyk, but only into the path of Kiernan, who had the simplest of tasks to tap the ball in from inside the six-yard box.
To their credit, City continued to push and went close to grabbing a goal back when Sam Hughes rose highest from Callum Wright's corner, heading narrowly wide.
The play soon opened up for Shade to strike, but his effort was blocked by Lokko, before Darragh O'Connor headed wide from the resulting corner.
The visitors finally grabbed a deserved goal six minutes from time, when after good work from Shane Flynn, Suengchitthawon applied the finishing touch.
Despite late pressure, Leicester were unable to find another way through Harrogate, who saw out the win.
Major moment…
The first goal proved crucial in giving Harrogate the chance to gain control early on and settled them into the game. Jon Stead's neat finish was followed up by further clinical finishes that secured three points for Town.
Where does it leave us?
City's youngsters are second in Group H on three points, with Harrogate now topping the group on five points from their two games, after winning a bonus point in the last fixture.
Elsewhere in the group, Hull City host Grimsby Town in their rearranged group stage clash at KCOM Stadium on Tuesday 17 November (5:30pm kick-off).
Coming up…
The Foxes return to Premier League 2 action on Sunday 18 October, facing Manchester City Under-23s at Academy Stadium (2pm kick-off).
The details…
City: Stolarczyk; Godsmark-Ford (Nelson 55'), O’Connor, Hughes (c), Flynn; Daley-Campbell, Leshabela, Wright, Shade; Reghba (McAteer 55'), Russ (Suengchitthawon 43').
Subs not used: Doherty, T.Pennant.
Goal: Suengchitthawon 84'
All times BST.