Giroud came off the bench with Arsenal trailing 3-2 in Friday's Premier League curtain-raiser at the Emirates Stadium and after fellow substitute Aaron Ramsey equalised, he headed in an 85th-minute winner.
Alexandre Lacazette's arrival has bumped Giroud even further down the Arsenal pecking order, but having given the 30-year-old the chance to leave, Wenger was thrilled he decided to stay.
Wenger hoping for better Premier League start
"It's fantastic. I opened the door for him at some stage because I know it's important for him to play, but I don't want to him to go," Wenger told reporters. "He decided to stay and it was one of the great days of recent weeks for me. He's a fantastic player, but as well he has a big weight in this squad. It was a relief for me he decided to stay."
Giroud outmuscled Harry Maguire to meet Granit Xhaka's corner with a strong header that hit the bar and bounced behind the line before Leicester goalkeeper Kasper Schmeichel could club it away.
An admiring Wenger said Giroud was "in his prime" and "at the top of his game".
Lacazette, a club-record £46.5 million signing from Lyon, made a dream start to his home debut by heading Arsenal in front with just 85 seconds on the clock.
"Lacazette is simple — in every game he's stronger since he joined us," said Wenger, who had seen his side lose at home in three of their previous four opening league fixtures. "Even when he played on the left, he did very well. He adapts slowly to the way we want to play. He's a collective player and an intelligent player."
Amid the joy — and relief — of victory, there was nonetheless plenty to concern Wenger, not least the manner in which his side conceded all three goals.
Shinji Okazaki cancelled out Lacazette's opener with a close-range header after Maguire had been left unmarked to nod Marc Albrighton's deep cross back across goal.
Albrighton crossed for Jamie Vardy to put Leicester ahead after Xhaka had gifted the ball straight to him.
Following Danny Welbeck's leveller in first-half stoppage time, another corner yielded Leicester's third goal as Riyad Mahrez's cross was headed home by Vardy.
"If you were a manager, you're on the bench when you're 3-2 down with 20 minutes to go, and you win 4-3, you're more happy than concerned," said Wenger. "The goals we conceded, we can work to get rid of that. One was a corner, one was a ball we lost in the build-up and the third goal was a corner again. We were quite good last year on corners. With a bit of work we can get that out of the system."
Leicester manager Craig Shakespeare said he was "disappointed" that referee Mike Dean did not spot what he described as a "definite handball" by Mesut Ozil in the build-up to Ramsey's equaliser.
But he drew encouragement from his side's display. "You have to take some positives," he said. "At the moment, sitting here, I've got a dejected dressing room. To come here and score three has to be pleasing in some respect, but it doesn't feel like it."
Like many of Leicester's players, Vardy endured a post-title slump last season, scoring only 13 times in the league, and Shakespeare was delighted to see him back among the goals.
"All pre-season he's looked sharp and fit," said Shakespeare. "We all know Jamie Vardy's attributes. He can be a nuisance for defenders. We encourage him to do that. We encourage our midfield players to play balls in behind. He came here tonight and got the goals his performance deserved."
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ