"With the sudden jump in full-speed training, my body has needed more time than my mind would like," he said on Instagram.
"That's led to the back issues that kept me out of Kitzbuehel and made racing in Crans-Montana anything but easy.
"Because of that, I've decided to end my season earlier than planned."
Kilde won a bronze medal in super-G at the 2022 Beijing Games but sustained serious injuries to his shoulder and leg, as well as a subsequent infection, after he crashed in Wengen, Switzerland, in January 2024 during a downhill race.
He missed last season but returned to the World Cup in November at Copper Mountain, Colorado. His best finish this season was 11th in the downhill at Beaver Creek.
His best finish this season was 11th in the downhill at Beaver Creek.
Kilde, whose fiancée is American slalom great Mikaela Shiffrin, said the decision had been difficult.
"I have done everything in my power to be able to participate in the Olympics, but my body and mind are not ready," he said.
"Returning to the World Cup this season already means more to me than I could have imagined a year ago. Just making it back was a victory I'll always be proud of."
Kilde won the World Cup downhill title in 2022 and 2023 and was the overall World Cup champion in 2020.