James, trying to finally win gold at his fifth Olympics, took the silver and the bronze went to Totsuka's compatriot Ryusei Yamada.
Totsuka put up a high score of 95 in his second run to lead the field during a nighttime event in the Alpine town of Livigno.
James made his high-flying tricks look effortless at the start of his first run but ran out of room to land and skidded to a stop at the very end. On his second run, he shot up to second place with a 93.50. The 31-year-old nearly landed a flawless third run but fell on his final trick. He hung his head at the bottom of the pipe.
James, Australia's flag-bearer at Pyeongchang 2018, took bronze at the Games in South Korea and silver in Beijing four years ago.
In the halfpipe, riders slide across a 22-foot-tall, U-shaped ramp and perform acrobatic manoeuvres in the air. Judges award points based on amplitude, variety, difficulty and other factors, with a maximum score of 100.
Defending champion Ayumu Hirano of Japan fell forward in his first run and skidded on his stomach. The 26-year-old returned with an impressive second run and then fell backwards on his final attempt. He finished seventh.
Jan Scherrer of Switzerland, who took bronze in 2022, hit his head in practice before the qualifiers and had to withdraw.