Abe, 67, fell to the ground and was seen bleeding after he was shot from behind at 11:30 a.m. local time, local police and firefighters said.
He was rushed by medevac to Nara Medical University Hospital, according to local police and emergency services.
Police arrested the suspected gunman Tetsuya Yamagami, 41, a resident of the western city, at the scene.
Abe first took office in 2006 as Japan's youngest prime minister since World War Two.
He served two terms as prime minister, stepping down in 2020 citing ill health.
