The armed assault in Jalalabad city, the capital of Nangarhar province, was the second deadly incident involving aid workers in recent days and prompted appalled reaction in Afghanistan and internationally.
Tetsu Nakamura, 73, was the head of Peace Japan Medical Services - known as Peshawar Kai in Japanese - and had been working in the region since the 1980s, when he began treating patients with leprosy in Peshawar in neighbouring Pakistan.
Stunned residents return after IS losses in east Afghanistan
A spokesperson for Afghan President Ashraf Ghani called Nakamura "one of the closest friends of Afghanistan".
He "dedicated his life to helping and cooperating with our people", spokesperson Sediq Seddiqi said.
Attaullah Khogyani, a spokesperson for Nangarhar's governor, said Nakamura, who had been shot in the chest, was in the process of being transferred to a hospital in Bagram near Kabul when he died.
Five Afghans were also killed: three of Nakamura's security guards, a driver and another colleague, Khogyani said.
The attack comes as humanitarian groups are on high alert just days after an aid worker for the UN was killed in a bombing in Kabul.
On November 24, Anil Raj, an American who worked for the UN Development Programme in Afghanistan, died when his vehicle was targeted.
In a statement condemning Wednesday's incident, the UN Assistance Mission in Afghanistan expressed "revulsion" at Nakamura's killing.
It was "a senseless act of violence against a man who dedicated much of his life to helping" Afghanistan's most vulnerable, UNAMA said.
Mitsuji Fukumoto, an official with Nakamura's organisation, told reporters in Tokyo the motive for the attack was unclear.
"We have no idea what was the reason behind the attack, whether it was a simple robbery or whether it was conflict of interest," Fukumoto said.
Nakamura was famous in Japan for his aid work, which dates back decades.
Peshawar-kai was founded by associates of Nakamura, who had lived and worked in Afghanistan and Pakistan since 1984.
In 2003 Nakamura, a native of the southwestern Japanese city of Fukuoka, won the Philippines' Ramon Magsaysay Award for peace and international understanding - often called Asia's Nobel Prize.
Fond of sporting Pashtun dress, Nakamura was an outspoken opponent of the 2001 US-led war that ousted the Taliban regime, whom he defended as able administrators.
"I am not fooled by the justification that violence is necessary for the sake of democracy and modernisation," he wrote in an old posting on his website.
"True happiness for mankind should be realised not through violence or money, but in a humane way."
Nakamura also described a variety of his organisation's projects to help Afghans, including the construction of wells and irrigation canals, as well as health services.
Jalalabad resident Auzubillah, who only uses one name, told AFP that he heard shooting at about 8:00 am (0330 GMT).
"I saw there were gunmen attacking a Japanese and his security guards," he said. "Then the gunmen left the area through a small street."
Photos from the scene showed a white pickup truck with a large cabin. Its side windows appeared to have been shot out, and at least three bullet holes could be seen in the windscreen.
The Taliban denied responsibility for the attack, saying they have "good relations" with organisations that "contributed to the reconstruction of Afghanistan".
Pakistan welcomes US willingness to help Afghan peace
Nangarhar was once a hotbed of activity for the Islamic State group's Afghanistan affiliate.
Authorities recently claimed the militants have been territorially defeated in the province - but small cells are thought to remain.
Aid agencies and non-governmental groups are sometimes targeted in Afghanistan's war.
The Taliban in May struck Counterpart International, a US-funded non-profit group working with marginalised people. Nine people were killed in that attack.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ