Japanese Prime Minister Shinzo Abe, in Jerusalem on the latest leg of a Middle East tour, demanded the militants immediately free the two hostages unharmed.
He flew home several hours earlier than planned to take charge of the crisis after meeting with Palestinian president Mahmud Abbas, who condemned the threat against the abductees as "despicable".
IS has murdered five Western hostages since August last year, but it is the first time that the extremist group -- which has seized swathes of Syria and neighbouring Iraq -- has threatened Japanese captives.
In footage posted on a websites, a black-clad militant brandishing a knife addresses the camera in English, standing between hostages Kenji Goto and Haruna Yukawa who are wearing orange jumpsuits.
"You now have 72 hours to pressure your government into making a wise decision by paying the $200 million to save the lives of your citizens," he says.
The militant says that the ransom demand is to compensate for non-military aid that the Japanese prime minister pledged to support countries affected by IS violence at the start of his Middle East tour.
But Abe said Japan would not bow to extremism and pledged to honour his promise of aid.
"I strongly demand that they not be harmed and that they be immediately released," he told a news conference in Jerusalem.
"The international community will not give in to any form of terrorism and we have to make sure that we work together."
Abe said the aid he had promised in Cairo on Saturday was to help the displaced and those made homeless by the conflict in Iraq and Syria.
"This position is unshakable," he said, describing the assistance as "absolutely necessary" for the survival of people who have fled fighting.
Since August, IS has murdered three Americans and two Britons, posting grisly video footage of their executions.
US journalists James Foley and Steven Sotloff, American aid worker Peter Kassig and British aid workers Alan Henning and David Haines were all beheaded.
The militant who appeared in the video threatening the Japanese hostages spoke with a very similar southern English accent to the militant who appeared in the footage posted of the executions of the Britons and Americans.
Goto is a freelance journalist, born in 1967, who set up a video production company, named Independent Press in Tokyo in 1996, feeding video documentaries on the Middle East and other regions to Japanese television networks, including public broadcaster NHK.
He had been out of contact since late October after telling family that he intended to return to Japan, NHK reported.
In early November, his wife received email demands for about one billion yen ($8.5 million) in ransom from a person claiming to be an Islamic State group member, Fuji TV said.
The emailed threats were later confirmed to have come from a sender implicated in the killing of US journalist Foley, Fuji TV said.
Yukawa is a 42-year-old widower who reportedly has a history of attempted suicide and self-mutilation after his military goods business went bankrupt and his wife died of cancer.
He came to widespread attention in Japan when he appeared in footage posted last August in which he was shown being roughly interrogated by his captors.
He offered brief responses to questions posed in English about why he was in Syria and the reason he was carrying a gun.
He replied in stilted English that he was a "photographer" and a "journalist, half doctor".
"I'm no soldier," he said.
Another video surfaced showing a man believed to be Yukawa test-firing an AK-47 assault rifle in Syria.
Japanese nationals' involvement as combatants in foreign conflicts is limited, although the country's extensive media is usually well-represented in hotspots.
Japan has been relatively isolated from the Islamist violence that has hit other developed countries, having tended to stay away from US-led military interventions.
In 2004, Japanese tourist Shosei Koda was among a series of foreign hostages beheaded by al Qaeda in Iraq in grisly videotaped executions.
He had ignored government advice to travel to the country in the midst of the bloody insurgency that followed the US-led invasion of the previous year.
In early 2013, Japan was rocked when militants overran a remote gas plant in the Algerian desert. The four-day ordeal that involved hundreds of hostages ended when Algerian commandos stormed the plant.
Ten Japanese died, giving the country the single biggest body count.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ