The assailant "died on the spot" after the guard opened fire, police spokesman Harald Soeros told AFP, adding that the attack took place just before midnight local time in a residential area.
The motive for the attack is not yet known, police said.
The elusive knife attacker
The guard, who was in a sentry box outside the building, suffered a wound to his upper arm in the attack.
"He initially used pepper spray" against the assailant, Soeros said, before opening fire.
According to early indications the guard "acted within the rules", said a spokesman from the ministry of defence, who said the attacker lived in Vienna but originated from the Tyrol region of Austria.
Police have ordered reinforcements around all diplomatic missions in the city.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ