"No to war," chanted many among some 6,000 demonstrators according to march organisers as they rallied outside the Reina Sofia museum in the Spanish capital.
Protesters answered a "not in our name" petition from dozens of artists demanding Spain does not become embroiled in the Syrian conflict.
Spain arrests suspected extremist about to head to Syria
The petition has received some 34,000 online signatures to date as European governments consider armed intervention against Islamic State following the November 13 Paris attacks for which it claimed responsibility.
With December 20 polls fast approaching Rajoy's conservative government has been holding off on any decision.
"Decisions have to be well thought though, as in any aspect of life," said Rajoy, who added Madrid was in touch with its allies pending a clear plan of action.
Spain jails 11 for sending suicide bombers to Syria
Rajoy's Popular Party is mindful of how in March 2004 under his predecessor Jose Maria Aznar, who had backed the US intervention in Iraq a year earlier, lost general elections which took place three days after Islamic extremists killed 191 people in a series of Madrid train bombings.
Aznar's stance on Iraq was in stark contrast to that of the public in a traditionally pacifist country.
Leftist opposition parties have voiced opposition to Spanish military involvement in the region while the leader of far-left grouping Podemos, Pablo Iglesias, backs putting the issue to a referendum.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ