His comments came after Israeli-Palestinian clashes in and around the holy site over the weekend left 170 people wounded, mostly Palestinian demonstrators at the hands of Israeli security forces.
The violence comes almost a year after similar tensions sparked an 11-day conflict between Israel and militant groups in the Palestinian enclave of Gaza.
Al-Aqsa is the holiest site in Judaism and the third-holiest in Islam.
The weekend of violence has further escalated tensions, with Israel carrying out its first air strike on the Gaza Strip in months early Tuesday, in response to a rocket fired from the Palestinian enclave.
Erdogan said on his official Twitter account he told Herzog “the fact that Al-Aqsa mosque was raided by fanatic groups after the morning prayer yesterday and the day before … and the spread of the tension to Gaza increased our sadness.”
He said that “these images which are seen every year because of some radicals hurt the consciences and cause justifiable reactions in the entire Islamic world.”
Erdogan told the Israeli president: “In this sensitive period, I would like to emphasise once again the necessity of not allowing provocations and threats against the status and spirituality of Al-Aqsa mosque.”
The Turkish leader repeated his call for everyone to “make the utmost effort” in order to preserve the spirituality of the holy site.
Israel and Turkey proclaimed a new era in relations following more than a decade of diplomatic rupture, after Herzog made a landmark visit to Ankara in March.
Erdogan, a vocal advocate of the Palestinian cause, had in the past criticised Israeli policies toward the Palestinians.
Turkey has maintained ties with Hamas who have controlled Gaza since 2007.
For all the latest Sports News Click Here