Scuffles broke out between Mr Zuma's supporters and opponents, resulting in all speeches being cancelled.
The main labour federation, Cosatu, called on Mr Zuma to step down last month after he sacked his widely respected finance minister.
Mr Zuma has vowed to remain in office until his term ends in 2019.
He was seen on live television hastily leaving the podium and being whisked away in a motorcade from the rally in Bloemfontein city, Reuters news agency reports.
Mr Zuma attended the rally despite the fact that powerful affiliates of Cosatu, the Congress of South African Trade Unions, had opposed his presence.
Cosatu is part of a formal alliance with the governing African National Congress (ANC).
Earlier, sections of the crowd sang a song which, loosely translated, means: "Have you heard the good news? Zuma is going", South Africa's privately owned News24 site reports.
Cosatu leader Sidumo Dlamini said the rally had been marred been "chaos", forcing its cancellation.
The protest required "thorough reflection" on the part of the country's leaders, he said.
Senior ANC officials were also booed at a May Day rally in Durban city, the political heartland of Mr Zuma.
Pressure on Mr Zuma to resign has been mounting since he sacked Pravin Gordhan as finance minister in March.
It led to global rating agencies downgrading South Africa to junk status.
The reshuffle was condemned by trade unions, big business, the opposition and and senior members of the government, including Deputy President Cyril Rampahosa.
The opposition has repeatedly accused Mr Zuma of being corrupt, and says the reshuffle was aimed at giving him and his allies greater access to government money.
Mr Zuma said the reshuffle was aimed at promoting "radical economic transformation" to benefit the poor black majority.
South Africa's president has been dogged by allegations of corruption for more than a decade.
Last year, a court ruled that he should face corruption charges over a 1999 arms deal.
Mr Zuma is appealing against the ruling.
In a separate case last year, South Africa's highest court ruled that he had breached his oath of office by failing to repay government money used to upgrade his private residence.
He repaid the money, but rejected calls to sep down.
Mr Zuma is due to step down as leader of the ANC in December, and as South Africa's president in 2019.
His ex-wife Nkosazana Dlamini-Zuma and Mr Rampahosa are vying to succeed him in both positions.