BBC correspondent Andrew Harding, who is at the Zanu-PF meeting, said cheering erupted as the decision was announced.
One senior official later told him: "It's the dawn of a new era. Mugabe can go farming."
The move has yet to be formalised, but it increases further the pressure on Mr Mugabe that has been building over the past few days, and there are now moves to impeach him as president if he does not resign.
The recent events:
Two weeks ago Mr Mugabe sacked his then-deputy Mr Mnangagwa, who then fled the country
The army's chief of staff, Gen Constantino Chiwenga, warned last Monday that the military might intervene to stop purges in the party - and was roundly criticised by allies of the Mugabes
On Wednesday, soldiers seized the headquarters of the national broadcaster
Mr Mugabe has been mostly under house arrest for several days
On Saturday, unprecedented mass protests further weakened Mr Mugabe's position
The head of the influential War Veterans Association, Chris Mutsvangwa, threatened to "bring back the crowds and they will do their business" if Mr Mugabe did not step down.
Mr Mugabe has been president of Zimbabwe for 37 years.
Are you in Zimbabwe? What is your reaction to the news that Robert Mugabe has been sacked as leader of Zanu-PF? Share your views and experiences by emailing firstname.lastname@example.org.
Please include a contact number if you are willing to speak to a BBC journalist. You can also contact us in the following ways: