Hours of heavy rains overnight caused rivers to burst their banks, flooding homes with mud in Putumayo province.
An unknown number of people are missing. One army officer said the main local hospital was struggling to cope.
Mr Santos, who has travelled to the area, said troops had been deployed as part of a national emergency response.
He declared a state of emergency in the region.
The national fire service said at least 190 people had been injured.
The region's governor, Sorrel Aroca, told Colombian media that whole neighbourhoods had been buried.
Rescue services said their efforts had been hampered by continuing bad weather and damaged infrastructure.
"There are mobility issues on almost 80% of the roads, and from where the road ends, it is three hours to where the landslide took place," said one police officer.
Bridges have also been swept away.
Jose Antonio Castro, mayor of the provincial capital Mocoa, told Caracol radio that the town was "totally isolated", without electricity and water.
On arriving at the scene, President Santos told reporters, "My heart and the hearts of all Colombians are with the victims of this tragedy."
An alarm had sounded as the river's levels rose, leading many people to leave their homes for shelter, and avoiding wider loss of life, Colombia's El Tiempo newspaper said.
The mudslides were caused by the rise of the Mocoa river and three tributaries, a representative of the National Disaster Risk Management Unit told AFP news agency.
Rescue services said 17 neighbourhoods had been affected and Mr Castro said his own house had also been destroyed.
"The mud is up to the roof," he told Caracol radio.
The region, bordering Ecuador and Peru, is heavily dependent on agriculture and the petroleum industry.
While landslides and heavy rains are common in the mountainous area, March was Colombia's rainiest month since 2011, according to the state meteorological agency.
In neighbouring Peru, more than 90 people have died since the start of the year because of unusually heavy rainfall, which also caused landslides and flash floods.