The former vice president said the embassy should never have been moved without that decision being part of a wider Middle East peace deal between Israel and the Palestinians.
"But now that it's done, I would not move the embassy back to Tel Aviv," Biden told a virtual fundraising event.
The location of the US embassy is a hot-button issue: the status of Jerusalem is one of the most hotly contested issues in the Israeli-Palestinian conflict.
Israel seized control of the east of the city in 1967 and later annexed it in moves never recognized by the international community.
Israel considers the city its undivided capital, but Palestinians believe the east is illegally occupied and see it as the capital of their future state.
Trump shattered the status quo when he recognized Jerusalem as Israel's capital and announced his decision to shift the US embassy to the holy city in December 2017.
He has repeatedly boasted that he is the most pro-Israeli US president in history, and has slashed aid to the Palestinians while making big concessions to the Israelis.
Biden said he would reopen the consulate in Jerusalem "to engage the Palestinians and my administration will urge both sides to take steps to keep the prospect of a two-state solution alive."
In January this year, Trump's son-in-law and senior aide Jared Kushner unveiled his Middle East peace plan.
It proposes the establishment of a Palestinian capital in Abu Dis, a suburb of Jerusalem, and gives consent for the annexation of Israeli settlements as well as the Jordan Valley -- Palestinian territories occupied by Israel since 1967.
The Palestinians -- backed by the Arab League, the Organization of Islamic Cooperation and the African Union -- have rejected the plan.
Biden, who during his time as vice president had an uneasy relationship with Israeli Prime Minister Benjamin Netanyahu, has said previously that "we can't be afraid to tell the truth to our closest friends."
"The two-state solution is the best, if not the only, way to secure a peaceful future for a Jewish, democratic state of Israel," he said in October last year.