News of a cruise ship suffering engine problems first came through at about 14:00 local time (13:00 GMT).
Reports suggest the ship has since managed to restart one engine and move slightly further from the coast as the evacuation continues.
"We were having lunch when [the ship] began to shake," John Curry, who was evacuated by helicopter, told public broadcaster NRK. "Window panes were broken and water came in. It was just chaos.
"The trip on the helicopter, I would rather forget. It was not fun," he said.
Janet Jacob, who was also rescued, told the channel she had "never seen anything so frightening".
"I started to pray. I prayed for the safety of everyone on board," she said. "The helicopter trip was terrifying."
Fisherman Jan Erik Fiskerstrand, whose boat was one of the first to come to help MV Viking Sky, told Aftenposten newspaper, "it was just minutes before this could have gone really wrong".
The ship could have hit the rocks "if they had not started the engine and fastened the anchor".
By 22:40, 155 people had been brought to shore, but police warned that the figures would vary, and said they would "wait for counting until further notice".
Eight people suffered injuries and three of those were thought to be in a serious condition, NRK reported.
According to NRK, most of the passengers were British and American. Sea rescue services said the ship was being held steady and the evacuation was ongoing.
The Norwegian Meteorological Institute says some waves are more than 10 metres high, with one local newspaper reporting lifeboats were forced to turn back en route to the ship due to the "brutal" conditions.
Those brought ashore are being taken to a local sports complex. One volunteer said about 80 had arrived so far, and accommodation had been found for them in local hotels.
MV Viking Sky is a Viking Ocean Cruises ship, which had its maiden voyage in 2017.