Between 06.04 and 06.11 hours on 9 March 1943, U-510 fired torpedoes during a second attack at convoy BT-6 about 175 miles north of Cayenne, French Guiana and reported again four ships sunk. The convoy became unorganized after the first attack because all ships performed evasive maneuvers. In fact, the Mark Hanna and James Smith were damaged and the Thomas Ruffin and James K. Polk were damaged and later declared a total loss.

The James Smith (Master William H. Aguilar) in station #73 was struck by one torpedo on the port side at the #5 hold. The explosion blew a large section out of the side and bottom of the ship and disabled the steering gear, knocked down the radio antenna and damaged the propeller shaft. Five armed guards and six crewmen sleeping on the tarpaulin cover of the #5 hatch died. The survivors among the eight officers, 34 crewmen and 16 armed guards (the ship was armed with one 5in, one 3in, four .50cal and two .30cal guns) calmly abandoned ship in four lifeboats after the engines were secured. After daybreak two of the boats returned to the vessel and reboarded her, while the occupants in the other boats were picked up by the American submarine chaser USS PC-592, which later came to the ship, whose bow lay out of the water. On 12 March, the remaining survivors on board were ordered by the commander of the submarine chaser to abandon ship, but the James Smith did not sink and the master, three crewmen and the armed guard officer again reboarded her. They stayed with the ship as HMS Zwarte Zee (W 163) towed her to Trinidad. The men on the submarine chaser were landed at Trinidad on 16 March.
The vessel was later towed to New Orleans, where she was repaired and returned to service on 10 Aug 1943.