At 09.26 hours on 23 May 1942 the unescorted Samuel Q. Brown (Master Aksel Andersen) was hit by one torpedo from U-103 about 100 miles south of Cape Corrientes, Cuba. The ship had been spotted at 01.39 hours and was missed by a spread of two torpedoes at 04.02 hours. The torpedo struck on the port side at the bulkhead between the #9 tank and the after fuel tanks and set the vessel on fire immediately. The engines were stopped and the master ordered the boats to be launched, but the eight officers, 31 crewmen and 16 armed guards (the ship was armed with one 4in, four 20mm and two .30cal guns) were forced to jump overboard and swam to two lifeboats and two rafts that were cut loose. Two crew members were lost. The survivors were questioned by the U-boat that surfaced 20 minutes after the hit and then left the area after hitting the tanker with a coup de grâce at 10.35 hours.

On 23 May, the survivors, now together in the two lifeboats, were spotted by a patrol plane of the US Navy from Upham, Canal Zone. The plane took five injured men on board and brought them to a hospital at Key West, Florida, but one of the men later died of wounds. The remaining survivors were picked up the next day by USS Goff (DD 247) and taken to Cristobal after the hulk was scuttled by gunfire from the destroyer on 25 May.