At 23.12 hours on 23 Oct 1942 the unescorted City of Johannesburg (Master Walter Armour Owen) was hit by one of two torpedoes from U-504 while steaming on a zigzag course at 10.75 knots about 80 miles east-southeast of East London, South Africa. The torpedo struck between the deep tank and #4 hold on the port side, blowing the hatches off the deep tanks and opening a hole that flooded these compartments immediately. All electricity failed and the engines had to be stopped when the engine room was flooded a few minutes later, causing the ship to settle by the stern with a slight list to port. No distress signals could be sent as the main aerial was brought down and fouled the emergency aerial and it was later discovered that the portable wireless set had been damaged while lowering into a lifeboat and was useless. Two lifeboats had been destroyed and the rafts aft were jammed in the rigging, so the 21 crew members, 60 Lascars and nine gunners (the ship was armed with one 4in, one 12pdr, two 20mm and four machine guns) abandoned ship in the four remaining lifeboats which were launched safely within five minutes despite rough sea and high swell. Before leaving the gunners tried to train the 4in gun, but it was damaged and could no longer be turned. The master was the last to leave the ship after making sure that no one was left on board. At 23.40 hours, the U-boat surfaced after firing a coup de grâce at the ship which was hit at #2 hatch on the port side, but only settled further on an even keel. The Germans then questioned the survivors in one of the lifeboats and left the area after the City of Johannesburg suddenly broke in two amidships and sank at 00.00 hours. Four Lascars were lost.

The boat in charge of the master had 11 persons on board and rescued two men swimming in the water. Quartermaster H. Birnie was picked up by a boat with 28 persons and later stated that he was alongside the U-boat after it had surfaced and even touched the plates when he was told to swim astern to the lifeboat. He was asked no questions and the engines of the U-boat remained stopped until the man was clear. The other two boats held 19 respectively 25 persons, but owing to the rough sea the lifeboats could not approach each other to even up the survivors in them. The master told the other boats to remain in the vicinity while his boat set sail in order to get help and made 140 miles in 24 hours before he and twelve survivors were picked up by the Dutch steam merchant Zypenberg about 8 miles from the coast in 33°50S/26°50E on 25 October and landed at Durban two days later. However, the boat with 19 occupants also set sail and only made half the distance before they were rescued by the British motor merchant King Edwin on 26 October and landed at Capetown three days later. The 54 survivors in the two boats that waited near the sinking position were picked up by the British steam merchant Fort George after 12 hours in 33°24S/28°31E and landed at Port Elizabeth on 25 October.