I assume the crew abandoned ship quickly after the hit and rowed away, probably in fear that the cargo of ammunition will blow up. Remembering the danger of Q-ships, the U-boat commander was probably very careful and first waited a time for the ship to sink and then surfaced to finish it off with gunfire. So the lifeboats were very likely further away from the torpedoed ship than the U-boat when it blew up.

There were several cases in WW2 in which U-boats were hit by debris from torpedoed ships. For example the explosion of EDWARD B. DUDLEY after being torpedoed by U-615, which was hit by debris, wounding the commander and forcing the Germans to abort their patrol.

As Spindler wrote, a truck crashing onto the strong hull of an U-boat will certainly damage it, but not enough to be the only cause for its loss.