Actual visibility at sea on a clear day (in miles) is 1.2246 x the square root of the observer's eye height above the sea (in feet). U-boats being fairly low in the water, say the lookout's eye level is about 10' above the surface gives a horizon of just under 4 miles. Of course, the target sticks up above the horizon, so if the highest visible part of the target in 40' above the surface, it can been seen (by that lookout) at a distance of about 11.6 miles (3.87 mi lookout's horizon + 7.74 mi, target's horizon).

Of course, what was most commonly the first thing spotted was smoke, which might be visible at 20 miles or more, presuming a skilled lookout (who could tell the difference between distant smoke and a dark cloud), and a few ships in the convoy with old coal-fired boilers to insure lots of black smoke).

Radar horizon is the same as the visual horizon from the same height, again plus the target's horizon distance, though radar is a bit more likely to notice something than a lookout. In the U-boat war, radar was mostly an enemy advantage in any case, most U-boats not having it, and the few that did probably (rightly) suspecting that the enemy could detect their signal a lot farther out than their radar could detect the enemy.

J.T. McDaniel