I noticed on the sketch attached to Lambrecht's report that their flight path over Gardner approached from the NE. To an observer camped near the Norwich City at 08:00 on an early July morning, that direction would have been obscured by the trees and roughly in the morning sun. In other words, there would have been no warning of approaching aircraft to anyone who might have been in that location.
Almost nothing is recorded of the detailed flight path over the island.
The flight departed to the SE.
What would a rational survivor on the NW part of the island do if a flight had suddenly appeared from the East, and departed to the south east? Staying on the NW part of the island when evidence of help might seem to be to the East and South?
One more thing I noticed from the sketch - Lambrecht's plotted return path to the ship passed within roughly 15 NM of Gardner, also to the east of the island, travelling from south to north. That's close enough to be heard and seen from Gardner, under decent conditions. The winds at the time were "northeast to east, 13-15 knots...", making the east side of the island in the lee of the wind, and therefore quieter and better able to hear aircraft in the distance.
To get from one side of the island to the other is a slow process. Anyone seeing aircraft overhead on the NW shore might not have time or inclination to move to the Eastern shore. They might not even be aware of the direction of approach, nor of departure.