But they missed out on a shot at extra-time as Quins outside-half Gareth Cull saw his conversion attempt shave the post.
Centre Luke Ford's try for Neath eight minutes from time proved to be the match winner.
Division One West club Tonmawr also failed to take their chance of an upset, losing 23-16 at home to Premiership Glamorgan Wanderers.
With the game locked at 16-16, Wanderers capitalised on the sin-binning of hooker Rhys Lawrence as winger Jamie Jones scored the game winning try with ten minutes left.
Llantwit Fardre, the lowest ranked team left in the competition, could not fashion a shock as they were hammered 74-0 at Aberavon.
The Division Three South East leaders, who had been unbeaten in 18 games, were sent crashing out by the Premiership side who scored 11 tries, including two each for centre Richard Thomas and No8 Rory Gallacher.
Caerphilly are now the only non-Premiership club remaining in the Swalec Cup after they won an all-Division One East tie at Rumney 17-12.
Outside-half Dan Ellis scored the Cheesemen's only try and kicked the rest of their points.
In three all-Premiership clashes, Cross Keys edged out Pontypridd 27-26, Swansea won at Ebbw Vale 23-13 and Newport beat Llandovery 38-18.