The Division One East leaders trailed by just two points at half-time after a brace of tries from Lenny Woodard cancelled out scores from Jimmy Norris and Deri Mattravers.
But the match was decided by centre John Hawker's try on the 55th minute, which was the only score of the second half.
Bedwas failed to recover and became the first and only Premiership team to depart the competition at the first hurdle.
There were routine wins for most of their Premiership counterparts, with Swansea picking up the biggest win of the fourth round, hammering Bala 104-3. Ospreys squad member Leigh Bevan crossed four times against the Division One North visitors.
Bridgend saw off local rivals Pencoed 37-6 at Brewery Field by running in six tries while Llandovery went one better in the 45-0 win at Beddau.
Welsh Premiership rivals Aberavon, Newport and Pontypridd also recorded comprehensive victories to ease into the fifth round.
But Pontypool were given a scare at Bonymaen before pulling through 13-5. Pooler went behind on ten minutes through a try from winger Robert Jones before first-half replies from Will Kershaw-Naylor and Calwyn Morgan gave the visitors a narrow 13-5 victory.
There was a dramatic finish in Llanelli, as Felinfoel led Caerphilly deep into injury-time. With the score tied at 31-31, having scored five tries to Caerphilly's four, the Carmarthenshire club looked to be heading through on the try count rule.
But Felinfoel lost their lineout with seconds remaining and outside-half Danny Ellis touched down under the posts to clinch victory for the visitors.
There was just time left for Ellis, a second-half supersub, to kick the conversion before the referee blew up to allow Caerphilly to go through 38-31.