Banfield almost made up for his error five minutes later when he cut through the Swansea defence with lock Gavin Evans in support. The ball was then swept out wide, but the Whites were penalised for killing the attack, though James Dixon missed his effort at goal.
The home side were awarded another penalty when the visitors were guilty of blocking but Dixon again missed the target, this time from in front of the posts. Swansea centre Aled Jenkins then made a dangerous break but the supporting Rhys Williams lost the ball forward.
Whites No 8 Rob Dudley-Jones spotted a gap in midfield and raced up to the twenty-two but the supporting Scott Otten lost the ball forward. The home side got their first points on the board courtesy of a Banfield penalty after the Whites' pack transgressed at the ruck.
The Quins sneaked ahead when a break by winger Darren Daniel forced the Swansea defence offside. Banfield kicked the penalty goal. Five minutes later centre Jason Harries made a half break and found Evans, but the lock lost the ball five metres short of the try line.
Swansea retook the lead five minutes into the second half with a Clement penalty after Quins centre Dylan Morgan was yellow carded for persistent offside. The lead lasted just a minute though as the Whites defence killed a dangerous attack to allow Banfield a third penalty goal.
Almost immediately after Morgan returned from the sin bin it was Swansea's turn to be temporarily reduced to 14 men when Otten was sent to the sidelines for killing a dangerous attack.
With 66 minutes on the clock the Quins got their first try when a break up to the line by flanker Rory Gallagher forced the Swansea defence onto the back foot, before substitute Rheon James took a crash ball to score. Banfield converted.
Swansea looked to hit back straightaway when scrum half Jake Cole raced into the opposing twenty-two, but replacement hooker Ieuan Dobbs was unable to take the pass and the move broke down.
In the closing stages Swansea were reduced to 13 men after substitutes Dobbs and David Blyth were shown red cards by referee Andrew Davies - Blyth for a stamp and Dobbs for a punch. Deep inside injury time the two man advantage paid off for Carmarthen when their pack earned a penalty try. Banfield converted.
Next up for Carmarthen is a trip to Aberavon, while Swansea host league leaders Pontypridd.