Scottish lock Scott MacLeod and Irish flanker Simon Easterby crossed for tries, and the boot of Wales's captain Stephen Jones saw the Scarlets home.
The French side have struggled in the French Championship and have now suffered two Heineken Cup defeats after they were hammered 30-3 at Ulster in the first round.
Scarlets head coach Phil Davies was able to bring back his Wales stars including British Lions' half-backs Stephen Jones and Dwayne Peel.
Ireland flanker Simon Easterby was back in to bolster the pack while promising centre Gavin Evans was fresh from winning his first cap for his country. Wales wing Mark Jones provided plenty of pace on the flanks.
In front of a packed house at Stradey Park, the Scarlets had the best possible start when Macleod stole line-out ball from a Toulouse throw and charged over from close-range much to the delight of the home fans.
Wales captain Stephen Jones landed the conversion from out wide and kicked two penalties to put the Welsh region in control at 13-0 after just 17 minutes.
Yet Toulouse showed their experience by fighting back as right-wing Vincent Clerc crossed for an unconverted try on 22 minutes after a missed tackle from fullback Ceiron Thomas. Less than four minutes later the three-times European Champions were in touch at 13-12 down.
A missed tackle by open-side flanker Gavin Thomas in midfield had the home defence scrambling back and when the ball was spilled behind their own line, Toulouse fullback Clement Poitrenaud dived on the bobbling ball for a try. Scrum half Valentin Courrent landed the conversion to make it 13-12 at half-time.
The French side took the lead early in the second-half when sustained pressure from the Toulouse pack saw Clerc go over for his second try. Courrent added the conversion to make it 19-13 to the visitors.
The Scarlets hit back as fullback Thomas was caught in midfield after fielding a clearance kick, but British Lions wing Dafydd James was in support and burst upfield.
The ball was recycled before Llanelli No.8 Alix Popham fed his captain Simon Easterby and the 2005 British Lion crashed over. Jones landed the conversion from out wide to make it 20-19 to the home side with time running out.
There were plenty of Llanelli fans with their hearts in the their mouths in the game's dying minutes as Toulouse had two drop goal attempts charged down. Thomas and Jones both missed touch-finders which gave the French side's dangerous runners chances to run at the Llanelli defence, but the home side were sure in the tackle.
When Irish referee Alain Rolland blew for full-time, Stradey Park erupted.