A former Wales Under 18 and Under 20 coach, King previously spent seven successful years at Aberavon. He led the Port Talbot club to a third placed finish in the Principality Premiership in 2009 and 2012 and to a runners-up place in the 2011 SWALEC Cup final.
King said he was looking forward to the challenge ahead at the Arms Park and targeted success both in the short and long-term.
"It's a fantastic opportunity and I'm absolutely delighted to be given this role," said King.
"I have experience at this level and now look forward to working at a club with such a rich heritage and proud history.
"There is undoubtedly huge potential at Cardiff RFC and if we can tap into that and drive everyone forward we can look forward to a very bright future at the Arms Park.
"There are really good people working in this region and at Cardiff RFC, passionate people who have a real affinity to the Blue and Blacks. I'm looking forward to working with them and taking us forward.
"The players can now look forward to being really challenged. We will challenge individuals and the team group and where they want to go. I'm confident that we can bring the very best out of them all.
"I had a fantastic seven years at Aberavon and the chance to get back into coaching at his level was a huge pull to me," he added. "I can't wait to get started and we have a lot to still play for this season.
"We have a good opportunity at home against Neath on Saturday and then seven more games in the Principality Premiership season and a home quarter-final in the SWALEC Cup to look forward to.
"There is a lot of rugby for us to get stuck into this season and I look forward to what we can achieve."