Will Smith had to take "two years off" of work when he realised his family "wasn't happy" with the direction he was heading in.