"Having not fired anybody for the first five years, [firing Damon Wayans] was really, really hard," the producer admitted in ...