Now now, Simon’s not just being mean to Mariah! The X-Factor boss explains that she would simply be too busy with her babies at home and family commitments. Simon says he would LOVE to have Madonna as a judge because of her experience and success. He already had Katy Perry work on the UK X-Factor as a guest judge and at the time, spoke about how impressed he was by her wit, her knowledge and her sassy nature. Simon wants us to know that “It’s all about the contestants.” But why does it always only seem to be about the celebrity judges. Let’s be honest, I don’t even remember who won!