Tulisa and other X Factor judges must accept their role in final five