It is the judges and not the acts currently making the X Factor headlines, with Simon Cowell now just 3/1 to return to the panel for the next live show.
Viewing numbers have been falling and the latest figures show that rival Saturday night programme Strictly Come Dancing attracted a bigger audience for the period of time the shows were on air together.
This has frustrated boss Cowell and his return may be imminent to boost numbers again.
He may not get a more ideal to retake his seat if he wants to return without making it as a direct response to the lack of viewers.
The fall-out between fellow judges Kelly Rowland and Tulisa shows no sign of clearing up and he could make a case of returning to settle this dispute.
Kelly is holding a grudge after one of her acts Misha B was criticised by Tulisa for apparently effectively bullying other contestants backstage and having too much of a feisty attitude.
Obviously Kelly did not take too well to these remarks, but arguably did not go about defending her act in the right manner and to some has not come across well on the live shows thus far.
The result of this could prove that her acts are more at risk of elimination than before as voters turn against Kelly.
It is now just 4/5 that one of Kelly’s girls wins the X Factor and 66/1 could be a big price that Misha is eliminated given the developments of ‘Bullygate’.
However, there is also the argument that has one of the most favoured contestants among the judges, if she was not voted for by the public, she would likely to saved in the sing-off.