Don't believe that our method of estimating hit potential works? On this page we'll try and convince you! Below is a detailed graph of our system running from the 1960's until 2010.

Performance Overview

In the graph below, time increases along the x-axis, and performance (measured by number of correct examples over number of examples) up the y-axis. The dark blue line shows the performance over the years.

Although we made sure that the number of songs in the 'hit' class was nearly equal to the 'flop' class over the entire dataset by choosing the classes to be 1-5 vs 30-40, note that it might be imbalanced in certain periods. To check that the performance of our equation can't be explained by just predicting the most common class more commonly than the less common class, the light blue line shows the performance we'd expect if we used this tactic. The dark green and light green lines show the predicted class proportion and actual class proportion, which we used to assess how well this tactic would work (i.e. to compute the light blue line).

When the light green line (actual class proportion) is far from 0.5, there are more hits than flops or vice-versa, in which case preferably (even if randomly) predicting the majority class would already lead to an accuracy higher than 50%. The dark green line (predicted class proportion) shows that our equation is indeed trying to take advantage of this to a certain extent. However, the fact that the dark blue line is above the light blue line means our equation is doing better than if it only exploited this varying class imbalance over time. Good! This means our equation is also picking up a signal actually differentiating between the hits and the flops.

Finally, the cumulative performance is shown as the black line. As you can see, the most recent cumulative performance is around 60%.