There is a tipping competition running at the moment on Twitter called Handinaps. You have to make 2 selections in every heritage handicap run this year. I have constructed two Machine Learning models to provide entries into the competition. One is built around races over greater than 8f and the other up to 8f in distance. Today I got around to playing around with the question of whether class or distance is a filter we should consider when constructing a model. By that I mean would a model built on all handicaps work just as well a number of models, all using the same input features but constructed only on data for 5f and then data for 6f etc etc. With class it would be a question of constructing data on class 6 races and then class 5. Would the sum of the parts be more predictive than one large model?.
To get a feel for the answer to this question I took a model I had created and trained it on all flat handicap data for 2011 to 2017 using MySportsAI. I then tested the model on 2018 to 2019. The results were as follows

So this is the baseline model. First of all I decided to see if there was any promise by binary chopping the data. That is to say I created two models for class and two for distance, The two class models were for handicaps class >= 5 and handicaps class < 5, here are the results
Class >= 5
VROI% top 3 -0.09% VPL top 3 -2.5
PL top 3 +676
Class < 5
VROI% top 3 -0.4%
PL top 3 -144
We can see the two models have not come close to matching the monolithic baseline model based on these PL metrics. Now lets take a look at race distance. Here I trained and tested on race distance > 8 furlongs and then <= 8 furlongs
Dist > 8
VROI% top 3 +0.47% VPL top 3 +10
PL top 3 +590
Dist <= 8
VROI% top 3 +0.78% VPL top 3 +24.2
PL top 3 +305
This is much closer to the overall performance with no criteria split. I therefore decided to drill down further into the individual race distances
Dist = 5
VROI top 3 +1.32% VPL top 3 +9
PL top 3 +42
Dist = 6
VROI top 3 +1.48% VPL top 3 + 11
PL top 3 +218
Dist = 7
VROI top3 -2.61% VPL top 3 -19.8
PL top 3 -304
Dist = 8
VROI top 3 -0.39% VPL top 3 -3.88
PL top 3 +69
Dist = 9
VROI top 3 +0.48% VPL top 3 +0.8
PL top 3 +217
Dist = 10
VROI top 3 +5.31% VPL top 3 +36
PL top 3 +137
Dist = 12
VROI top 3 -2.97% VPL top 3 -18
PL top 3 +221
Dist > 12
VROI top 3 +2.55% VPL top 3 +15.4
PL Top 3 +426
The above is close to the performance of the overall baseline model but not encouraging enough to think that splicing the model into sub models by distance is worthwhile. Of course the input features play a part, your model may improve for such splitting and I encourage you to experiment. Incidentally one hot encoding the race distance into the baseline model and then training and testing did not improve matters.
Was improvement found anywhere, well yes actually but not down the routes I was playing with above. Can you spot the difference ?
