The NR Conspiracy

In the 8.10 at Chelmsford today there are bang on 16 runners in a handicap. Bookmakers do not like 16 runners in handicaps, perhaps even more so than 8 runners in a handicap. It always seems however that they have little to worry about. Doesn’t one always come out and drop the runners to 15 or less. The punter friendly each way terms of 16 runner handicaps seemed to be inevitably snatched away from them by a non runner or two.

Is something going on here or are we imagining it. Is this the equivalent of a football manager bung, with trainers getting a brown envelope for saving the bookmakers thousands of pounds ?.

Lets look at some figures covering 2009 to 2011 inclusive for handicaps across both Flat and NH.
It shows the number of decs, number of races for that many decs, number of NRs and the NR percent expressed in relation to total decs for that dec size race.
This is for hcps only.
We can see that for 8 dec races there was 1646 such races and they were subject to 1058 NR’s which is 8.03% ie 1058 / (1646 * 8).
This is below the total NR percentage across all races of 8.62%
16 decs by comparison stand at 10.39%
Is there a confounding reason why 16 runner races would get more NR’s as a percentage ?
There does seem to be a spike around the 15,16 and 17 dec’ mark although quite why 15 decs should be spiking is a puzzle

Decs NumOfRaces NRs NRpercent
1 1 0 0
2 3 0 0
3 22 6 9.090909
4 148 30 5.067568
5 468 143 6.111111
6 863 375 7.242178
7 1199 580 6.910521
8 1646 1058 8.034629
9 1554 1104 7.893608
10 1630 1374 8.429448
11 1550 1416 8.304985
12 1740 1769 8.472222
13 1312 1535 8.999765
14 1369 1756 9.162058
15 529 853 10.74984
16 613 1020 10.39967
17 420 738 10.33613
18 247 397 8.929375
19 61 95 8.196721
20 170 260 7.647059
21 22 53 11.47186
22 32 61 8.664773
23 7 2 1.242236
24 28 25 3.720238
25 3 4 5.333333
26 3 11 14.10256
27 10 18 6.666667
28 10 17 6.071429
29 8 9 3.87931
30 7 6 2.857143
31 2 9 14.51613
32 2 7 10.9375
34 2 3 4.411765
35 2 3 4.285714
36 1 4 11.11111

Newcastle AW Pace

It is perhaps a little early to be evaluating the pace angle at the new Newcastle AW track but I thought an update on how things are measuring up might be in order.

So far working to BFSP before commission and using SmarterSig pre race pace figures and I emphaseise pre race here, we have the following figures

Hold up ie Less than 1.4 pace figures for prior race runs 372  wins 31 PL -86pts

Prominent ie pacefig greater than 1.4 and less than 3.2  runs 308 wins 35 wins PL +47.7

Led ie pacefig greater than  3.2 runs 114 wins 9 PL -59.55

Horse with a prominent run in their previous race as opposed to held up or led have done very well next time out at Newcastle so far although the duration and sample sizes are very small.

ITV More Of The Same

We are soon to get a new TV presentation team for Racing with one or two familiar faces still in place. There is much debate at the moment with sighs of relief as some are left out whilst groans as others are not included. The truth is that the ITV team is much like the next USA government or indeed our own UK government. The faces change but the underlying driving force and message remains the same. The illusion is that you are getting something new whilst the truth is that you are in for pretty much the same.

So what is the ‘same’ and why do I no longer listen to TV pundits who in reality should come with a wealth warning.

First of all we all know that the punditry will be geared towards protecting the main sponsors namely the bookmakers, this is even more so on a commercial run TV channel. This is not all bad if you learn to recognise how this anti punter stance can work to your advantage. TV pundits provide one huge disservice to losing punters and paradoxically a positive service to winning or potentially winning punters. They pretty much all to a man or woman approach a race from the point of view of finding the winner. Sure every now and then they mention that so and so is too short for a bet but but this is hardly value betting.  Anyone who has managed to move from being a losing punter to a winning one over the years will without doubt have first suffered from the whats going to win mentality inflicted on them by the media. Once you have twigged that those on the screen either do not know how successful punting works or prefer the short term safety of finding a few winners over trusting that the watching public might stick with them with more riskier long term selection methods that actually yield a long term profit. But don’t lets moan about this, the fact that your presenters are incompetent is a bonus. It keeps the majority of punters in the dark which is where they need to be if you are to keep winning.

That last sentence might sound a little harsh but the reality is that most punters have to lose in order for some punters to win. The truth is that they dont have to lose as much or as rapidly as they do. If every losing punter switched to the exchanges they would immediately as a group lose less without impacting on winning punters in a negative fashion, in fact quite the opposite.

So in conclusion do not complain about the dumbing down or the lack of smart punting advice without it your life would be a lot harder unless of course you really are listening to them.

Six Degrees of Separation

What do Sean Bean the actor and I have in common. At first I thought it might be that we are both from Sheffield, or maybe that he studied drama at Rotherham College of Art and Tech where I taught for 4 years back in the 80’s. Even closer to home is that my 81 year old cycling buddy in Portugal is a guy who has a regular ‘last of the summer wine’ Friday meet up with his mates in Sheffield of which one used to be Sean’s dad until he passed away. It could also be that we both support Sheffield United and Idolise Tony Currie, the best English midfield player of the 70’s.

But closer to home for me is that a playwright called Steve Wakelam, a yorkshire lad, wrote a play back in the late 70’s about two young lads who try their arm at professional punting. I know Steve although my friend,and the guy who introduced me to Racing, knows him better having been taught by him when Steve was a school teacher. I met Steve on several occasions at our annual York races soiree in August and I have always been aware that the play was based on my friend John and myself. What I did not know until recently is that it was filmed as a BBC1 play with Sean playing what appears to be the third lead role (alas not John or I). I have not seen it but at least it would be one role in which Sean would not have a problem with the accent.

I do not think the two parts are quite distinct in terms of me and my mate John, rather they appear to be an amalgamation of both of us. My friend did work as a groundsman at a monument and does have a more romantic view of Racing whilst I hold the more hard nosed Mathematical viewpoint. The second character who appears to be the proverbial loser, along for the ride,  is hopefully purely a fictional character.

AE Ratings V Random Forests

I have spent the last few days working on a Random Forest version of my own flat handicap ratings. The original ratings are based on AE values or Actual divided by Expected values to give them their full name. Let me remind you of what AE values are. If we are say calculating the AE values of last time out winners, we can look at all lto winners and for each horse calculate its market chance by taking its SP or BFSP, stripping out the over round and then take the odds as its chance of winning. So an even money shot should win 0.5 times if the odds are true. We sum up all these win chances along with the actual win count for these horses. this gives an E (expected) value and an A (actual) value. If we divide the A value by the E value and it is greater than 1 then, in our example, last time out winners are winning more times than the market estimates. If the value is less than 1 then the market is over betting them.

I trained a Random Forest model on my data for 2009 to 2013 and then tested on the years 2014 and 2015. The original AE model produced the following results for top rated horses.

Bets 7918 Wins 1276 PL +305 to BFSP after comm’ ROI +3.2%

The Random Forest model produced the following results for top rated horses

Bets 7699 Wins 1164 PL +323 to BFSP after comm’ ROI +4.1%

The software used was Python with the Skicit Learn Random Forests library. See my intro blog entry on this software.

The initial interest in this area stems from an excellent article published by Stefan Lessman which is linked below

The next step for me is to extend the model by taking the Lessman and co’s example of moving to a second step of using the resulting RF ratings and combining with the market price of each horse using regression to eventually produce an oddsline. Of course BFSP is not known until after the off but final prices can be a good estimation. Lessman and Bentner argue this two step separation of fundamental race parameters and odds to stop the odds swamping the model parameters when used together at the same time.

I should also perhaps look at some more metrics on this model first as it may have not escaped your notice that the win rate on the AE model is greater than that of the RF model. Lessmann puts up some strong arguments for Random Forests in his article so if you are interested in race modelling it might be worth taking a look.

Watching Frankel

Today saw the first son of Frankel make his debut in the UK and this also coincided with my finishing the sequel to that excellent book Watching Racehorses by Geoffrey Hutson, the obviously named Watching more Racehorses.

I loved the first copy which attempted to numerically represent those soft subjective observations we get thrown at us every weekend by so called paddock watchers. The new book is not as good simply because it is padded out somewhat with observations on areas outside the paddock. Nevertheless it still adds more data to some of those familiar and unfamiliar areas of paddock watching. For example in the first issue sweating is not cited as a negative but in the second issue he puts more meat on this observation by stating that when the temperature is above 21c sweating is not a negative. Another interesting observation is that coltishness is also not a negative.  So what is a negative, well if you want a negative you can get your teeth into sample size wise then consider cross nose bands.

How does this all relate to Cunco the son of Frankel who has just bolted in. Well he drifted like a barge after becoming coltish in the parade ring. Only he and Mr Hutson seemed to know.

Betfair SP’s Part 2

In my previous blog post I mentioned the care needed when doing research to Betfair SP. This was courtesy of an alert by an observant member of the SmarterSig email forum.

Today I will demonstrate just how much difference this anomaly can make. At the moment I am tracking a betting method based on a combination of racing selection strategy and financial trading methods. At first glance the option of betting to BFSP seemed more attractive than taking a price provided you can find some sort of stake threshold by which you do not cannabalise your own BFSP with the size of your stake.

Using the odds displayed when you download your betting summary to calculate a level stake PL to BFSP I get the following results when comparing backing the selections to available price compared to BFSP.

Available price,  Bets = 1717 PL = +27.3 points after comm

SP Price, Bets 1717 PL = +35.2 points after comm

A small increase using BFSP

Of course things are never that simple and the  prices handed to me via the Betfair download do not account for R4’s. Now taking the the profit by calculating the winnings divided by the bet stake we get the following profit for the two categories

Available price Bets 1717 PL -3.04 points after comm

Bets 1717 PL = +22.1 points after comm

The profit from then live prices simply has not survived the R4’s occurred during the time from taking the bets in the last few minutes to off time. The BFSP’s however will have fewer R4’s, perhaps only being affected by markets that have not reformed perhaps due to a stall non entry. There was a 0.7% drop in ROI when the R4’s on BFSP were accounted for.

Conclusion – You need to make sure when calculating points profit on bet summaries that you use the profit divided by stake and not the price to calculate. Also when assessing new strategies retrospectively to BFSP you need to account for late R4’s. A reduction of 1% on ROI would seem prudent.

Betfair SP’s

I have mentioned before about the fact that Betfair SP’s seem to produce a race overround or should I say underround, below 100% on a good number of races. A possible explanation for this was put forward by a member of the Smartersig email forum

The member stated the following which I have to admit I had overlooked.

I assume that I’m not alone in using Betfair SPs as the benchmark to assess the profitability of a potential new system.  Of course there’s an argument that this isn’t entirely accurate as your own theoretical bets might have altered the BFSP but nothing is perfect.

However, I recently noticed that Betfair SPs are NOT recalculated to allow for Rule 4 deductions after a late withdrawal.  I know that Betfair apply their own deduction to any bets (whether a price was taken or BFSP) but had assumed that SPs would be re-normalised after the race to account for this.  Unfortunately it seems they are not, and the historical BFSPs that are released by Betfair in CSV format (or on the Timeform website) do not account for withdrawals.  As far as I know there’s no easy way to get this information, so it means that the profit/loss of any system researched using Betfair SPs is flawed because of this.

The other thing to watch is dead-heats as this will also affect the bottom line.  It’s relatively straightforward to calculate in Win markets but it becomes more complex in place markets.  For instance if there’s a 6-runner race with 2-places paid out in the place market and your horse dead-heats for first place then the dead heat is irrelevant.  It will be treated as a full-stake bet.  However if another horse wins and your horse dead-heats for second in the place market then your return will be calculated to a half stake.  In other words there’s no ‘one size fits all’ solution for dealing with dead-heats.

The main point is about the Rule 4’s though.  Just wondering if anyone else has dealt with this issue before?  it’s hard to assess how much it might actually affect the ‘true’ bottom line of a researched sequence of 1000s of bets.

Clearly a pinch of salt is needed when assessing any betting approach to Betfair SP which these days is the common method used. There is probably less of a problem with NH racing as the bulk of late withdrawels will be stll problems on the flat.
One possible simple solution would be to readjust all prices so that a book which comes out at sub 100 is normalised to around 101 or 102% which is more likely to reflect the true BFSP after R4’s. If there is any interest in this topic I could look into it further. If the writer of the above is correct then there should be some whopping underrounds when an even money shot gets withdrawn at the stalls.

Do The Survey

If you have ever had an account closed or restricted, and I guess that’s just about everyone, (If you have not then you are doing something terribly wrong with your betting) then please do the HBF survey into the problem and maybe just maybe we can get something done. Please tweet, blog, wear a T shirt and anything it takes to get the word around

Account Restriction/Closure Survey

Which would be more lucrative to Racing and even to Bookmakers

1. Restrict and close all punters that beat prices and show any inkling of a brain and hence see the betting turnover shrink as existing punters turn away and new punters do not arrive. Kind of like a greater ROI but lower turnover for racing and bookies.

2. Allow an Aussie type system where winners allowed to certain individual bet liabilities. Winners now feel free to be more visible which in turn encourages new players into the sport who of course in the majority will lose. Net result lower ROI but much greater turnover = greater profit than option 1

In many ways the above scenario is rather like betting on Exchanges V betting with bookmakers. With Bookmakers you can make a greater ROI but you cannot get much on if anything at all. When you switch to bet on the Exchanges you have to learn that ROI will shrink so turnover must be vastly increased.

Oddslines a Line In The Sand

I have been busy this week not just watching Cheltenham but also working on a Bayesian set of flat handicap ratings or should I say a Bayesian version of my flat handicap ratings.What has prompted me in this direction are two things. First of all my ratings do well but less well towards the off time of races which is a pity because the greatest liquidity on Betfair exists the closer you get to the off. My reasoning therefore is to try and produce the most efficient oddsline with the minimum amount of time and effort and with Bayes producing natural percentage chances of winning it is hoped that this may prove a way of cranking up the action towards the off.

The second reason is that I have already had some success in this area and hence the title of this post. No I do not have a wonderful oddsline approach to AW racing as the title suggests. What I have done in a completely unrelated area to my ratings is produced a novel oddsline approach.  Now forgive me, for obvious reasons, if I do not spell out the nuts and bolts of this method, but I do feel there is a more general message which is worth passing on and could be of use to some of you.

My ‘novel oddsline’ approach initially did not generate profits, if memory serves me right I think it produced a small loss on Betfair. What I did next was simply to move the dividing line which specified bet or no bet. In other words where I had previously simply bet if the odds were greater than X I changed this to bet where the odds are greater than X + y%. I moved the line in the sand so to speak. This has now generated around +6% after commission profit off around 5,000 bets all under 10.0 on Betfair.

For older members of SmartSig this may well have a familiar ring to it. Some of you may recall an excellent article in the original SmartSig mag’ by an author called Filly. In this article Filly’s main point, if memory serves me well, was that just about any sensible approach to ranking or selecting horses could be made profitable by moving the threshold from a price point of view. At the time I was not sure whether I could buy into this. Some members were even wondering whether a forecast such as the Racing Post could be used and then, using an appropriate percentage above the forecast, a profit would ensue.

It seemed at the time to be pushing the boundaries of acceptance but the more experienced members of the group were more in the Filly camp of thinking than out. In fact I seemed to recall that it either won the article of the year award or came second.

I will keep you informed of progress on the Bayes approach but in the mean time remember not all fillies are bad bets.

If you recall the article or have some personal opinion on oddslines in general please leave a comment. All bloggers appreciate some sort of feedback.