This article is automatically translated.
Hello, this is Takahashi from Dwango Media Village. Last year, DMV developed a horse racing prediction AI called “Mamba” and carried out two horse racing prediction AI projects.
The first project, from March to June 2018, was a live broadcast on Nico Nico titled “Artificial Intelligence Donation,” where we actually purchased betting tickets. The project collected donations for animal welfare organizations from users, and Dwango matched the amount for horse racing betting, with the profits also being donated. Over the three-month period, the return rate was 135%, and even including the test period, the return rate exceeded 100% over four months. AI Donation Project Results: Horse Racing Prediction AI “Mamba” achieved a return rate of 135.13% during its operation period, doubling the total donation amount to 1,074,090 yen.
The second project, from October to December 2018, was a web service called “Mamba 2nd Season,” where Mamba evaluated users’ bets and revealed its own bets a few minutes before the deadline. Over the approximately three-month period, the return rate was about 131.7%, achieving another great success. Horse Racing Prediction AI Project 2nd Season Results: Achieving Better Results than the First Season with AI “Mamba” recording an average return rate of 131.7%, and a total profit of about 7.3 million yen.
In this article, I will briefly explain how Mamba predicts horse racing, and then show how Mamba calculated its predictions for the 2018 Arima Kinen, which was also broadcasted live on Nico Nico.
Horse racing involves numerous unpredictable factors such as falls, injuries, disadvantages, and the horses’ motivation and physical condition. Additionally, since the race is often decided by mere centimeters after the horses run over 1 km at full speed, there is an inherent limit to improving prediction accuracy, and it cannot be perfect.
| Mamba’s Top Predicted Horse | Most Popular Horse | |
|---|---|---|
| Single Win Hit Rate | 31.9% | 31.6% |
| Single Win Return Rate | 79.3% | 75.7% |
In the test dataset (all races in central horse racing from July 2017 to June 2018), the single win hit rate for horses predicted by Mamba to have the highest winning probability was about 31.9%, and the return rate was 79.3% (Table 1).
During the same period, the hit rate for the most popular horse was about 31.6%, and the return rate was 75.7%. Therefore, by this metric, Mamba is 0.3 points more accurate than the most popular horse. With about 3500 races per year, this means Mamba predicted correctly about 10 more times than the most popular horse. While not an overwhelming difference, Mamba’s return rate was 3.6 points higher than continuously betting on the most popular horse, roughly aligning with the payout rate after deducting the single win ticket’s cut. The odds formed by numerous people using various data for excellent predictions are quite accurate and roughly reflect each ticket’s hit probability.
However, when the goal of horse racing AI is set to “making a profit” rather than “hitting the target,” the approach changes. Instead of randomly selecting the horse with the highest probability of winning, Mamba compares predicted probabilities with odds and purchases tickets with high expected value. By doing this, Mamba aims to achieve a positive long-term profit even with a lower hit rate. For now, the method used by Mamba to estimate the hit probability of each ticket remains undisclosed.
| 単勝 | 期待値1.0以上 | 期待値1.1以上 | 期待値1.2以上 | 期待値1.3以上 | 期待値1.4以上 | 期待値1.5以上 |
|---|---|---|---|---|---|---|
| 全レース数 | 1675 | 1675 | 1675 | 1675 | 1675 | 1675 |
| 購入レース数 | 1560 | 1348 | 1081 | 867 | 665 | 504 |
| 購入点数 | 4383 | 3066 | 2090 | 1448 | 1021 | 713 |
| 的中点数 | 450 | 279 | 159 | 108 | 75 | 40 |
| 購入金額 | 44,142,500円 | 25,721,600円 | 14,254,300円 | 8,274,200円 | 5,174,700円 | 3,119,900円 |
| 払戻金額 | 41,764,650円 | 25,558,820円 | 14,608,360円 | 10,008,790円 | 6,924,570円 | 3,601,680円 |
| 収支 | -2,377,850円 | -162,780円 | +354,060円 | +1,813,700円 | +1,749,870円 | +481,780円 |
| 回収率 | 94.61% | 99.37% | 102.48% | 121.92% | 133.82% | 115.44% |
Table 2 and Figure 1 show a part of the investment simulation results for single win bets on all 1675 turf races in the test dataset for Mamba 2nd Season. The threshold was set in increments of 0.1, and all bets evaluated to be above each threshold were purchased.
For example, looking at the column with an expected value of 1.0 or higher in Table 2, there were 1560 races out of 1675 where at least one horse had an expected value of 1.0 or higher, and 4383 horses were eligible. The purchase amount for each bet was set so that the payout amount would be 100,000 yen based on the odds five minutes before the race. With 4386 bets and a total purchase amount of approximately 44.14 million yen, the purchase amount per bet was about 10,000 yen. This indicates that, on average, horses with odds around 10 times were being bet on in this group. Although there were 450 winning bets, the return rate did not reach 100%.
When betting on horses with an expected value of 1.0, the theoretical return rate should be 100%, but it often falls short. There are two possible reasons for this: first, there is an error in probability estimation, and second, the odds may fluctuate further after the purchase decision is made.
In this investment simulation, the purchase decision and purchase amount were determined based on the odds five minutes before the race, and payouts were assumed to be received at the final odds. This mirrors the actual environment of Mamba 2nd Season. Ideally, the total payout amount for 450 winning bets should be 45 million yen, but it was only 41.76 million yen. This implies that the odds decreased by about 7.3% after the purchase, so tickets with an expected value of exactly 1 five minutes before the race had their odds drop, resulting in an expected value below 1.
In Mamba 2nd Season, the payout amount was set to 500,000 yen for trifecta bets and 100,000 yen for other bets. Figure 2 shows the histogram of payout amounts for bets other than trifecta that Mamba hit during the project. The average payout was 91,840 yen, and the median was 92,460 yen, indicating that the odds dropped by around 8% as predicted. Moreover, for place and wide bets, the expected value was calculated using the lower odds limit, and the purchase amount was rounded up (e.g., if the odds were 300 times, 400 yen would be purchased), so the odds drop just before the deadline is even more significant.
In simulations, it is necessary to replicate the real environment as closely as possible and to explore purchase rules that can generate sufficient profits even if the odds drop slightly.
For the Arima Kinen held on December 23, 2018, the bet Mamba placed was a single 100 yen trifecta bet on 15-12-13. The reason for this is that all other bets did not meet the established purchase conditions. However, let’s examine the predicted probabilities for single win bets that Mamba did not place any bets on.
| 馬名 | 予測勝率（順番） | オッズ（人気） | 予測期待値 | 着順 | |
|---|---|---|---|---|---|
| 1 | オジュウチョウサン | 2.76% (11) | 8.9 (3) | 0.2458 | 9 |
| 2 | クリンチャー | 3.54% (9) | 31.6 (11) | 1.1176 | 15 |
| 3 | モズカッチャン | 8.96% (3) | 9 (4) | 0.8060 | 8 |
| 4 | マカヒキ | 2.30% (12) | 42.3 (12) | 0.9727 | 10 |
| 5 | パフォーマプロミス | 5.09% (6) | 21.2 (7) | 1.0798 | 14 |
| 6 | サトノダイヤモンド | 4.26% (7) | 17.8 (6) | 0.7586 | 6 |
| 7 | サウンズオブアース | 0.00% (16) | 137.1 (14) | 0.0028 | 16 |
| 8 | ブラストワンピース | 8.72% (4) | 9.8 (5) | 0.8542 | 1 |
| 9 | リッジマン | 0.01% (14) | 111.6 (13) | 0.0053 | 12 |
| 10 | ミッキースワロー | 3.33% (10) | 28.7 (10) | 0.9551 | 11 |
| 11 | ミッキーロケット | 4.09% (8) | 22.2 (8) | 0.9074 | 4 |
| 12 | レイデオロ | 40.61% (1) | 2.2 (1) | 0.8934 | 2 |
| 13 | スマートレイアー | 0.00% (15) | 187.8 (16) | 0.0183 | 13 |
| 14 | キセキ | 10.91% (2) | 5.9 (2) | 0.6439 | 5 |
| 15 | シュヴァルグラン | 5.38% (5) | 24 (9) | 1.2918 | 3 |
| 16 | サクラアンプルール | 0.04% (13) | 141 (15) | 0.0523 | 7 |
Mamba predicted that the horse with the highest winning probability was number 12, レイデオロ, with a probability of 40.61%. Since the odds were 2.2 times, the expected value was approximately 0.89.
The horse with the second highest predicted winning probability was number 14, キセキ, which was the second favorite in the race. Mamba estimated its winning probability to be 10.91%, indicating that it would win approximately once in every nine races. The odds for キセキ were 5.9 times, leading Mamba to determine that the bet was not worthwhile.
A noteworthy point was the prediction for horse number 1, オジュウチョウサン, which had the third highest odds but was rated 11th by Mamba. オジュウチョウサン’s switch between hurdles and flat races posed a challenge, as there was limited historical data, making it difficult for Mamba to make accurate predictions. Nonetheless, considering past performances of horses transitioning to flat races, Mamba concluded that winning in this group would be challenging.
On the other hand, Mamba rated horse number 15, シュヴァルグラン, fifth despite its ninth-place odds. Given its consistent performance in many significant races, including a third place in the previous year’s Arima Kinen, Mamba found the ninth-place odds to be favorable. シュヴァルグラン had the highest single win expected value among the 16 horses. However, Mamba’s threshold for placing single win bets in the Arima Kinen was an expected value of 1.4 or higher, and シュヴァルグラン’s odds fell short by 2.1 times, resulting in no bet being placed.
This concludes the overview of Mamba. The method of calculating probabilities and purchasing bets with high expected values is not unique, but Mamba uses machine learning technology to estimate the probabilities of each bet more accurately than traditional methods.
As many of you who followed the AI Donation program are aware, Mamba tends to refrain from betting on major races like G1 races. My assessment is that Mamba currently lacks the predictive accuracy to outmaneuver the meticulous predictions made by many people for such significant races. As the accuracy of predictions by betters continues to improve, all bets will likely have odds that match their probabilities, leading Mamba to refrain from betting more frequently.
