This tree is very accurate, but we would like to reduce the size to make it more understandable and easier to inspect. The original tree uses the traditional approach of splitting on one variable at a time with so-called parallel splits, which are limited in power and often require large trees to achieve high performance. We can improve the power of the splits by considering more than one variable at a time, yielding hyperplane splits and hopefully smaller trees as a result.

In the past, decision trees with hyperplane splits were impractical to fit on even moderately-sized datasets. However, it is straightforward to extend the global optimization model that powers Optimal Trees to incorporate hyperplane splits. This means that Optimal Trees are unique in offering a practical way to train these hyperplane split trees at scale, allowing us to revisit the problem with more modeling power.