ppforest2

A couple of years ago I started thinking about my Statistics thesis. I had already finished a degree in Computer Science, so I didn’t want a textbook exercise. I wanted to treat it like a master’s project: something real, that somebody actually needed, where the software engineering would count as much as the statistics, and where I could push the scientific rigor further than I had before. I wanted real experience in scientific software development.

That search took me to Natalia da Silva, the person working on computational statistics at the Faculty of Economics, which houses the Statistics program. She introduced me to her research on oblique trees, a family of methods studied extensively in the literature but with no consolidated reference implementation behind it. That gap was exactly what I had been looking for.

The method

Most tree ensembles split on one variable at a time. A node asks “is c·x3 < t?” and sends the sample left or right. It is simple, fast, and interpretable, and it draws every decision boundary parallel to a coordinate axis. When the structure in the data runs diagonally across several variables at once, an axis-aligned forest can only approximate that diagonal with a staircase of axis-aligned splits.

Projection Pursuit Random Forests (PPF) take a different route. Instead of picking a single variable at each node, they search for a linear combination of variables, a projection, that best separates the classes. The node asks “is a·x1 + c·x3 < t?”, and splits on the answer. The boundaries become oblique, and a diagonal boundary that an axis-aligned tree can only stairstep toward is captured by a single cut. Choosing that projection is projection pursuit: you define an index that measures how well a direction separates the classes, and take the direction that maximises it. The idea is old (Friedman & Tukey, 1974); pointing it at class separation is due to Lee et al. (2005), and combining such trees into a forest follows Breiman (2001).

That index comes out of linear discriminant analysis, either LDA or PDA for correlated or high-dimensional data (Hastie et al., 1995). The result is a parsimonious representation of a diagonal boundary, but it still lacks flexibility: an LDA/PDA oblique tree gives each group exactly one decision region, however well oriented that region happens to be.

ppforest2 is my implementation of that idea: a C++ core with R and command-line interfaces, built as the successor to the R PPforest package (da Silva et al., 2021). Natalia signs it as an author, and it is my Statistics thesis at the Universidad de la República, which she advised.

What the next step needed

The method was not short of implementations. PPtree (Y. D. Lee et al., 2013), now archived, and PPtreeViz (E.-K. Lee, 2019) covered the trees; PPforest put the forest around them. To make the ensemble feasible Natalia reimplemented the pieces she needed and got that package onto CRAN. And PPtreeExt (da Silva et al., 2026) is where she started on the extensions, among them more than one decision region per group. The lineage runs through one group of people: the authors behind those packages are her co-authors. Everything here builds on their work.

What the method did not have, when I started, was a single consolidated implementation solid enough for the next step to stand on.

The landscape moved while I worked. ODRF appeared partway through (Liu & Xia, 2025), with a scope close to this one: oblique forests in general. Projection pursuit is among the split methods it offers, and its PP is built on PPtreeViz, the same lineage. But it is not where the package’s weight sits. Its main method fits each node’s coefficients by regression.

The questions that came next were the expensive ones. Growing a forest means fitting thousands of trees, and at every node of every tree you solve for the projection that best separates the data reaching that node. That cost is already showing on large datasets, and the extensions Natalia started in PPtreeExt would multiply it again once they reach the forest.

Studying the method, and improving it, means paying that cost over and over: across datasets, across repetitions, across the configurations worth comparing. Performance, in time and in memory, was the barrier standing in front of that. The faster and leaner it runs, the cheaper it is to iterate, and the more the scale of what you try is a statistical decision rather than a budget the machine imposes.

The design

ppforest2 was never meant to be only an R package. The target was efficient, portable, reproducible software that happens to implement a statistical method. That is a different thing to build than a package that works from one language.

The shape follows from one decision: put the hot path in C++, and keep everything else thin. The method lives in a standalone C++ library that owns its own memory; the R package and the CLI are thin adapters onto it; training runs in parallel across trees.

Keeping the interfaces thin buys more than speed. There is one implementation of the method to reason about, test, and get right; not three that can quietly drift apart. A Python binding would be another adapter rather than another rewrite. And with few dependencies to drag along, the core is something you can deploy somewhere that is not a research laptop.

Thin does not mean foreign, though. An adapter still has to speak the idiom of the language it lands in, so the R package registers a parsnip engine: it slots into the tidymodels pipelines people already have, instead of asking them to build something around it.

Finally, the package also had to be a laboratory. ppforest2 does classification and regression, but the point is that it is extensible: those multiple partitions have to be implementable without fighting the codebase. Research software that cannot absorb the next idea has a short life.

Where it stands

ppforest2 is on CRAN: install.packages("ppforest2"). Source and documentation:

• CRAN: cran.r-project.org/package=ppforest2
• Source: github.com/andres-vidal/ppforest2
• Docs: andres-vidal.github.io/ppforest2

This is the first of several posts about building it. The ones that follow go into what this one skipped: whether oblique forests actually earn their extra cost, how a projection pursuit index picks a direction, what it takes to move a compute-bound method off interpreted R, and how you test something whose output is supposed to be random.