[FIXED] How to visualize a Regression Tree in Python

Issue

I'm looking to visualize a regression tree built using any of the ensemble methods in scikit learn (gradientboosting regressor, random forest regressor,bagging regressor). I've looked at this question which comes close, and this question which deals with classifier trees. But these questions require the 'tree' method, which is not available to the regression models in SKLearn.

but it didn't seem to yield a result. I'm running into issues because there is no .tree method for the regression versions of these trees (the method only exists for the classification versions). I'd like an output resembling this but based on a sci kit learn-constructed tree.

I've explored the methods associated with the objects but just cannot produce an answer.

Solution

As I commented, there is no functional difference between a classification and a regression decision tree plot. Adapting the regression toy example from the docs:

from sklearn import tree
X = [[0, 0], [2, 2]]
y = [0.5, 2.5]
clf = tree.DecisionTreeRegressor()
clf = clf.fit(X, y)

and then, similarly, some code from the classification docs regarding graphviz:

import graphviz 
dot_data = tree.export_graphviz(clf, out_file='tree.dot') 

we end up with a file tree.dot, looking like that:

digraph Tree {
node [shape=box] ;
0 [label="X[0] <= 1.0\nmse = 1.0\nsamples = 2\nvalue = 1.5"] ;
1 [label="mse = 0.0\nsamples = 1\nvalue = 0.5"] ;
0 -> 1 [labeldistance=2.5, labelangle=45, headlabel="True"] ;
2 [label="mse = 0.0\nsamples = 1\nvalue = 2.5"] ;
0 -> 2 [labeldistance=2.5, labelangle=-45, headlabel="False"] ;
}

Now, you can proceed to visualize it as shown in the docs - but if for whatever reason you cannot render Graphviz objects, you can use the handy service WebGraphviz (+1 to the relevant answer in the linked question); the result looks like this:

Your own answer, i.e. going all the way to install graphlab just for the visualization, sounds like overkill...

Last remark: don't get deceived by the superficial differences in the tree layouts, which reflect only design choices of the respective visualization packages; the regression tree you have plotted (which, admittedly, does not look much like a tree) is structurally similar to the classification one taken from the docs - simply imagine a top-down tree, with your odor node at the top, followed by your green nodes and terminating to your blue & orange nodes (and replace "yes/no" with "True/False")...

Answered By - desertnaut