Last time on Supervised Learning’s Got Talent, we were introduced to the talented decision maker of the show, Logistic Regression.

Today, we’re going to branch out and meet our next Supervised Learning contestant. This contestant, like Logistic Regression, is also skilled at making decisions, but the difference is?

They are a lot more detective-like in their process.

Introducing the Decision Tree!

Remember how Logistic Regression is good at making decisions? Well, Decision Trees do this too (I mean, it’s right in their name, right?), but with a key difference:

Decision Trees ask a lot of questions before getting to a decision.

Imagine that you’re trying to decide whether to go out for ice cream. What are you going to do? You’ll probably ask a series of questions:

Notice how each question is relevant to the decision? That’s exactly how a Decision Tree works!

It’s literally a flowchart of questions that leads to a decision or a prediction.

At their core, Decision Trees break your data into smaller and smaller chunks by asking yes/no questions at every level—like a nosy neighbor who won't stop until they know everything about your weekend.

Root Node: The topmost node representing the most important feature that splits the data first.

Internal Nodes: These are points where the tree makes decisions based on certain features. Each node asks a yes/no question to help further divide the data! Kind of like mini crossroad signs within the tree.

Branches: These are the paths you follow after answering a question at a node. They lead you to the next node/outcome, like following a breadcrumb trail.

Leaf Nodes: These nodes provide the final decision/result. Think of them as the end of the trail where you get your answer or prediction, such as classifying data or predicting a value.

Let’s say you’re trying to decide what show to watch on Netflix.

It all starts at the Root Node, with a big question like “Am I feeling adventurous?”

Based on your answer (that’s a branch!), you travel down to an Internal Node for another question, say, “Is my partner awake?”

You keep following branches through more internal questions until you land on a Leaf Node - that’s your definitive choice, such as “Rewatch Game Of Thrones!”.

Here’s yet another example of the Decision Tree in action:

If you’re intuitive, you may say to yourself: Okay, yeah, great. Decision Trees ask lots of questions, but how do they know which question to ask next?

The answer to that is something called Information Gain and GINI Index.

Information Gain is all about finding the question that makes your data less confusing. It’s based on the concept of Entropy: the measurement of how messy your data is. Here’s the formula:

One more thing I want to show you is something called the GINI Index. It measures how mixed or impure a dataset is.

While Information Gain asks, “How much cleaner did I make this data?”, the GINI Index asks, “How clean is this pile?”

It measures impurity directly, by following this equation:

And that’s the Decision Tree. The best way to actually make this stick is to get your hands on it. Run the code, and test your knowledge.

Let’s now decide to practice.