30 Days of Python #23: Predicting Patients You Have Yet to See

"Reading Data" is a series on Python and machine learning for clinicians and medical researchers. We start by acquiring programming skills to sharpen our ability to "read and interpret" our own data. In this #23, we will have a trained decision tree predict patients it has never seen before. We will use predict to guess whether they are malignant or benign, and even read the "probability of being malignant." The estimated time is 5 minutes: 3 minutes to read, 2 minutes to try it out.

Testing the skills you've learned on patients you've never seen before

Last time, we had the decision tree gain experience using training data. In terms of a resident, this is the stage where they have seen hundreds of cases and cultivated their intuition.

Now, let's test that ability. We will pass 171 test patients—people the model has never seen and were not used for training—to the trained model. The model follows the tree it has learned from the top and returns a prediction of "malignant or benign" for each individual.

And there is one more thing that is clinically much more important: "how confident it is," or in other words, the probability. Today, we will learn two ways to read the results: prediction and probability.

Key Points

• Passing .predict(X_test) to a trained model returns a prediction (0=malignant, 1=benign) for each individual.

• .predict_proba(X_test) returns the probability of being malignant and the probability of being benign as the "confidence level" of the prediction.

• By looking at the probability in addition to the binary prediction, you can read "how confident the machine is."

1. Making predictions

Pass the test patients to the trained model (the model from last time).

pred = model.predict(X_test)
pred[:10]

When you run it, it will be displayed like this.

array([0, 1, 1, 1, 1, 1, 1, 1, 1, 1])

These are the predictions for the first 10 test patients. 0 is malignant, 1 is benign. It predicts the first person is malignant and the rest are benign. model.predict(X_test) is a command that says, "Predict the patients in X_test using this model." The "function that returns a result when you pass it data" from the 7th session has now returned a prediction as predict.

Let's line them up with the correct answers.

y_test.values[:10]

When you run it, it will be displayed like this.

array([0, 1, 1, 1, 1, 1, 1, 1, 1, 1])

The predictions and the correct answers match perfectly for these first 10 people. We were able to correctly guess patients that were not used for training.

2. Reading the confidence level

Predictions are a binary choice of "malignant or benign," but in clinical practice, there is one more thing we want to know: how confident is the model? predict_proba returns that.

proba = model.predict_proba(X_test)
proba[:5].round(3)

When you run it, it will be displayed like this.

array([[0.977, 0.023],
       [0.017, 0.983],
       [0.017, 0.983],
       [0.017, 0.983],
       [0.017, 0.983]])

Each row is one patient, and the two numbers are the "probability of being malignant" on the left and the "probability of being benign" on the right (they add up to 1). The first person is 0.977 malignant and 0.023 benign, which can be read as 97.7% confidence that it is malignant. From the second person onwards, they are 98.3% benign.

3. What changes when you read probabilities?

Why look at probabilities instead of just a binary choice?

For example, if a patient is flagged with a "97.7% probability of malignancy," the model is stating with strong conviction that it is malignant. However, if a patient is flagged with a "55% probability of malignancy," even though the prediction is malignant (since it exceeds 0.5), the model is somewhat uncertain. Should we really treat these two people the same by labeling them both as "malignant"?

Probabilities reflect that uncertainty. Predictions with low confidence serve as clues that prompt additional testing or careful judgment. Do not just look at the binary answer and feel at ease; read the probability behind it. This is the first step toward not taking machine predictions at face value.

Today's Exercise (2 minutes)

In a new cell, try the following in order.
(Assuming the trained model from the previous session, #22, is available)

1. Run pred = model.predict(X_test) and view the predictions for the first 10 people using pred[:10].

2. Compare the predictions with the actual results by lining them up with y_test.values[:10].

3. Run model.predict_proba(X_test)[:5].round(3) and read the [malignancy probability, benign probability] for the first 5 people. Confirm that the first person has a 97.7% probability of malignancy.

The binary prediction and the probability behind it. Try comparing both with your own eyes.

Today's Summary

Passing .predict(X_test) to a trained model returns the prediction (0=malignant, 1=benign), while passing .predict_proba(X_test) returns the confidence level (the predicted probability for each of malignant and benign).

By reading the probabilities, you can understand "how confident the machine is" and spot uncertain predictions. There is always a probability behind a binary answer. Reading that as well is the first step toward not taking machines at face value.

Next time, #24 "Peering into the Machine's Judgment"

We will visualize the trained decision tree itself to read what kind of tree the machine has grown. You will encounter the true form of the tree we saw as a conceptual diagram in session #21.