ForBo7 // Salman Naqvi
  • Home
  • ForBlog
  • Playground
  • Dictionary
  • About

Softmax

A function that calculates the probabilities of a set of predictions.

Let’s say that we have a model that tells us what sort of vehicle is in a picture. It outputs the following predictions.

Vehicle Prediction
car \(-4.89\)
bus \(2.60\)
truck \(0.59\)
motorbike \(-2.07\)
bicycle \(-4.57\)

These predictions aren’t very meaningful to us as humans. So what we can do is convert these predictions into probabilities. The steps to do this are below.

1. Take the exponent of each prediction to base \(e\). So for the car category, \(e^{-4.89} \approx 7.52 \cdot 10^{-3}\).

The results of the calculations below are displayed with 3 significant figures.

Vehicle Prediction \(e^{\text{prediction}}\)
car \(-4.89\) \(7.52 \cdot 10^{-3}\)
bus \(2.60\) \(13.4\)
truck \(0.59\) \(1.80\)
motorbike \(-2.07\) \(0.126\)
bicycle \(-4.57\) \(0.010\)

2. Sum all the calculated values.

Vehicle Prediction \(e^{\text{prediction}}\) \(\text{sum of} e^{\text{prediction}}\)
car \(-4.89\) \(7.52 \cdot 10^{-3}\) \(15.4\)
bus \(2.60\) \(13.4\) \(15.4\)
truck \(0.59\) \(1.80\) \(15.4\)
motorbike \(-2.07\) \(0.126\) \(15.4\)
bicycle \(-4.57\) \(0.010\) \(15.4\)

3. For each respective category, divide \(e^{\text{prediction}}\) by \(\text{sum of} e^{\text{prediction}}\). This is your probability. So the probability of the vehicle in the picture being a car is

\[ \frac{7.52 \cdot 10^{-3}}{15.4} \approx 4.88 \cdot 10^{-4} = 0.000488 = 0.0488 \% \]

Vehicle Prediction \(e^{\text{prediction}}\) \(\text{sum of} e^{\text{prediction}}\) \(\frac{e^{\text{prediction}}}{\text{sum of}e^{\text{prediction}}}\)
car \(-4.89\) \(7.52 \cdot 10^{-3}\) \(15.4\) \(4.88 \cdot 10^{-4}\)
bus \(2.60\) \(13.4\) \(15.4\) \(0.874\)
truck \(0.59\) \(1.80\) \(15.4\) \(0.117\)
motorbike \(-2.07\) \(0.126\) \(15.4\) \(8.19 \cdot 10^{-3}\)
bicycle \(-4.57\) \(0.010\) \(15.4\) \(6.72 \cdot 10^{-4}\)

From the table above, it can be seen that the vehicle in the picture is most likely a bus with probability \(87.4\%\).

ForBo7 // Salman Naqvi © 2023 and ForBlog™ by Salman Naqvi | Site Version 2.1.0.0 | Site Feedback | Website made by me!