Thanks!

Changing the threshold allows you to control how confident you are when labeling a new data point. For example, you can say that a data point is positive when the corresponding probability exceeds 0.90 rather than 0.50.

Changing threshold is a way to further calibrate your recall/ precision based on business requirement. Remember there is an offset between recall/ precision - so if you improve one, the other will degrade. F-score is a metric that is based on both precision and recall - and in some cases, you pick a threshold that maximizes F-score.

@@uditg i wonder if it is reasonable to change the threshold during the training process? i mean if we have an imbalanced dataset and a simple logistic regression can we calculate the confusion matrix using eg. threshold=0.1 during the training?

@@teamtom Yes, you should certainly determine best cutoff/threshold during training. There is no reason to use the default 0.5 cutoff - it's just there be default, because objectively (from bayesian standpoint) it makes most sense. I'll add one more thing, finding threshold by optimizing F-score is one approach which basically assigns same 'weight' to false positives and false negatives. But in a specific business use case - you might have low cost for (say) false positives, but high cost for false negatives - in that case, instead of calculating and optimizing F-score, you can calculate a cost function and find a cutoff that optimizes that cost function.

Sure, this is a good one: link.springer.com/article/10.1007/s13748-016-0094-0