🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:
Response:
Calculated Shard: 169 (from laksa042)

2. Crawled Status Check

Query:
Response:

3. Robots.txt Check

Query:
Response:

4. Spam/Ban Check

Query:
Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄
INDEXABLE
CRAWLED
1 day ago
🤖
ROBOTS ALLOWED

Page Info Filters

FilterStatusConditionDetails
HTTP statusPASSdownload_http_code = 200HTTP 200
Age cutoffPASSdownload_stamp > now() - 6 MONTH0 months ago
History dropPASSisNull(history_drop_reason)No drop reason
Spam/banPASSfh_dont_index != 1 AND ml_spam_score = 0ml_spam_score=0
CanonicalPASSmeta_canonical IS NULL OR = '' OR = src_unparsedNot set

Page Details

PropertyValue
URLhttps://catboost.ai/docs/en/concepts/loss-functions-classification
Last Crawled2026-04-15 21:29:18 (1 day ago)
First Indexed2024-11-20 00:02:18 (1 year ago)
HTTP Status Code200
Meta TitleClassification: objectives and metrics | CatBoost
Meta DescriptionObjectives and metrics. Used for optimization. Objectives and metrics Logloss.
Meta Canonicalnull
Boilerpipe Text
Objectives and metrics Used for optimization Objectives and metrics Logloss − ∑ i = 1 N w i ( c i log ⁡ ( p i ) + ( 1 − c i ) log ⁡ ( 1 − p i ) ) ∑ i = 1 N w i \displaystyle\frac{ - \sum\limits_{i=1}^N w_{i}\left(c_i \log(p_{i}) + (1-c_{i}) \log(1 - p_{i})\right)}{\sum\limits_{i = 1}^{N} w_{i}} Usage information See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true CrossEntropy − ∑ i = 1 N w i ( t i log ⁡ ( p i ) + ( 1 − t i ) log ⁡ ( 1 − p i ) ) ∑ i = 1 N w i \displaystyle\frac{- \sum\limits_{i=1}^N w_{i} \left(t_{i} \log(p_{i}) + (1 - t_{i}) \log(1 - p_{i})\right)}{\sum\limits_{i = 1}^{N} w_{i}} Usage information See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true Precision T P T P + F P \frac{TP}{TP + FP} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true Recall T P T P + F N \frac{TP}{TP+FN} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true F ( 1 + β 2 ) ⋅ P r e c i s i o n ∗ R e c a l l ( β 2 ⋅ P r e c i s i o n ) + R e c a l l (1 + \beta^2) \cdot \frac{Precision * Recall}{(\beta^2 \cdot Precision) + Recall} Can't be used for optimization. See more . User-defined parameters beta The β \beta parameter of the F metric. Valid values are real numbers in the following range: ( 0 ; + ∞ ) (0; +\infty) . Default: This parameter is obligatory (the default value is not defined) use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true F1 2 P r e c i s i o n ∗ R e c a l l P r e c i s i o n + R e c a l l 2 \frac{Precision * Recall}{Precision + Recall} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true BalancedAccuracy 1 2 ( T P P + T N N ) \frac{1}{2} \left(\frac{TP}{P} + \frac{TN}{N} \right) User-defined parameters: Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true BalancedErrorRate 1 2 ( F P T N + F P + F N F N + T P ) \frac{1}{2} \left( \displaystyle\frac{FP}{TN + FP} + \displaystyle\frac{FN}{FN + TP} \right) Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true MCC T P ∗ T N − F P ∗ F N ( T P + F P ) ( T P + F N ) ( T N + F P ) ( T N + F N ) \displaystyle\frac{TP * TN - FP * FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true Accuracy T P + T N ∑ i = 1 N w i \frac{TP + TN}{\sum\limits_{i=1}^{N} w_{i}} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true CtrFactor ( ∑ i = 1 N w i t i / N ) ( ∑ i = 1 N w i p i / N ) \displaystyle\frac{\left(\sum\limits_{i = 1}^{N} w_{i} t_{i}/N\right)}{\left(\sum\limits_{i = 1}^{N} w_{i} p_{i} /N\right)} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true AUC The calculation of this metric is disabled by default for the training dataset to speed up the training. Use the hints=skip_train~false parameter to enable the calculation. Classic ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ w i ⋅ w j \displaystyle\frac{\sum I(a_{i}, a_{j}) \cdot w_{i} \cdot w_{j}} {\sum w_{i} \cdot w_{j}} The sum is calculated on all pairs of objects ( i , j ) (i,j) such that: t i = 0 t_{i} = 0 t j = 1 t_{j} = 1 I ( x , y ) = { 0 , x < y 0.5 , x = y 1 , x > y I(x, y) = \begin{cases} 0 { , } & x < y \\ 0.5 { , } & x=y \\ 1 { , } & x>y \end{cases} Refer to the Wikipedia article for details. If the target type is not binary, then every object with target value t t and weight w w is replaced with two objects for the metric calculation: o 1 o_{1} with weight t ⋅ w t \cdot w and target value 1 o 2 o_{2} with weight ( 1 – t ) ⋅ w (1 – t) \cdot w and target value 0. Target values must be in the range [0; 1]. Ranking ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ w i ∗ w j \displaystyle\frac{\sum I(a_{i}, a_{j}) \cdot w_{i} \cdot w_{j}} {\sum w_{i} * w_{j}} The sum is calculated on all pairs of objects ( i , j ) (i,j) such that: t i < t j t_{i} < t_{j} I ( x , y ) = { 0 , x < y 0.5 , x = y 1 , x > y I(x, y) = \begin{cases} 0 { , } & x < y \\ 0.5 { , } & x=y \\ 1 { , } & x>y \end{cases} User-defined parameters type The type of AUC. Defines the metric calculation principles. Default : Ranking . Possible values: Classic , Ranking . Examples: AUC:type=Classic , AUC:type=Ranking . use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default : False . Examples : QueryAUC:type=Ranking;use_weights=False . QueryAUC Classic type ∑ q ∑ i , j ∈ q ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ q ∑ i , j ∈ q ∑ w i ⋅ w j \displaystyle\frac{ \sum_q \sum_{i, j \in q} \sum I(a_{i}, a_{j}) \cdot w_{i} \cdot w_{j}} { \sum_q \sum_{i, j \in q} \sum w_{i} \cdot w_{j}} The sum is calculated on all pairs of objects ( i , j ) (i,j) such that: t i = 0 t_{i} = 0 t j = 1 t_{j} = 1 I ( x , y ) = { 0 , x < y 0.5 , x = y 1 , x > y I(x, y) = \begin{cases} 0 { , } & x < y \\ 0.5 { , } & x=y \\ 1 { , } & x>y \end{cases} Refer to the Wikipedia article for details. If the target type is not binary, then every object with target value t t and weight w w is replaced with two objects for the metric calculation: o 1 o_{1} with weight t ⋅ w t \cdot w and target value 1 o 2 o_{2} with weight ( 1 – t ) ⋅ w (1 – t) \cdot w and target value 0. Target values must be in the range [0; 1]. Ranking type ∑ q ∑ i , j ∈ q ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ q ∑ i , j ∈ q ∑ w i ∗ w j \displaystyle\frac{ \sum_q \sum_{i, j \in q} \sum I(a_{i}, a_{j}) \cdot w_{i} \cdot w_{j}} { \sum_q \sum_{i, j \in q} \sum w_{i} * w_{j}} The sum is calculated on all pairs of objects ( i , j ) (i,j) such that: t i < t j t_{i} < t_{j} I ( x , y ) = { 0 , x < y 0.5 , x = y 1 , x > y I(x, y) = \begin{cases} 0 { , } & x < y \\ 0.5 { , } & x=y \\ 1 { , } & x>y \end{cases} Can't be used for optimization. See more . User-defined parameters type The type of QueryAUC. Defines the metric calculation principles. Default : Ranking . Possible values : Classic , Ranking . Examples : QueryAUC:type=Classic , QueryAUC:type=Ranking . use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default : False . Examples : QueryAUC:type=Ranking;use_weights=False . PRAUC PRAUC is the area under the curve P r e c i s i o n ( q ) Precision(q) vs R e c a l l ( q ) Recall(q) for q ∈ [ 0 , 1 ] q \in [0,1] where P r e c i s i o n ( q ) Precision(q) and R e c a l l ( q ) Recall(q) are defined as follows. P r e c i s i o n ( q ) = T P ( q ) T P ( q ) + F P ( q ) , R e c a l l ( q ) = T P ( q ) T P ( q ) + F N ( q ) Precision(q) = \frac{TP(q)}{TP(q) + FP(q)}, Recall(q) = \frac{TP(q)}{TP(q) + FN(q)} Above T P ( q ) TP(q) , F P ( q ) FP(q) , F N ( q ) FN(q) are weights of the true positive, false positive, and false negative samples, respectively. To calculate PRAUC for a binary classification model, specify type Classic . In this case, T P ( q ) = ∑ w i [ p i > q ] c i TP(q)=\sum w_i [p_i > q] c_i , etc. To calculate PRAUC for a multi-classification model, specify type OneVsAll . In this case, positive samples are samples having class 0, all other samples are negative, and T P ( q ) = ∑ w i [ p i 0 > q ] [ c i = 0 ] TP(q)=\sum w_i [p_{i0} > q] [c_i = 0] , etc. type The type of PRAUC. Defines the metric calculation principles. Type Classic is compatible with binary classification models. Type OneVsAll is compatible with multi-classification models. Default : Classic . Possible values : Classic , OneVsAll . Examples : PRAUC:type=Classic , PRAUC:type=OneVsAll . use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default : False . Examples : PRAUC:type=Classic;use_weights=False . NormalizedGini See AUC. 2 A U C − 1 2 AUC - 1 Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true BrierScore ∑ i = 1 N w i ( p i − t i ) 2 ∑ i = 1 N w i \displaystyle\frac{\sum\limits_{i=1}^{N} w_{i}\left(p_{i} - t_{i} \right)^{2}}{\sum\limits_{i=1}^{N} w_{i}} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true HingeLoss ∑ i = 1 N w i m a x { 1 − t i p i , 0 } ∑ i = 1 N w i , t i = ± 1 \displaystyle\frac{\sum\limits_{i=1}^{N} w_{i} max\{1 - t_{i} p_{i}, 0\}}{\sum\limits_{i=1}^{N} w_{i}} , t_{i} = \pm 1 Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true HammingLoss ∑ i = 1 N w i [ [ p i > 0.5 ] = = t i ] ] ∑ i = 1 N w i \displaystyle\frac{\sum\limits_{i = 1}^{N} w_{i} [[p_{i} > 0.5] == t_{i}]]}{\sum\limits_{i=1}^{N} w_{i}} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true ZeroOneLoss 1 − A c c u r a c y 1 - Accuracy Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true Kappa 1 − 1 − A c c u r a c y 1 − R A c c u r a c y 1 - \displaystyle\frac{1 - Accuracy}{1 - RAccuracy} R A c c u r a c y = ( T N + F P ) ( T N + F N ) + ( F N + T P ) ( F P + T P ) ( ∑ i = 1 N w i ) 2 RAccuracy = \displaystyle\frac{(TN + FP) (TN + FN) + (FN + TP) (FP + TP)}{(\sum\limits_{i=1}^{N} w_{i})^{2}} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true WKappa See the formula on page 3 of the  A note on the linearly weighted kappa coefficient for ordinal scales paper. Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true LogLikelihoodOfPrediction The calculation consists of the following steps: Define the sum of weights ( W W ) and the mean target ( t ˉ \bar{t} ): W = ∑ i w i W = \sum\limits_{i} w_{i} t ˉ = 1 W ∑ i t i w i \bar{t} = \frac{1}{W} \sum\limits_{i} t_{i} w_{i} Denote log-likelihood of a constant prediction: l l 0 = ∑ i w i ( t ˉ ⋅ l o g ( t ˉ ) + ( 1 − t ˉ ) ⋅ l o g ( 1 − t ˉ ) ) ll_0 = \sum\limits_{i} w_{i} (\bar{t} \cdot log(\bar{t}) + (1 - \bar{t}) \cdot log(1 - \bar{t})) Calculate LogLikelihoodOfPrediction ( l l p llp ), which reflects how the likelihood ( l l ll ) differs from the constant prediction: l l p = l l ( t , w ) − l l 0 ∑ i t i w i llp = \displaystyle\frac{ll(t, w) - ll_0}{\sum\limits_{i} t_{i} w_{i}} Can't be used for optimization. See more . User-defined parameters use_weights Use object/group weights to calculate metrics if the specified value is  true and set all weights to 1 regardless of the input data if the specified value is false . Default: true Used for optimization Name Optimization GPU Support Logloss + + CrossEntropy + + Precision - + Recall - + F - - F1 - + BalancedAccuracy - - BalancedErrorRate - - MCC - + Accuracy - + CtrFactor - - AUC - - QueryAUC - - NormalizedGini - - BrierScore - - HingeLoss - - HammingLoss - - ZeroOneLoss - + Kappa - - WKappa - - LogLikelihoodOfPrediction - -
Markdown
[![Logo icon](https://yastatic.net/s3/locdoc/daas-static/catboost/71b237a322eec6f2889af0dae2a9c549.svg)](https://catboost.ai/ "CatBoost") - Installation - [Overview](https://catboost.ai/docs/en/concepts/en/concepts/installation) - Python package installation - CatBoost for Apache Spark installation - R package installation - Command-line version binary - Build from source - Key Features - Training parameters - Python package - CatBoost for Apache Spark - R package - Command-line version - Applying models - Objectives and metrics - [Overview](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions) - [Variables used in formulas](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-variables-used) - [Regression](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-regression) - [Multiregression](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-multiregression) - [Classification](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification) - [Multiclassification](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-multiclassification) - [Multilabel Classification](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-multilabel-classification) - [Ranking](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-ranking) - Model analysis - Data format description - [Parameter tuning](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning) - [Speeding up the training](https://catboost.ai/docs/en/concepts/en/concepts/speed-up-training) - Data visualization - Algorithm details - [FAQ](https://catboost.ai/docs/en/concepts/en/concepts/faq) - Educational materials - [Development and contributions](https://catboost.ai/docs/en/concepts/en/concepts/development-and-contributions) - [Contacts](https://catboost.ai/docs/en/concepts/en/concepts/contacts) Classification: objectives and metrics ## In this article: - [Objectives and metrics](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#objectives-and-metrics) - [Logloss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Logit) - [CrossEntropy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#CrossEntropy) - [Precision](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Precision) - [Recall](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Recall) - [F](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#F) - [F1](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#F1) - [BalancedAccuracy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#BalancedAccuracy) - [BalancedErrorRate](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#BalancedErrorRate) - [MCC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#MCC) - [Accuracy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Accuracy) - [CtrFactor](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#CtrFactor) - [AUC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#AUC) - [QueryAUC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#QueryAUC) - [PRAUC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#PRAUC) - [NormalizedGini](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#NormalizedGini) - [BrierScore](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Brierscore) - [HingeLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#HingeLoss) - [HammingLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#HammingLoss) - [ZeroOneLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#ZeroOneLoss) - [Kappa](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Kappa) - [WKappa](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#WKappa) - [LogLikelihoodOfPrediction](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#LLP) - [Used for optimization](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information) 1. [Objectives and metrics](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions) 2. Classification # Classification: objectives and metrics - [Objectives and metrics](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#objectives-and-metrics) - [Logloss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Logit) - [CrossEntropy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#CrossEntropy) - [Precision](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Precision) - [Recall](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Recall) - [F](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#F) - [F1](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#F1) - [BalancedAccuracy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#BalancedAccuracy) - [BalancedErrorRate](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#BalancedErrorRate) - [MCC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#MCC) - [Accuracy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Accuracy) - [CtrFactor](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#CtrFactor) - [AUC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#AUC) - [QueryAUC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#QueryAUC) - [PRAUC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#PRAUC) - [NormalizedGini](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#NormalizedGini) - [BrierScore](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Brierscore) - [HingeLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#HingeLoss) - [HammingLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#HammingLoss) - [ZeroOneLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#ZeroOneLoss) - [Kappa](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Kappa) - [WKappa](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#WKappa) - [LogLikelihoodOfPrediction](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#LLP) - [Used for optimization](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information) - [Objectives and metrics](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#objectives-and-metrics) - [Used for optimization](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information) ## Objectives and metrics ### Logloss − ∑ i \= 1 N w i ( c i log ⁡ ( p i ) \+ ( 1 − c i ) log ⁡ ( 1 − p i ) ) ∑ i \= 1 N w i \\displaystyle\\frac{ - \\sum\\limits\_{i=1}^N w\_{i}\\left(c\_i \\log(p\_{i}) + (1-c\_{i}) \\log(1 - p\_{i})\\right)}{\\sum\\limits\_{i = 1}^{N} w\_{i}} i\=1∑N​wi​−i\=1∑N​wi​(ci​log(pi​)\+(1−ci​)log(1−pi​))​ **Usage information** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### CrossEntropy − ∑ i \= 1 N w i ( t i log ⁡ ( p i ) \+ ( 1 − t i ) log ⁡ ( 1 − p i ) ) ∑ i \= 1 N w i \\displaystyle\\frac{- \\sum\\limits\_{i=1}^N w\_{i} \\left(t\_{i} \\log(p\_{i}) + (1 - t\_{i}) \\log(1 - p\_{i})\\right)}{\\sum\\limits\_{i = 1}^{N} w\_{i}} i\=1∑N​wi​−i\=1∑N​wi​(ti​log(pi​)\+(1−ti​)log(1−pi​))​ **Usage information** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### Precision T P T P \+ F P \\frac{TP}{TP + FP} TP\+FPTP​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### Recall T P T P \+ F N \\frac{TP}{TP+FN} TP\+FNTP​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### F ( 1 \+ β 2 ) ⋅ P r e c i s i o n ∗ R e c a l l ( β 2 ⋅ P r e c i s i o n ) \+ R e c a l l (1 + \\beta^2) \\cdot \\frac{Precision \* Recall}{(\\beta^2 \\cdot Precision) + Recall} (1\+β2)⋅(β2⋅Precision)\+RecallPrecision∗Recall​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** beta The β \\beta β parameter of the F metric. Valid values are real numbers in the following range: ( 0 ; \+ ∞ ) (0; +\\infty) (0;\+∞). *Default:* This parameter is obligatory (the default value is not defined) use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### F1 2 P r e c i s i o n ∗ R e c a l l P r e c i s i o n \+ R e c a l l 2 \\frac{Precision \* Recall}{Precision + Recall} 2Precision\+RecallPrecision∗Recall​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### BalancedAccuracy 1 2 ( T P P \+ T N N ) \\frac{1}{2} \\left(\\frac{TP}{P} + \\frac{TN}{N} \\right) 21​(PTP​\+NTN​) User-defined parameters: **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### BalancedErrorRate 1 2 ( F P T N \+ F P \+ F N F N \+ T P ) \\frac{1}{2} \\left( \\displaystyle\\frac{FP}{TN + FP} + \\displaystyle\\frac{FN}{FN + TP} \\right) 21​(TN\+FPFP​\+FN\+TPFN​) **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### MCC T P ∗ T N − F P ∗ F N ( T P \+ F P ) ( T P \+ F N ) ( T N \+ F P ) ( T N \+ F N ) \\displaystyle\\frac{TP \* TN - FP \* FN}{\\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}} (TP\+FP)(TP\+FN)(TN\+FP)(TN\+FN) ​ TP∗TN−FP∗FN ​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### Accuracy T P \+ T N ∑ i \= 1 N w i \\frac{TP + TN}{\\sum\\limits\_{i=1}^{N} w\_{i}} i\=1∑N​wi​TP\+TN​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### CtrFactor ( ∑ i \= 1 N w i t i / N ) ( ∑ i \= 1 N w i p i / N ) \\displaystyle\\frac{\\left(\\sum\\limits\_{i = 1}^{N} w\_{i} t\_{i}/N\\right)}{\\left(\\sum\\limits\_{i = 1}^{N} w\_{i} p\_{i} /N\\right)} (i\=1∑N​wi​pi​/N)(i\=1∑N​wi​ti​/N)​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### AUC The calculation of this metric is disabled by default for the training dataset to speed up the training. Use the `hints=skip_train~false` parameter to enable the calculation. #### Classic ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ w i ⋅ w j \\displaystyle\\frac{\\sum I(a\_{i}, a\_{j}) \\cdot w\_{i} \\cdot w\_{j}} {\\sum w\_{i} \\cdot w\_{j}} ∑wi​⋅wj​∑I(ai​,aj​)⋅wi​⋅wj​​ The sum is calculated on all pairs of objects ( i , j ) (i,j) (i,j) such that: - t i \= 0 t\_{i} = 0 ti​\=0 - t j \= 1 t\_{j} = 1 tj​\=1 - I ( x , y ) \= { 0 , x \< y 0\.5 , x \= y 1 , x \> y I(x, y) = \\begin{cases} 0 { , } & x \< y \\\\ 0.5 { , } & x=y \\\\ 1 { , } & x\>y \\end{cases} I(x,y)\= ⎩ ⎨ ⎧ ​ 0,0\.5,1,​x\<yx\=yx\>y​ Refer to the [Wikipedia article](https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve) for details. If the target type is not binary, then every object with target value t t t and weight w w w is replaced with two objects for the metric calculation: - o 1 o\_{1} o1​ with weight t ⋅ w t \\cdot w t⋅w and target value 1 - o 2 o\_{2} o2​ with weight ( 1 – t ) ⋅ w (1 – t) \\cdot w (1–t)⋅w and target value 0. Target values must be in the range \[0; 1\]. #### Ranking ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ w i ∗ w j \\displaystyle\\frac{\\sum I(a\_{i}, a\_{j}) \\cdot w\_{i} \\cdot w\_{j}} {\\sum w\_{i} \* w\_{j}} ∑wi​∗wj​∑I(ai​,aj​)⋅wi​⋅wj​​ The sum is calculated on all pairs of objects ( i , j ) (i,j) (i,j) such that: - t i \< t j t\_{i} \< t\_{j} ti​\<tj​ - I ( x , y ) \= { 0 , x \< y 0\.5 , x \= y 1 , x \> y I(x, y) = \\begin{cases} 0 { , } & x \< y \\\\ 0.5 { , } & x=y \\\\ 1 { , } & x\>y \\end{cases} I(x,y)\= ⎩ ⎨ ⎧ ​ 0,0\.5,1,​x\<yx\=yx\>y​ **User-defined parameters** type The type of AUC. Defines the metric calculation principles. *Default*: `Ranking`. *Possible values:* `Classic`, `Ranking`. *Examples:* `AUC:type=Classic`, `AUC:type=Ranking`. use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default*: `False`. *Examples*: `QueryAUC:type=Ranking;use_weights=False`. ### QueryAUC #### Classic type ∑ q ∑ i , j ∈ q ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ q ∑ i , j ∈ q ∑ w i ⋅ w j \\displaystyle\\frac{ \\sum\_q \\sum\_{i, j \\in q} \\sum I(a\_{i}, a\_{j}) \\cdot w\_{i} \\cdot w\_{j}} { \\sum\_q \\sum\_{i, j \\in q} \\sum w\_{i} \\cdot w\_{j}} ∑q​∑i,j∈q​∑wi​⋅wj​∑q​∑i,j∈q​∑I(ai​,aj​)⋅wi​⋅wj​​ The sum is calculated on all pairs of objects ( i , j ) (i,j) (i,j) such that: - t i \= 0 t\_{i} = 0 ti​\=0 - t j \= 1 t\_{j} = 1 tj​\=1 - I ( x , y ) \= { 0 , x \< y 0\.5 , x \= y 1 , x \> y I(x, y) = \\begin{cases} 0 { , } & x \< y \\\\ 0.5 { , } & x=y \\\\ 1 { , } & x\>y \\end{cases} I(x,y)\= ⎩ ⎨ ⎧ ​ 0,0\.5,1,​x\<yx\=yx\>y​ Refer to the [Wikipedia article](https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve) for details. If the target type is not binary, then every object with target value t t t and weight w w w is replaced with two objects for the metric calculation: - o 1 o\_{1} o1​ with weight t ⋅ w t \\cdot w t⋅w and target value 1 - o 2 o\_{2} o2​ with weight ( 1 – t ) ⋅ w (1 – t) \\cdot w (1–t)⋅w and target value 0. Target values must be in the range \[0; 1\]. #### Ranking type ∑ q ∑ i , j ∈ q ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ q ∑ i , j ∈ q ∑ w i ∗ w j \\displaystyle\\frac{ \\sum\_q \\sum\_{i, j \\in q} \\sum I(a\_{i}, a\_{j}) \\cdot w\_{i} \\cdot w\_{j}} { \\sum\_q \\sum\_{i, j \\in q} \\sum w\_{i} \* w\_{j}} ∑q​∑i,j∈q​∑wi​∗wj​∑q​∑i,j∈q​∑I(ai​,aj​)⋅wi​⋅wj​​ The sum is calculated on all pairs of objects ( i , j ) (i,j) (i,j) such that: - t i \< t j t\_{i} \< t\_{j} ti​\<tj​ - I ( x , y ) \= { 0 , x \< y 0\.5 , x \= y 1 , x \> y I(x, y) = \\begin{cases} 0 { , } & x \< y \\\\ 0.5 { , } & x=y \\\\ 1 { , } & x\>y \\end{cases} I(x,y)\= ⎩ ⎨ ⎧ ​ 0,0\.5,1,​x\<yx\=yx\>y​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#optimization). **User-defined parameters** type The type of QueryAUC. Defines the metric calculation principles. *Default*: `Ranking`. *Possible values*: `Classic`, `Ranking`. *Examples*: `QueryAUC:type=Classic`, `QueryAUC:type=Ranking`. use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default*: `False`. *Examples*: `QueryAUC:type=Ranking;use_weights=False`. ### PRAUC PRAUC is the area under the curve P r e c i s i o n ( q ) Precision(q) Precision(q) vs R e c a l l ( q ) Recall(q) Recall(q) for q ∈ \[ 0 , 1 \] q \\in \[0,1\] q∈\[0,1\] where P r e c i s i o n ( q ) Precision(q) Precision(q) and R e c a l l ( q ) Recall(q) Recall(q) are defined as follows. P r e c i s i o n ( q ) \= T P ( q ) T P ( q ) \+ F P ( q ) , R e c a l l ( q ) \= T P ( q ) T P ( q ) \+ F N ( q ) Precision(q) = \\frac{TP(q)}{TP(q) + FP(q)}, Recall(q) = \\frac{TP(q)}{TP(q) + FN(q)} Precision(q)\=TP(q)\+FP(q)TP(q)​,Recall(q)\=TP(q)\+FN(q)TP(q)​ Above T P ( q ) TP(q) TP(q), F P ( q ) FP(q) FP(q), F N ( q ) FN(q) FN(q) are weights of the true positive, false positive, and false negative samples, respectively. To calculate PRAUC for a binary classification model, specify type `Classic`. In this case, T P ( q ) \= ∑ w i \[ p i \> q \] c i TP(q)=\\sum w\_i \[p\_i \> q\] c\_i TP(q)\=∑wi​\[pi​\>q\]ci​, etc. To calculate PRAUC for a multi-classification model, specify type `OneVsAll`. In this case, positive samples are samples having class 0, all other samples are negative, and T P ( q ) \= ∑ w i \[ p i 0 \> q \] \[ c i \= 0 \] TP(q)=\\sum w\_i \[p\_{i0} \> q\] \[c\_i = 0\] TP(q)\=∑wi​\[pi0​\>q\]\[ci​\=0\], etc. type The type of PRAUC. Defines the metric calculation principles. Type `Classic` is compatible with binary classification models. Type `OneVsAll` is compatible with multi-classification models. *Default*: `Classic`. *Possible values*: `Classic`, `OneVsAll`. *Examples*: `PRAUC:type=Classic`, `PRAUC:type=OneVsAll`. use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default*: `False`. *Examples*: `PRAUC:type=Classic;use_weights=False`. ### NormalizedGini See AUC. 2 A U C − 1 2 AUC - 1 2AUC−1 **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### BrierScore ∑ i \= 1 N w i ( p i − t i ) 2 ∑ i \= 1 N w i \\displaystyle\\frac{\\sum\\limits\_{i=1}^{N} w\_{i}\\left(p\_{i} - t\_{i} \\right)^{2}}{\\sum\\limits\_{i=1}^{N} w\_{i}} i\=1∑N​wi​i\=1∑N​wi​(pi​−ti​)2​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### HingeLoss ∑ i \= 1 N w i m a x { 1 − t i p i , 0 } ∑ i \= 1 N w i , t i \= ± 1 \\displaystyle\\frac{\\sum\\limits\_{i=1}^{N} w\_{i} max\\{1 - t\_{i} p\_{i}, 0\\}}{\\sum\\limits\_{i=1}^{N} w\_{i}} , t\_{i} = \\pm 1 i\=1∑N​wi​i\=1∑N​wi​max{1−ti​pi​,0}​,ti​\=±1 **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### HammingLoss ∑ i \= 1 N w i \[ \[ p i \> 0\.5 \] \= \= t i \] \] ∑ i \= 1 N w i \\displaystyle\\frac{\\sum\\limits\_{i = 1}^{N} w\_{i} \[\[p\_{i} \> 0.5\] == t\_{i}\]\]}{\\sum\\limits\_{i=1}^{N} w\_{i}} i\=1∑N​wi​i\=1∑N​wi​\[\[pi​\>0\.5\]\==ti​\]\]​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### ZeroOneLoss 1 − A c c u r a c y 1 - Accuracy 1−Accuracy **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### Kappa 1 − 1 − A c c u r a c y 1 − R A c c u r a c y 1 - \\displaystyle\\frac{1 - Accuracy}{1 - RAccuracy} 1−1−RAccuracy1−Accuracy​ R A c c u r a c y \= ( T N \+ F P ) ( T N \+ F N ) \+ ( F N \+ T P ) ( F P \+ T P ) ( ∑ i \= 1 N w i ) 2 RAccuracy = \\displaystyle\\frac{(TN + FP) (TN + FN) + (FN + TP) (FP + TP)}{(\\sum\\limits\_{i=1}^{N} w\_{i})^{2}} RAccuracy\=(i\=1∑N​wi​)2(TN\+FP)(TN\+FN)\+(FN\+TP)(FP\+TP)​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### WKappa See the formula on page 3 of the [A note on the linearly weighted kappa coefficient for ordinal scales](https://orbi.uliege.be/bitstream/2268/2262/1/STATMED-174.pdf) paper. **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### LogLikelihoodOfPrediction The calculation consists of the following steps: 1. Define the sum of weights (W W W) and the mean target (t ˉ \\bar{t} tˉ): W \= ∑ i w i W = \\sum\\limits\_{i} w\_{i} W\=i∑​wi​ t ˉ \= 1 W ∑ i t i w i \\bar{t} = \\frac{1}{W} \\sum\\limits\_{i} t\_{i} w\_{i} tˉ\=W1​i∑​ti​wi​ 2. Denote log-likelihood of a constant prediction: l l 0 \= ∑ i w i ( t ˉ ⋅ l o g ( t ˉ ) \+ ( 1 − t ˉ ) ⋅ l o g ( 1 − t ˉ ) ) ll\_0 = \\sum\\limits\_{i} w\_{i} (\\bar{t} \\cdot log(\\bar{t}) + (1 - \\bar{t}) \\cdot log(1 - \\bar{t})) ll0​\=i∑​wi​(tˉ⋅log(tˉ)\+(1−tˉ)⋅log(1−tˉ)) 3. Calculate LogLikelihoodOfPrediction (l l p llp llp), which reflects how the likelihood (l l ll ll) differs from the constant prediction: l l p \= l l ( t , w ) − l l 0 ∑ i t i w i llp = \\displaystyle\\frac{ll(t, w) - ll\_0}{\\sum\\limits\_{i} t\_{i} w\_{i}} llp\=i∑​ti​wi​ll(t,w)−ll0​​ **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ## Used for optimization | Name | Optimization | GPU Support | |---|---|---| | [Logloss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Logit) | \+ | \+ | | [CrossEntropy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#CrossEntropy) | \+ | \+ | | [Precision](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Precision) | \- | \+ | | [Recall](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Recall) | \- | \+ | | [F](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#F) | \- | \- | | [F1](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#F1) | \- | \+ | | [BalancedAccuracy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#BalancedAccuracy) | \- | \- | | [BalancedErrorRate](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#BalancedErrorRate) | \- | \- | | [MCC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#MCC) | \- | \+ | | [Accuracy](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Accuracy) | \- | \+ | | [CtrFactor](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#CtrFactor) | \- | \- | | [AUC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#AUC) | \- | \- | | [QueryAUC](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#QueryAUC) | \- | \- | | [NormalizedGini](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#ormalizedGini) | \- | \- | | [BrierScore](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Brierscore) | \- | \- | | [HingeLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#HingeLoss) | \- | \- | | [HammingLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#HammingLoss) | \- | \- | | [ZeroOneLoss](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#ZeroOneLoss) | \- | \+ | | [Kappa](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#Kappa) | \- | \- | | [WKappa](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#WKappa) | \- | \- | | [LogLikelihoodOfPrediction](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-classification#LLP) | \- | \- | ### Was the article helpful? Yes No Previous [Multiregression](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-multiregression) Next [Multiclassification](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-multiclassification) ![](https://mc.yandex.ru/watch/60763294)
Readable Markdown
- [Objectives and metrics](https://catboost.ai/docs/en/concepts/loss-functions-classification#objectives-and-metrics) - [Used for optimization](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information) ## Objectives and metrics ### Logloss − ∑ i \= 1 N w i ( c i log ⁡ ( p i ) \+ ( 1 − c i ) log ⁡ ( 1 − p i ) ) ∑ i \= 1 N w i \\displaystyle\\frac{ - \\sum\\limits\_{i=1}^N w\_{i}\\left(c\_i \\log(p\_{i}) + (1-c\_{i}) \\log(1 - p\_{i})\\right)}{\\sum\\limits\_{i = 1}^{N} w\_{i}} **Usage information** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### CrossEntropy − ∑ i \= 1 N w i ( t i log ⁡ ( p i ) \+ ( 1 − t i ) log ⁡ ( 1 − p i ) ) ∑ i \= 1 N w i \\displaystyle\\frac{- \\sum\\limits\_{i=1}^N w\_{i} \\left(t\_{i} \\log(p\_{i}) + (1 - t\_{i}) \\log(1 - p\_{i})\\right)}{\\sum\\limits\_{i = 1}^{N} w\_{i}} **Usage information** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### Precision T P T P \+ F P \\frac{TP}{TP + FP} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### Recall T P T P \+ F N \\frac{TP}{TP+FN} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### F ( 1 \+ β 2 ) ⋅ P r e c i s i o n ∗ R e c a l l ( β 2 ⋅ P r e c i s i o n ) \+ R e c a l l (1 + \\beta^2) \\cdot \\frac{Precision \* Recall}{(\\beta^2 \\cdot Precision) + Recall} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** beta The β \\beta parameter of the F metric. Valid values are real numbers in the following range: ( 0 ; \+ ∞ ) (0; +\\infty). *Default:* This parameter is obligatory (the default value is not defined) use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### F1 2 P r e c i s i o n ∗ R e c a l l P r e c i s i o n \+ R e c a l l 2 \\frac{Precision \* Recall}{Precision + Recall} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### BalancedAccuracy 1 2 ( T P P \+ T N N ) \\frac{1}{2} \\left(\\frac{TP}{P} + \\frac{TN}{N} \\right) User-defined parameters: **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### BalancedErrorRate 1 2 ( F P T N \+ F P \+ F N F N \+ T P ) \\frac{1}{2} \\left( \\displaystyle\\frac{FP}{TN + FP} + \\displaystyle\\frac{FN}{FN + TP} \\right) **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### MCC T P ∗ T N − F P ∗ F N ( T P \+ F P ) ( T P \+ F N ) ( T N \+ F P ) ( T N \+ F N ) \\displaystyle\\frac{TP \* TN - FP \* FN}{\\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### Accuracy T P \+ T N ∑ i \= 1 N w i \\frac{TP + TN}{\\sum\\limits\_{i=1}^{N} w\_{i}} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### CtrFactor ( ∑ i \= 1 N w i t i / N ) ( ∑ i \= 1 N w i p i / N ) \\displaystyle\\frac{\\left(\\sum\\limits\_{i = 1}^{N} w\_{i} t\_{i}/N\\right)}{\\left(\\sum\\limits\_{i = 1}^{N} w\_{i} p\_{i} /N\\right)} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### AUC The calculation of this metric is disabled by default for the training dataset to speed up the training. Use the `hints=skip_train~false` parameter to enable the calculation. #### Classic ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ w i ⋅ w j \\displaystyle\\frac{\\sum I(a\_{i}, a\_{j}) \\cdot w\_{i} \\cdot w\_{j}} {\\sum w\_{i} \\cdot w\_{j}} The sum is calculated on all pairs of objects ( i , j ) (i,j) such that: - t i \= 0 t\_{i} = 0 - t j \= 1 t\_{j} = 1 - I ( x , y ) \= { 0 , x \< y 0\.5 , x \= y 1 , x \> y I(x, y) = \\begin{cases} 0 { , } & x \< y \\\\ 0.5 { , } & x=y \\\\ 1 { , } & x\>y \\end{cases} Refer to the [Wikipedia article](https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve) for details. If the target type is not binary, then every object with target value t t and weight w w is replaced with two objects for the metric calculation: - o 1 o\_{1} with weight t ⋅ w t \\cdot w and target value 1 - o 2 o\_{2} with weight ( 1 – t ) ⋅ w (1 – t) \\cdot w and target value 0. Target values must be in the range \[0; 1\]. #### Ranking ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ w i ∗ w j \\displaystyle\\frac{\\sum I(a\_{i}, a\_{j}) \\cdot w\_{i} \\cdot w\_{j}} {\\sum w\_{i} \* w\_{j}} The sum is calculated on all pairs of objects ( i , j ) (i,j) such that: - t i \< t j t\_{i} \< t\_{j} - I ( x , y ) \= { 0 , x \< y 0\.5 , x \= y 1 , x \> y I(x, y) = \\begin{cases} 0 { , } & x \< y \\\\ 0.5 { , } & x=y \\\\ 1 { , } & x\>y \\end{cases} **User-defined parameters** type The type of AUC. Defines the metric calculation principles. *Default*: `Ranking`. *Possible values:* `Classic`, `Ranking`. *Examples:* `AUC:type=Classic`, `AUC:type=Ranking`. use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default*: `False`. *Examples*: `QueryAUC:type=Ranking;use_weights=False`. ### QueryAUC #### Classic type ∑ q ∑ i , j ∈ q ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ q ∑ i , j ∈ q ∑ w i ⋅ w j \\displaystyle\\frac{ \\sum\_q \\sum\_{i, j \\in q} \\sum I(a\_{i}, a\_{j}) \\cdot w\_{i} \\cdot w\_{j}} { \\sum\_q \\sum\_{i, j \\in q} \\sum w\_{i} \\cdot w\_{j}} The sum is calculated on all pairs of objects ( i , j ) (i,j) such that: - t i \= 0 t\_{i} = 0 - t j \= 1 t\_{j} = 1 - I ( x , y ) \= { 0 , x \< y 0\.5 , x \= y 1 , x \> y I(x, y) = \\begin{cases} 0 { , } & x \< y \\\\ 0.5 { , } & x=y \\\\ 1 { , } & x\>y \\end{cases} Refer to the [Wikipedia article](https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve) for details. If the target type is not binary, then every object with target value t t and weight w w is replaced with two objects for the metric calculation: - o 1 o\_{1} with weight t ⋅ w t \\cdot w and target value 1 - o 2 o\_{2} with weight ( 1 – t ) ⋅ w (1 – t) \\cdot w and target value 0. Target values must be in the range \[0; 1\]. #### Ranking type ∑ q ∑ i , j ∈ q ∑ I ( a i , a j ) ⋅ w i ⋅ w j ∑ q ∑ i , j ∈ q ∑ w i ∗ w j \\displaystyle\\frac{ \\sum\_q \\sum\_{i, j \\in q} \\sum I(a\_{i}, a\_{j}) \\cdot w\_{i} \\cdot w\_{j}} { \\sum\_q \\sum\_{i, j \\in q} \\sum w\_{i} \* w\_{j}} The sum is calculated on all pairs of objects ( i , j ) (i,j) such that: - t i \< t j t\_{i} \< t\_{j} - I ( x , y ) \= { 0 , x \< y 0\.5 , x \= y 1 , x \> y I(x, y) = \\begin{cases} 0 { , } & x \< y \\\\ 0.5 { , } & x=y \\\\ 1 { , } & x\>y \\end{cases} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#optimization). **User-defined parameters** type The type of QueryAUC. Defines the metric calculation principles. *Default*: `Ranking`. *Possible values*: `Classic`, `Ranking`. *Examples*: `QueryAUC:type=Classic`, `QueryAUC:type=Ranking`. use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default*: `False`. *Examples*: `QueryAUC:type=Ranking;use_weights=False`. ### PRAUC PRAUC is the area under the curve P r e c i s i o n ( q ) Precision(q) vs R e c a l l ( q ) Recall(q) for q ∈ \[ 0 , 1 \] q \\in \[0,1\] where P r e c i s i o n ( q ) Precision(q) and R e c a l l ( q ) Recall(q) are defined as follows. P r e c i s i o n ( q ) \= T P ( q ) T P ( q ) \+ F P ( q ) , R e c a l l ( q ) \= T P ( q ) T P ( q ) \+ F N ( q ) Precision(q) = \\frac{TP(q)}{TP(q) + FP(q)}, Recall(q) = \\frac{TP(q)}{TP(q) + FN(q)} Above T P ( q ) TP(q), F P ( q ) FP(q), F N ( q ) FN(q) are weights of the true positive, false positive, and false negative samples, respectively. To calculate PRAUC for a binary classification model, specify type `Classic`. In this case, T P ( q ) \= ∑ w i \[ p i \> q \] c i TP(q)=\\sum w\_i \[p\_i \> q\] c\_i, etc. To calculate PRAUC for a multi-classification model, specify type `OneVsAll`. In this case, positive samples are samples having class 0, all other samples are negative, and T P ( q ) \= ∑ w i \[ p i 0 \> q \] \[ c i \= 0 \] TP(q)=\\sum w\_i \[p\_{i0} \> q\] \[c\_i = 0\], etc. type The type of PRAUC. Defines the metric calculation principles. Type `Classic` is compatible with binary classification models. Type `OneVsAll` is compatible with multi-classification models. *Default*: `Classic`. *Possible values*: `Classic`, `OneVsAll`. *Examples*: `PRAUC:type=Classic`, `PRAUC:type=OneVsAll`. use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default*: `False`. *Examples*: `PRAUC:type=Classic;use_weights=False`. ### NormalizedGini See AUC. 2 A U C − 1 2 AUC - 1 **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### BrierScore ∑ i \= 1 N w i ( p i − t i ) 2 ∑ i \= 1 N w i \\displaystyle\\frac{\\sum\\limits\_{i=1}^{N} w\_{i}\\left(p\_{i} - t\_{i} \\right)^{2}}{\\sum\\limits\_{i=1}^{N} w\_{i}} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### HingeLoss ∑ i \= 1 N w i m a x { 1 − t i p i , 0 } ∑ i \= 1 N w i , t i \= ± 1 \\displaystyle\\frac{\\sum\\limits\_{i=1}^{N} w\_{i} max\\{1 - t\_{i} p\_{i}, 0\\}}{\\sum\\limits\_{i=1}^{N} w\_{i}} , t\_{i} = \\pm 1 **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### HammingLoss ∑ i \= 1 N w i \[ \[ p i \> 0\.5 \] \= \= t i \] \] ∑ i \= 1 N w i \\displaystyle\\frac{\\sum\\limits\_{i = 1}^{N} w\_{i} \[\[p\_{i} \> 0.5\] == t\_{i}\]\]}{\\sum\\limits\_{i=1}^{N} w\_{i}} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### ZeroOneLoss 1 − A c c u r a c y 1 - Accuracy **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### Kappa 1 − 1 − A c c u r a c y 1 − R A c c u r a c y 1 - \\displaystyle\\frac{1 - Accuracy}{1 - RAccuracy} R A c c u r a c y \= ( T N \+ F P ) ( T N \+ F N ) \+ ( F N \+ T P ) ( F P \+ T P ) ( ∑ i \= 1 N w i ) 2 RAccuracy = \\displaystyle\\frac{(TN + FP) (TN + FN) + (FN + TP) (FP + TP)}{(\\sum\\limits\_{i=1}^{N} w\_{i})^{2}} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### WKappa See the formula on page 3 of the [A note on the linearly weighted kappa coefficient for ordinal scales](https://orbi.uliege.be/bitstream/2268/2262/1/STATMED-174.pdf) paper. **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ### LogLikelihoodOfPrediction The calculation consists of the following steps: 1. Define the sum of weights (W W) and the mean target (t ˉ \\bar{t}): W \= ∑ i w i W = \\sum\\limits\_{i} w\_{i} t ˉ \= 1 W ∑ i t i w i \\bar{t} = \\frac{1}{W} \\sum\\limits\_{i} t\_{i} w\_{i} 2. Denote log-likelihood of a constant prediction: l l 0 \= ∑ i w i ( t ˉ ⋅ l o g ( t ˉ ) \+ ( 1 − t ˉ ) ⋅ l o g ( 1 − t ˉ ) ) ll\_0 = \\sum\\limits\_{i} w\_{i} (\\bar{t} \\cdot log(\\bar{t}) + (1 - \\bar{t}) \\cdot log(1 - \\bar{t})) 3. Calculate LogLikelihoodOfPrediction (l l p llp), which reflects how the likelihood (l l ll) differs from the constant prediction: l l p \= l l ( t , w ) − l l 0 ∑ i t i w i llp = \\displaystyle\\frac{ll(t, w) - ll\_0}{\\sum\\limits\_{i} t\_{i} w\_{i}} **Can't be used for optimization.** See [more](https://catboost.ai/docs/en/concepts/loss-functions-classification#usage-information). **User-defined parameters** use\_weights Use object/group weights to calculate metrics if the specified value is "true" and set all weights to "1" regardless of the input data if the specified value is "false". *Default:* true ## Used for optimization | Name | Optimization | GPU Support | |---|---|---| | [Logloss](https://catboost.ai/docs/en/concepts/loss-functions-classification#Logit) | \+ | \+ | | [CrossEntropy](https://catboost.ai/docs/en/concepts/loss-functions-classification#CrossEntropy) | \+ | \+ | | [Precision](https://catboost.ai/docs/en/concepts/loss-functions-classification#Precision) | \- | \+ | | [Recall](https://catboost.ai/docs/en/concepts/loss-functions-classification#Recall) | \- | \+ | | [F](https://catboost.ai/docs/en/concepts/loss-functions-classification#F) | \- | \- | | [F1](https://catboost.ai/docs/en/concepts/loss-functions-classification#F1) | \- | \+ | | [BalancedAccuracy](https://catboost.ai/docs/en/concepts/loss-functions-classification#BalancedAccuracy) | \- | \- | | [BalancedErrorRate](https://catboost.ai/docs/en/concepts/loss-functions-classification#BalancedErrorRate) | \- | \- | | [MCC](https://catboost.ai/docs/en/concepts/loss-functions-classification#MCC) | \- | \+ | | [Accuracy](https://catboost.ai/docs/en/concepts/loss-functions-classification#Accuracy) | \- | \+ | | [CtrFactor](https://catboost.ai/docs/en/concepts/loss-functions-classification#CtrFactor) | \- | \- | | [AUC](https://catboost.ai/docs/en/concepts/loss-functions-classification#AUC) | \- | \- | | [QueryAUC](https://catboost.ai/docs/en/concepts/loss-functions-classification#QueryAUC) | \- | \- | | [NormalizedGini](https://catboost.ai/docs/en/concepts/loss-functions-classification#ormalizedGini) | \- | \- | | [BrierScore](https://catboost.ai/docs/en/concepts/loss-functions-classification#Brierscore) | \- | \- | | [HingeLoss](https://catboost.ai/docs/en/concepts/loss-functions-classification#HingeLoss) | \- | \- | | [HammingLoss](https://catboost.ai/docs/en/concepts/loss-functions-classification#HammingLoss) | \- | \- | | [ZeroOneLoss](https://catboost.ai/docs/en/concepts/loss-functions-classification#ZeroOneLoss) | \- | \+ | | [Kappa](https://catboost.ai/docs/en/concepts/loss-functions-classification#Kappa) | \- | \- | | [WKappa](https://catboost.ai/docs/en/concepts/loss-functions-classification#WKappa) | \- | \- | | [LogLikelihoodOfPrediction](https://catboost.ai/docs/en/concepts/loss-functions-classification#LLP) | \- | \- |
Shard169 (laksa)
Root Hash17435841955170310369
Unparsed URLai,catboost!/docs/en/concepts/loss-functions-classification s443