🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 169 (from laksa068)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa169.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical, ml_categories_json AS ml_categories_json, ml_types_json AS ml_types_json, ml_intent_types_json AS ml_intent_types_json, meta_language AS meta_language, attrs_author AS attrs_author, ifNull(toUnixTimestamp(attrs_publish_time), 0) AS attrs_publish_time, ifNull(toUnixTimestamp(attrs_original_publish_time), 0) AS attrs_original_publish_time, ifNull(attrs_is_republished, 0) AS attrs_is_republished, ifNull(attrs_nr_words, 0) AS attrs_nr_words, ifNull(attrs_boilerpipe_nr_words, 0) AS attrs_boilerpipe_nr_words, ifNull(body_ext_links_number, 0) AS body_ext_links_number, ifNull(body_int_links_number, 0) AS body_int_links_number, ifNull(meta_nofollow, 0) AS meta_nofollow, ifNull(meta_noarchive, 0) AS meta_noarchive, ifNull(props_was_rendered, 0) AS props_was_rendered, ifNull(src_redirect, \'\') AS src_redirect, ifNull(download_time_msec, 0) AS download_time_msec, ifNull(download_ttfb_msec, 0) AS download_ttfb_msec, ifNull(download_size, 0) AS download_size FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://catboost.ai/docs/en/concepts/parameter-tuning\')), getAhrefsUnparsedNoserviceFromURL(\'https://catboost.ai/docs/en/concepts/parameter-tuning\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/catboost.ai\/docs\/en\/concepts\/parameter-tuning","crawl_time":1776843817,"first_indexed_time":1731946335,"http_code":200,"src_unparsed":"ai,catboost!\/docs\/en\/concepts\/parameter-tuning s443","src_root_hash":"17435841955170310369","history_drop_reason":null,"meta_title":"Parameter tuning | CatBoost","meta_descriptions":["CatBoost provides a flexible interface for parameter tuning and can be configured to suit different tasks."],"attrs_boilerpipe_text":"CatBoost provides a flexible interface for parameter tuning and can be configured to suit different tasks.\nThis section contains some tips on the possible parameter settings.\nOne-hot encoding\nWarning\nDo not use one-hot encoding during preprocessing. This affects both the training speed and the resulting quality.\nSometimes when categorical features don't have a lot of values, one-hot encoding works well.\nUsually one-hot encoding does not significantly improve the quality of the model. But if it is required, use the inbuilt parameters instead of preprocessing the dataset.\nParameters\nCommand-line version parameters:\n--one-hot-max-size\nPython parameters:\none_hot_max_size\nR parameters:\none_hot_max_size\nDescription\nUse one-hot encoding for all categorical features with a number of different values less than or equal to the given parameter value. Ctrs are not calculated for such features.\nDefault value\nThe default value depends on various conditions:\nN\/A if training is performed on CPU in Pairwise scoring mode\nRead more about  Pairwise scoring\nThe following loss functions use Pairwise scoring:\nYetiRankPairwise\nPairLogitPairwise\nQueryCrossEntropy\nPairwise scoring is slightly different from regular training on pairs, since pairs are generated only internally during the training for the corresponding metrics. One-hot encoding is not available for these loss functions.\n255 if training is performed on GPU and the selected Ctr types require target data that is not available during the training\n10 if training is performed in \nRanking\nmode\n2 if none of the conditions above is met\nNumber of trees\nIt is recommended to check that there is no obvious underfitting or overfitting before tuning any other parameters. In order to do this it is necessary to analyze the metric value on the validation dataset and select the appropriate number of iterations.\nThis can be done by setting the number of \niterations\nto a large value, using the \noverfitting detector\nparameters and turning the\nuse best model\noptions on. In this case the resulting model contains only the first\nk\nbest iterations, where\nk\nis the iteration with the best loss value on the validation dataset.\nAlso, the metric for choosing the best model may differ from the one used for optimizing the objective value. For example, it is possible to set the optimized function to Logloss and use the AUC function for the overfitting detector. To do so, use the\nevaluation metric\nparameter.\nParameters\nCommand-line version parameters:\n-i\n,\n--iterations\nPython parameters:\n--iterations\nR parameters:\n--iterations\nDescription\nThe maximum number of trees that can be built when solving machine learning problems.\nWhen using other parameters that limit the number of iterations, the final number of trees may be less than the number specified in this parameter.\nCommand-line version parameters:\n--use-best-model\nPython parameters:\n--use-best-model\nR parameters:\n--use-best-model\nDescription\nIf this parameter is set, the number of trees that are saved in the resulting model is defined as follows:\nBuild the number of trees defined by the training parameters.\nUse the validation dataset to identify the iteration with the optimal value of the metric specified in  \n--eval-metric\n(\n--eval-metric\n).\nNo trees are saved after this iteration.\nThis option requires a validation dataset to be provided.\nCommand-line version parameters:\n--eval-metric\nPython parameters:\n--eval-metric\nR parameters:\n--eval-metric\nDescription\nThe metric used for overfitting detection (if enabled) and best model selection (if enabled). Some metrics support optional parameters (see the \nObjectives and metrics\nsection for details on each metric).\nFormat:\n<Metric>[:<parameter 1>=<value>;..;<parameter N>=<value>]\nSupported metrics\nExamples:\nR2\nQuantile:alpha=0.3\nCommand-line version parameters:\nOverfitting detection settings\nCommand-line version parameters:\n--od-type\nPython parameters:\nod_type\nR parameters:\nod_type\nDescription\nThe type of the overfitting detector to use.\nPossible values:\nIncToDec\nIter\nCommand-line version parameters:\n--od-pval\nPython parameters:\nod_pval\nR parameters:\nod_pval\nDescription\nThe threshold for the IncToDec\noverfitting detector\ntype. The training is stopped when the specified value is reached. Requires that a validation dataset was input.\nFor best results, it is recommended to set a value in the range\n[\n1\n0\n–\n10\n;\n1\n0\n−\n2\n]\n[10^{–10}; 10^{-2}]\n.\nThe larger the value, the earlier overfitting is detected.\nAlert\nDo not use this parameter with the Iter overfitting detector type.\nCommand-line version parameters:\n--od-wait\nPython parameters:\nod_wait\nR parameters:\nod_wait\nDescription\nThe number of iterations to continue the training after the iteration with the optimal metric value.\nThe purpose of this parameter differs depending on the selected overfitting detector type:\nIncToDec — Ignore the overfitting detector when the threshold is reached and continue learning for the specified number of iterations after the iteration with the optimal metric value.\nIter — Consider the model overfitted and stop training after the specified number of iterations since the iteration with the optimal metric value.\nLearning rate\nThis setting is used for reducing the gradient step. It affects the overall time of training: the smaller the value, the more iterations are required for training. Choose the value based on the performance expectations.\nBy default, the learning rate is defined automatically based on the dataset properties and the number of iterations. The automatically defined value should be close to the optimal one.\nPossible ways of adjusting the learning rate depending on the overfitting results:\nThere is no overfitting on the last iterations of training (the training does not converge) — increase the learning rate.\nOverfitting is detected — decrease the learning rate.\nParameters\nCommand-line version parameters:\n-w\n,\n--learning-rate\nPython parameters:\nlearning_rate\nR parameters:\nlearning_rate\nDescription\nThe learning rate. Used for reducing the gradient step.\nTree depth\nIn most cases, the optimal depth ranges from 4 to 10. Values in the range from 6 to 10 are recommended.\nNote\nThe maximum depth of the trees is limited to 8 for pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) when the training is performed on GPU.\nParameters\nCommand-line version parameters:\n-n\n,\n--depth\nPython parameters:\ndepth\nR parameters:\ndepth\nDescription\nDepth of the trees. The range of supported values depends on the processing unit type and the type of the selected loss function:\nCPU — Any integer up to  16.\nGPU — Any integer up to 8 pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) and up to   16 for all other loss functions.\nL2 regularization\nTry different values for the regularizer to find the best possible.\nParameters\nCommand-line version parameters:\n--l2-leaf-reg\nPython parameters:\nl2_leaf_reg\nR parameters:\nl2_leaf_reg\nDescription\nCoefficient at the L2 regularization term of the cost function.\nAny positive value is allowed.\nRandom strength\nTry setting different values for the \nrandom_strength\nparameter.\nParameters\nCommand-line version parameters:\n--random-strength\nPython parameters:\nrandom_strength\nR parameters:\nrandom_strength\nDescription\nThe amount of randomness to use for scoring splits when the tree structure is selected. Use this parameter to avoid overfitting the model.\nThe value of this parameter is used when selecting splits. On every iteration each possible split gets a score (for example, the score indicates how much adding this split will improve the loss function for the training dataset). The split with the highest score is selected.\nThe scores have no randomness. A normally distributed random variable is added to the score of the feature. It has a zero mean and a variance that decreases during the training. The value of this parameter is the multiplier of the variance.\nThis parameter is not supported for the following loss functions:\nQueryCrossEntropy\nYetiRankPairwise\nPairLogitPairwise\nBagging temperature\nTry setting different values for the\nbagging_temperature\nparameter\nParameters\nCommand-line version parameters:\n--bagging-temperature\nPython parameters:\nbagging_temperature\nR parameters:\nbagging_temperature\nDescription\nDefines the settings of the Bayesian bootstrap. It is used by default in classification and regression modes.\nUse the Bayesian bootstrap to assign random weights to objects.\nThe weights are sampled from exponential distribution if the value of this parameter is set to\n1\n. All weights are equal to 1 if the value of this parameter is set to\n0\n.\nPossible values are in the range\n[\n0\n;\ninf\n⁡\n)\n[0; \\inf)\n. The higher the value the more aggressive the bagging is.\nThis parameter can be used if the selected bootstrap type is Bayesian.\nBorder count\nThe number of splits for numerical features.\nThe default value depends on the processing unit type and other parameters:\nCPU: 254\nGPU in PairLogitPairwise and YetiRankPairwise modes: 32\nGPU in all other modes: 128\nThe value of this parameter significantly impacts the speed of training on GPU. The smaller the value, the faster the training is performed (refer to the \nNumber of splits for numerical features\nsection for details).\n128 splits are enough for many datasets. However, try to set the value of this parameter to 254 when training on GPU if the best possible quality is required.\nThe value of this parameter does not significantly impact the speed of training on CPU. Try to set it to 254 for the best possible quality.\nParameters\nCommand-line version parameters:\n-x\n,\n--border-count\nPython parameters:\nborder_count\nAlias:\nmax_bin\nR parameters:\nborder_count\nDescription\nRecommended values are up to 255. Larger values slow down the training.\nThe number of splits for numerical features. Allowed values are integers from 1 to 65535 inclusively.\nInternal dataset order\nUse this option if the objects in your dataset are given in the required order. In this case, random permutations are not performed during the \nTransforming categorical features to numerical features\nand \nChoosing the tree structure\nstages.\nParameters\nCommand-line version parameters:\n--has-time\nPython parameters:\n--has-time\nR parameters:\n--has-time\nDescription\nUse the order of objects in the input data (do not perform random permutations during the\nTransforming categorical features to numerical features\nand\nChoosing the tree structure\nstages).\nThe Timestamp column type is used to determine the order of objects if specified in the \ninput data\n.\nTree growing policy\nBy default, CatBoost uses symmetric trees, which are built if the growing policy is set to SymmetricTree.\nSuch trees are built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric.\nSymmetric trees have a very good prediction speed (roughly 10 times faster than non-symmetric trees) and give better quality in many cases.\nHowever, in some cases, other tree growing strategies can give better results than growing symmetric trees.\nTry to analyze the results obtained with different growing trees strategies.\nSpecifics: Symmetric trees, that are used by default, can be applied much faster (up to 10 times faster).\nParameters\nCommand-line version parameters:\n--grow-policy\nPython parameters:\ngrow_policy\nR parameters:\ngrow_policy\nDescription\nThe tree growing policy. Defines how to perform greedy tree construction.\nPossible values:\nSymmetricTree — A tree is built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric.\nDepthwise — A tree is built level by level until the specified depth is reached. On each iteration, all non-terminal leaves from the last tree level are split. Each leaf is split by condition with the best loss improvement.\nNote\nModels with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm.\nLossguide — A tree is built leaf by leaf until the specified maximum number of leaves is reached. On each iteration, non-terminal leaf with the best loss improvement is split.\nNote\nModels with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm.\nCommand-line version parameters:\n--min-data-in-leaf\nPython parameters:\nmin_data_in_leaf\nAlias:\nmin_child_samples\nR parameters:\nmin_data_in_leaf\nDescription\nThe minimum number of training samples in a leaf. CatBoost does not search for new splits in leaves with samples count less than the specified value.\nCan be used only with the Lossguide and Depthwise growing policies.\nCommand-line version parameters:\n--max-leaves\nPython parameters:\nmax_leaves\nAlias:\nnum_leaves\nR parameters:\nmax_leaves\nDescription\nThe maximum number of leafs in the resulting tree. Can be used only with the Lossguide growing policy.\nNote\nIt is not recommended to use values greater than 64, since it can significantly slow down the training process.\nGolden features\nIf the dataset has a feature, which is a strong predictor of the result, the pre-quantisation of this feature may decrease the information that the model can get from it. It is recommended to use an increased number of borders (1024) for this feature.\nNote\nAn increased number of borders should not be set for all features. It is recommended to set it for one or two golden features.\nCommand-line\nPython\nR\nParameter\nDescription\n--per-float-feature-quantization\nA semicolon separated list of quantization descriptions.\nFormat:\nFeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]\nExamples:\n--per-float-feature-quantization 0:border_count=1024\nIn this example, the feature indexed 0 has 1024 borders.\n--per-float-feature-quantization 0:border_count=1024;1:border_count=1024\nIn this example, features indexed 0 and 1 have 1024 borders.\nParameter\nDescription\nper_float_feature_quantization\nThe quantization description for the specified feature or list of features.\nDescription format for a single feature:\nFeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]\nExamples:\nper_float_feature_quantization='0:border_count=1024'\nIn this example, the feature indexed 0 has 1024 borders.\nper_float_feature_quantization=[\n'0:border_count=1024'\n,\n'1:border_count=1024'\n]\nIn this example, features indexed 0 and 1 have 1024 borders.\nParameter\nDescription\nper_float_feature_quantization\nThe quantization description for the specified feature or list of features.\nDescription format for a single feature:\nFeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]\nExamples:\nper_float_feature_quantization\n=\n'0:border_count=1024'\n)\nIn this example, the feature indexed 0 has 1024 borders.\nper_float_feature_quantization\n=\nc\n(\n'0:border_count=1024'\n,\n'1:border_count=1024'\nIn this example, features indexed 0 and 1 have 1024 borders.\nMethods for hyperparameter search\nThe Python package provides Grid and Randomized search methods for searching optimal parameter values for training the model with the given dataset.\nParameters\nClass\nMethod\nDescription\nCatBoost\ngrid_search\nA simple grid search over specified parameter values for a model.\nCatBoost\nrandomized_search\nA simple randomized search on hyperparameters.\nCatBoostClassifier\ngrid_search\nA simple grid search over specified parameter values for a model.\nCatBoostClassifier\nrandomized_search\nA simple randomized search on hyperparameters.\nCatBoostRegressor\ngrid_search\nA simple grid search over specified parameter values for a model.\nCatBoostRegressor\nrandomized_search\nA simple randomized search on hyperparameters.\nMethods for hyperparameter search by optuna\nOptuna is a famous hyperparameter optimization framework.\nOptuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials.\nCatboost supports to stop unpromising trial of hyperparameter by callbacking after iteration functionality.\nPull Request\nThe following is an optuna example that demonstrates a pruner for CatBoost.\nExample","attrs_markdown":"[![Logo icon](https:\/\/yastatic.net\/s3\/locdoc\/daas-static\/catboost\/71b237a322eec6f2889af0dae2a9c549.svg)](https:\/\/catboost.ai\/ \"CatBoost\")\n\n- Installation\n  - [Overview](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/installation)\n  - Python package installation\n  - CatBoost for Apache Spark installation\n  - R package installation\n  - Command-line version binary\n  - Build from source\n- Key Features\n- Training parameters\n- Python package\n- CatBoost for Apache Spark\n- R package\n- Command-line version\n- Applying models\n- Objectives and metrics\n- Model analysis\n- Data format description\n- [Parameter tuning](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning)\n- [Speeding up the training](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/speed-up-training)\n- Data visualization\n- Algorithm details\n- [FAQ](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/faq)\n- Educational materials\n- [Development and contributions](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/development-and-contributions)\n- [Contacts](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/contacts)\n\nParameter tuning\n\n## In this article:\n- [One-hot encoding](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#one-hot-enc)\n- [Number of trees](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#trees-number)\n- [Learning rate](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#learning-rate)\n- [Tree depth](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#tree-depth)\n- [L2 regularization](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#l2-reg)\n- [Random strength](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#rand-str)\n- [Bagging temperature](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#bagg-temp)\n- [Border count](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#border-count)\n- [Internal dataset order](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#internal-dataset-order)\n- [Tree growing policy](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#tree-growing-policy)\n- [Golden features](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#golden-features)\n- [Methods for hyperparameter search](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#defining-optimal-parameter-values)\n- [Methods for hyperparameter search by optuna](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#methods-for-hyperparameter-search-by-optuna)\n\n# Parameter tuning\n- [One-hot encoding](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#one-hot-enc)\n- [Number of trees](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#trees-number)\n- [Learning rate](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#learning-rate)\n- [Tree depth](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#tree-depth)\n- [L2 regularization](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#l2-reg)\n- [Random strength](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#rand-str)\n- [Bagging temperature](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#bagg-temp)\n- [Border count](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#border-count)\n- [Internal dataset order](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#internal-dataset-order)\n- [Tree growing policy](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#tree-growing-policy)\n- [Golden features](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#golden-features)\n- [Methods for hyperparameter search](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#defining-optimal-parameter-values)\n- [Methods for hyperparameter search by optuna](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/parameter-tuning#methods-for-hyperparameter-search-by-optuna)\n\nCatBoost provides a flexible interface for parameter tuning and can be configured to suit different tasks.\n\nThis section contains some tips on the possible parameter settings.\n\n## One-hot encoding\nWarning\n\nDo not use one-hot encoding during preprocessing. This affects both the training speed and the resulting quality.\n\nSometimes when categorical features don't have a lot of values, one-hot encoding works well.\n\nUsually one-hot encoding does not significantly improve the quality of the model. But if it is required, use the inbuilt parameters instead of preprocessing the dataset.\n\nParameters\n\n**Command-line version parameters:** `--one-hot-max-size`\n\n**Python parameters:** `one_hot_max_size`\n\n**R parameters:** `one_hot_max_size`\n\n#### Description\nUse one-hot encoding for all categorical features with a number of different values less than or equal to the given parameter value. Ctrs are not calculated for such features.\n\n**Default value**\n\nThe default value depends on various conditions:\n\n- N\/A if training is performed on CPU in Pairwise scoring mode\n  Read more about Pairwise scoring\n  \n  The following loss functions use Pairwise scoring:\n  \n  - YetiRankPairwise\n  - PairLogitPairwise\n  - QueryCrossEntropy\n  \n  Pairwise scoring is slightly different from regular training on pairs, since pairs are generated only internally during the training for the corresponding metrics. One-hot encoding is not available for these loss functions.\n- 255 if training is performed on GPU and the selected Ctr types require target data that is not available during the training\n- 10 if training is performed in [Ranking](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/loss-functions-ranking) mode\n- 2 if none of the conditions above is met\n## Number of trees\nIt is recommended to check that there is no obvious underfitting or overfitting before tuning any other parameters. In order to do this it is necessary to analyze the metric value on the validation dataset and select the appropriate number of iterations.\n\nThis can be done by setting the number of [iterations](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/references\/training-parameters\/common#iterations) to a large value, using the [overfitting detector](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/overfitting-detector) parameters and turning the [use best model](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/references\/training-parameters\/common#use_best_model) options on. In this case the resulting model contains only the first `k` best iterations, where `k` is the iteration with the best loss value on the validation dataset.\n\nAlso, the metric for choosing the best model may differ from the one used for optimizing the objective value. For example, it is possible to set the optimized function to Logloss and use the AUC function for the overfitting detector. To do so, use the [evaluation metric](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/references\/training-parameters\/common#eval_metric) parameter.\n\nParameters\n\n**Command-line version parameters:** `-i`, `--iterations`\n\n**Python parameters:** `--iterations`\n\n**R parameters:** `--iterations`\n\n#### Description\nThe maximum number of trees that can be built when solving machine learning problems.\n\nWhen using other parameters that limit the number of iterations, the final number of trees may be less than the number specified in this parameter.\n\n**Command-line version parameters:** `--use-best-model`\n\n**Python parameters:** `--use-best-model`\n\n**R parameters:** `--use-best-model`\n\n#### Description\nIf this parameter is set, the number of trees that are saved in the resulting model is defined as follows:\n\n1. Build the number of trees defined by the training parameters.\n2. Use the validation dataset to identify the iteration with the optimal value of the metric specified in `--eval-metric` (`--eval-metric`).\n\nNo trees are saved after this iteration.\n\nThis option requires a validation dataset to be provided.\n\n**Command-line version parameters:** `--eval-metric`\n\n**Python parameters:** `--eval-metric`\n\n**R parameters:** `--eval-metric`\n\n#### Description\nThe metric used for overfitting detection (if enabled) and best model selection (if enabled). Some metrics support optional parameters (see the [Objectives and metrics](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/loss-functions) section for details on each metric).\n\nFormat:\n```\n<Metric>[:<parameter 1>=<value>;..;<parameter N>=<value>]\n```\n[Supported metrics](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/references\/eval-metric__supported-metrics)\n\nExamples:\n```\nR2\n```\n```\nQuantile:alpha=0.3\n```\n**Command-line version parameters:** **Overfitting detection settings**\n\n**Command-line version parameters:** `--od-type`\n\n**Python parameters:** `od_type`\n\n**R parameters:** `od_type`\n\n#### Description\nThe type of the overfitting detector to use.\n\nPossible values:\n\n- IncToDec\n- Iter\n\n**Command-line version parameters:** `--od-pval`\n\n**Python parameters:** `od_pval`\n\n**R parameters:** `od_pval`\n\n#### Description\nThe threshold for the IncToDec [overfitting detector](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/overfitting-detector) type. The training is stopped when the specified value is reached. Requires that a validation dataset was input.\n\nFor best results, it is recommended to set a value in the range \\[ 1 0 – 10 ; 1 0 − 2 \\] \\[10^{–10}; 10^{-2}\\] \\[10–10;10−2\\].\n\nThe larger the value, the earlier overfitting is detected.\n\nAlert\n\nDo not use this parameter with the Iter overfitting detector type.\n\n**Command-line version parameters:** `--od-wait`\n\n**Python parameters:** `od_wait`\n\n**R parameters:** `od_wait`\n\n#### Description\nThe number of iterations to continue the training after the iteration with the optimal metric value.  \n The purpose of this parameter differs depending on the selected overfitting detector type:\n\n- IncToDec — Ignore the overfitting detector when the threshold is reached and continue learning for the specified number of iterations after the iteration with the optimal metric value.\n- Iter — Consider the model overfitted and stop training after the specified number of iterations since the iteration with the optimal metric value.\n## Learning rate\nThis setting is used for reducing the gradient step. It affects the overall time of training: the smaller the value, the more iterations are required for training. Choose the value based on the performance expectations.\n\nBy default, the learning rate is defined automatically based on the dataset properties and the number of iterations. The automatically defined value should be close to the optimal one.\n\nPossible ways of adjusting the learning rate depending on the overfitting results:\n\n- There is no overfitting on the last iterations of training (the training does not converge) — increase the learning rate.\n- Overfitting is detected — decrease the learning rate.\n\nParameters\n\n**Command-line version parameters:** `-w`, `--learning-rate`\n\n**Python parameters:** `learning_rate`\n\n**R parameters:** `learning_rate`\n\n#### Description\nThe learning rate. Used for reducing the gradient step.\n## Tree depth\nIn most cases, the optimal depth ranges from 4 to 10. Values in the range from 6 to 10 are recommended.\n\nNote\n\nThe maximum depth of the trees is limited to 8 for pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) when the training is performed on GPU.\n\nParameters\n\n**Command-line version parameters:** `-n`, `--depth`\n\n**Python parameters:** `depth`\n\n**R parameters:** `depth`\n\n#### Description\nDepth of the trees. The range of supported values depends on the processing unit type and the type of the selected loss function:\n\n- CPU — Any integer up to 16.\n- GPU — Any integer up to 8 pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) and up to 16 for all other loss functions.\n## L2 regularization\nTry different values for the regularizer to find the best possible.\n\nParameters\n\n**Command-line version parameters:** `--l2-leaf-reg`\n\n**Python parameters:** `l2_leaf_reg`\n\n**R parameters:** `l2_leaf_reg`\n\n#### Description\nCoefficient at the L2 regularization term of the cost function.  \n Any positive value is allowed.\n## Random strength\nTry setting different values for the `random_strength` parameter.\n\nParameters\n\n**Command-line version parameters:** `--random-strength`\n\n**Python parameters:** `random_strength`\n\n**R parameters:** `random_strength`\n\n#### Description\nThe amount of randomness to use for scoring splits when the tree structure is selected. Use this parameter to avoid overfitting the model.\n\nThe value of this parameter is used when selecting splits. On every iteration each possible split gets a score (for example, the score indicates how much adding this split will improve the loss function for the training dataset). The split with the highest score is selected.\n\nThe scores have no randomness. A normally distributed random variable is added to the score of the feature. It has a zero mean and a variance that decreases during the training. The value of this parameter is the multiplier of the variance.\n\nThis parameter is not supported for the following loss functions:\n\n- QueryCrossEntropy\n- YetiRankPairwise\n- PairLogitPairwise\n## Bagging temperature\nTry setting different values for the `bagging_temperature` parameter\n\nParameters\n\n**Command-line version parameters:** `--bagging-temperature`\n\n**Python parameters:** `bagging_temperature`\n\n**R parameters:** `bagging_temperature`\n\n#### Description\nDefines the settings of the Bayesian bootstrap. It is used by default in classification and regression modes.\n\nUse the Bayesian bootstrap to assign random weights to objects.\n\nThe weights are sampled from exponential distribution if the value of this parameter is set to \"1\". All weights are equal to 1 if the value of this parameter is set to \"0\".\n\nPossible values are in the range \\[ 0 ; inf ⁡ ) \\[0; \\\\inf) \\[0;inf). The higher the value the more aggressive the bagging is.\n\nThis parameter can be used if the selected bootstrap type is Bayesian.\n## Border count\nThe number of splits for numerical features.\n\nThe default value depends on the processing unit type and other parameters:\n\n- CPU: 254\n- GPU in PairLogitPairwise and YetiRankPairwise modes: 32\n- GPU in all other modes: 128\n\nThe value of this parameter significantly impacts the speed of training on GPU. The smaller the value, the faster the training is performed (refer to the [Number of splits for numerical features](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/speed-up-training) section for details).\n\n128 splits are enough for many datasets. However, try to set the value of this parameter to 254 when training on GPU if the best possible quality is required.\n\nThe value of this parameter does not significantly impact the speed of training on CPU. Try to set it to 254 for the best possible quality.\n\nParameters\n\n**Command-line version parameters:** `-x`, `--border-count`\n\n**Python parameters:** `border_count`\n\n*Alias:*`max_bin`\n\n**R parameters:** `border_count`\n\n#### Description\nRecommended values are up to 255. Larger values slow down the training.\n\nThe number of splits for numerical features. Allowed values are integers from 1 to 65535 inclusively.\n## Internal dataset order\nUse this option if the objects in your dataset are given in the required order. In this case, random permutations are not performed during the [Transforming categorical features to numerical features](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/algorithm-main-stages_cat-to-numberic) and [Choosing the tree structure](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/algorithm-main-stages_choose-tree-structure) stages.\n\nParameters\n\n**Command-line version parameters:** `--has-time`\n\n**Python parameters:** `--has-time`\n\n**R parameters:** `--has-time`\n\n#### Description\nUse the order of objects in the input data (do not perform random permutations during the [Transforming categorical features to numerical features](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/algorithm-main-stages_cat-to-numberic) and [Choosing the tree structure](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/algorithm-main-stages_choose-tree-structure) stages).\n\nThe Timestamp column type is used to determine the order of objects if specified in the [input data](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/input-data).\n## Tree growing policy\nBy default, CatBoost uses symmetric trees, which are built if the growing policy is set to SymmetricTree.\n\nSuch trees are built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric.\n\nSymmetric trees have a very good prediction speed (roughly 10 times faster than non-symmetric trees) and give better quality in many cases.\n\nHowever, in some cases, other tree growing strategies can give better results than growing symmetric trees.\n\nTry to analyze the results obtained with different growing trees strategies.\n\nSpecifics: Symmetric trees, that are used by default, can be applied much faster (up to 10 times faster).\n\nParameters\n\n**Command-line version parameters:** `--grow-policy`\n\n**Python parameters:** `grow_policy`\n\n**R parameters:** `grow_policy`\n\n#### Description\nThe tree growing policy. Defines how to perform greedy tree construction.\n\nPossible values:\n\n- SymmetricTree — A tree is built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric.\n- Depthwise — A tree is built level by level until the specified depth is reached. On each iteration, all non-terminal leaves from the last tree level are split. Each leaf is split by condition with the best loss improvement.\n  Note\n  \n  Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm.\n- Lossguide — A tree is built leaf by leaf until the specified maximum number of leaves is reached. On each iteration, non-terminal leaf with the best loss improvement is split.\n  Note\n  \n  Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm.\n\n**Command-line version parameters:** `--min-data-in-leaf`\n\n**Python parameters:** `min_data_in_leaf`\n\n*Alias:*`min_child_samples`\n\n**R parameters:** `min_data_in_leaf`\n\n#### Description\nThe minimum number of training samples in a leaf. CatBoost does not search for new splits in leaves with samples count less than the specified value.  \n Can be used only with the Lossguide and Depthwise growing policies.\n\n**Command-line version parameters:** `--max-leaves`\n\n**Python parameters:** `max_leaves`\n\n*Alias:*`num_leaves`\n\n**R parameters:** `max_leaves`\n\n#### Description\nThe maximum number of leafs in the resulting tree. Can be used only with the Lossguide growing policy.\n\nNote\n\nIt is not recommended to use values greater than 64, since it can significantly slow down the training process.\n## Golden features\nIf the dataset has a feature, which is a strong predictor of the result, the pre-quantisation of this feature may decrease the information that the model can get from it. It is recommended to use an increased number of borders (1024) for this feature.\n\nNote\n\nAn increased number of borders should not be set for all features. It is recommended to set it for one or two golden features.\n\nCommand-line\n\nPython\n\nR\n\n| Parameter | Description |\n|---|---|\n| `--per-float-feature-quantization` | A semicolon separated list of quantization descriptions.  Format:   `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` |\n\nExamples:\n\n- ```\n    --per-float-feature-quantization 0:border_count=1024\n  ```\n  In this example, the feature indexed 0 has 1024 borders.\n- ```\n    --per-float-feature-quantization 0:border_count=1024;1:border_count=1024\n  ```\n  In this example, features indexed 0 and 1 have 1024 borders.\n\n| Parameter | Description |\n|---|---|\n| `per_float_feature_quantization` | The quantization description for the specified feature or list of features.  Description format for a single feature:  `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` |\n\nExamples:\n\n- ```\n    per_float_feature_quantization='0:border_count=1024'\n  ```\n  In this example, the feature indexed 0 has 1024 borders.\n- ```\n    per_float_feature_quantization=['0:border_count=1024', '1:border_count=1024']\n  ```\n  In this example, features indexed 0 and 1 have 1024 borders.\n\n| Parameter | Description |\n|---|---|\n| `per_float_feature_quantization` | The quantization description for the specified feature or list of features.  Description format for a single feature:  `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` |\n\nExamples:\n\n- ```\n    per_float_feature_quantization = '0:border_count=1024')\n  ```\n  In this example, the feature indexed 0 has 1024 borders.\n- ```\n    per_float_feature_quantization = c('0:border_count=1024', '1:border_count=1024'\n  ```\n  In this example, features indexed 0 and 1 have 1024 borders.\n## Methods for hyperparameter search\nThe Python package provides Grid and Randomized search methods for searching optimal parameter values for training the model with the given dataset.\n\nParameters\n\n| Class | Method | Description |\n|---|---|---|\n| [CatBoost](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboost) | [grid\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboost_grid_search) | A simple grid search over specified parameter values for a model. |\n| [CatBoost](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboost) | [randomized\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboost_randomized_search) | A simple randomized search on hyperparameters. |\n| [CatBoostClassifier](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboostclassifier) | [grid\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboostclassifier_grid_search) | A simple grid search over specified parameter values for a model. |\n| [CatBoostClassifier](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboostclassifier) | [randomized\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboostclassifier_randomized_search) | A simple randomized search on hyperparameters. |\n| [CatBoostRegressor](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboostregressor) | [grid\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboostregressor_grid_search) | A simple grid search over specified parameter values for a model. |\n| [CatBoostRegressor](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboostregressor) | [randomized\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/python-reference_catboostregressor_randomized_search) | A simple randomized search on hyperparameters. |\n## Methods for hyperparameter search by optuna\nOptuna is a famous hyperparameter optimization framework.  \n Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials.  \n Catboost supports to stop unpromising trial of hyperparameter by callbacking after iteration functionality. [Pull Request](https:\/\/github.com\/catboost\/catboost\/pull\/1697\/files#diff-ccca44461ac6b094190f29fec157a227996e226ea483213680dd0a152cd412eaR9679)\n\nThe following is an optuna example that demonstrates a pruner for CatBoost. [Example](https:\/\/github.com\/optuna\/optuna-examples\/blob\/main\/catboost\/catboost_pruning.py)\n\n### Was the article helpful?\nYes\n\nNo\n\nPrevious\n\n[ROC curve points](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/output-data_roc-curve-points)\n\nNext\n\n[Speeding up the training](https:\/\/catboost.ai\/docs\/en\/concepts\/en\/concepts\/speed-up-training)\n\n![](https:\/\/mc.yandex.ru\/watch\/60763294)","attrs_readable_markdown":"CatBoost provides a flexible interface for parameter tuning and can be configured to suit different tasks.\n\nThis section contains some tips on the possible parameter settings.\n\n## One-hot encoding\nWarning\n\nDo not use one-hot encoding during preprocessing. This affects both the training speed and the resulting quality.\n\nSometimes when categorical features don't have a lot of values, one-hot encoding works well.\n\nUsually one-hot encoding does not significantly improve the quality of the model. But if it is required, use the inbuilt parameters instead of preprocessing the dataset.\n\nParameters\n\n**Command-line version parameters:** `--one-hot-max-size`\n\n**Python parameters:** `one_hot_max_size`\n\n**R parameters:** `one_hot_max_size`\n\n#### Description\nUse one-hot encoding for all categorical features with a number of different values less than or equal to the given parameter value. Ctrs are not calculated for such features.\n\n**Default value**\n\nThe default value depends on various conditions:\n\n- N\/A if training is performed on CPU in Pairwise scoring mode\n  Read more about Pairwise scoring\n  \n  The following loss functions use Pairwise scoring:\n  \n  - YetiRankPairwise\n  - PairLogitPairwise\n  - QueryCrossEntropy\n  \n  Pairwise scoring is slightly different from regular training on pairs, since pairs are generated only internally during the training for the corresponding metrics. One-hot encoding is not available for these loss functions.\n- 255 if training is performed on GPU and the selected Ctr types require target data that is not available during the training\n- 10 if training is performed in [Ranking](https:\/\/catboost.ai\/docs\/en\/concepts\/loss-functions-ranking) mode\n- 2 if none of the conditions above is met\n## Number of trees\nIt is recommended to check that there is no obvious underfitting or overfitting before tuning any other parameters. In order to do this it is necessary to analyze the metric value on the validation dataset and select the appropriate number of iterations.\n\nThis can be done by setting the number of [iterations](https:\/\/catboost.ai\/docs\/en\/references\/training-parameters\/common#iterations) to a large value, using the [overfitting detector](https:\/\/catboost.ai\/docs\/en\/concepts\/overfitting-detector) parameters and turning the [use best model](https:\/\/catboost.ai\/docs\/en\/references\/training-parameters\/common#use_best_model) options on. In this case the resulting model contains only the first `k` best iterations, where `k` is the iteration with the best loss value on the validation dataset.\n\nAlso, the metric for choosing the best model may differ from the one used for optimizing the objective value. For example, it is possible to set the optimized function to Logloss and use the AUC function for the overfitting detector. To do so, use the [evaluation metric](https:\/\/catboost.ai\/docs\/en\/references\/training-parameters\/common#eval_metric) parameter.\n\nParameters\n\n**Command-line version parameters:** `-i`, `--iterations`\n\n**Python parameters:** `--iterations`\n\n**R parameters:** `--iterations`\n\n#### Description\nThe maximum number of trees that can be built when solving machine learning problems.\n\nWhen using other parameters that limit the number of iterations, the final number of trees may be less than the number specified in this parameter.\n\n**Command-line version parameters:** `--use-best-model`\n\n**Python parameters:** `--use-best-model`\n\n**R parameters:** `--use-best-model`\n\n#### Description\nIf this parameter is set, the number of trees that are saved in the resulting model is defined as follows:\n\n1. Build the number of trees defined by the training parameters.\n2. Use the validation dataset to identify the iteration with the optimal value of the metric specified in `--eval-metric` (`--eval-metric`).\n\nNo trees are saved after this iteration.\n\nThis option requires a validation dataset to be provided.\n\n**Command-line version parameters:** `--eval-metric`\n\n**Python parameters:** `--eval-metric`\n\n**R parameters:** `--eval-metric`\n\n#### Description\nThe metric used for overfitting detection (if enabled) and best model selection (if enabled). Some metrics support optional parameters (see the [Objectives and metrics](https:\/\/catboost.ai\/docs\/en\/concepts\/loss-functions) section for details on each metric).\n\nFormat:\n```\n<Metric>[:<parameter 1>=<value>;..;<parameter N>=<value>]\n```\n[Supported metrics](https:\/\/catboost.ai\/docs\/en\/references\/eval-metric__supported-metrics)\n\nExamples:\n```\nR2\n```\n```\nQuantile:alpha=0.3\n```\n**Command-line version parameters:** **Overfitting detection settings**\n\n**Command-line version parameters:** `--od-type`\n\n**Python parameters:** `od_type`\n\n**R parameters:** `od_type`\n\n#### Description\nThe type of the overfitting detector to use.\n\nPossible values:\n\n- IncToDec\n- Iter\n\n**Command-line version parameters:** `--od-pval`\n\n**Python parameters:** `od_pval`\n\n**R parameters:** `od_pval`\n\n#### Description\nThe threshold for the IncToDec [overfitting detector](https:\/\/catboost.ai\/docs\/en\/concepts\/overfitting-detector) type. The training is stopped when the specified value is reached. Requires that a validation dataset was input.\n\nFor best results, it is recommended to set a value in the range \\[ 1 0 – 10 ; 1 0 − 2 \\] \\[10^{–10}; 10^{-2}\\].\n\nThe larger the value, the earlier overfitting is detected.\n\nAlert\n\nDo not use this parameter with the Iter overfitting detector type.\n\n**Command-line version parameters:** `--od-wait`\n\n**Python parameters:** `od_wait`\n\n**R parameters:** `od_wait`\n\n#### Description\nThe number of iterations to continue the training after the iteration with the optimal metric value.  \n The purpose of this parameter differs depending on the selected overfitting detector type:\n\n- IncToDec — Ignore the overfitting detector when the threshold is reached and continue learning for the specified number of iterations after the iteration with the optimal metric value.\n- Iter — Consider the model overfitted and stop training after the specified number of iterations since the iteration with the optimal metric value.\n## Learning rate\nThis setting is used for reducing the gradient step. It affects the overall time of training: the smaller the value, the more iterations are required for training. Choose the value based on the performance expectations.\n\nBy default, the learning rate is defined automatically based on the dataset properties and the number of iterations. The automatically defined value should be close to the optimal one.\n\nPossible ways of adjusting the learning rate depending on the overfitting results:\n\n- There is no overfitting on the last iterations of training (the training does not converge) — increase the learning rate.\n- Overfitting is detected — decrease the learning rate.\n\nParameters\n\n**Command-line version parameters:** `-w`, `--learning-rate`\n\n**Python parameters:** `learning_rate`\n\n**R parameters:** `learning_rate`\n\n#### Description\nThe learning rate. Used for reducing the gradient step.\n## Tree depth\nIn most cases, the optimal depth ranges from 4 to 10. Values in the range from 6 to 10 are recommended.\n\nNote\n\nThe maximum depth of the trees is limited to 8 for pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) when the training is performed on GPU.\n\nParameters\n\n**Command-line version parameters:** `-n`, `--depth`\n\n**Python parameters:** `depth`\n\n**R parameters:** `depth`\n\n#### Description\nDepth of the trees. The range of supported values depends on the processing unit type and the type of the selected loss function:\n\n- CPU — Any integer up to 16.\n- GPU — Any integer up to 8 pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) and up to 16 for all other loss functions.\n## L2 regularization\nTry different values for the regularizer to find the best possible.\n\nParameters\n\n**Command-line version parameters:** `--l2-leaf-reg`\n\n**Python parameters:** `l2_leaf_reg`\n\n**R parameters:** `l2_leaf_reg`\n\n#### Description\nCoefficient at the L2 regularization term of the cost function.  \n Any positive value is allowed.\n## Random strength\nTry setting different values for the `random_strength` parameter.\n\nParameters\n\n**Command-line version parameters:** `--random-strength`\n\n**Python parameters:** `random_strength`\n\n**R parameters:** `random_strength`\n\n#### Description\nThe amount of randomness to use for scoring splits when the tree structure is selected. Use this parameter to avoid overfitting the model.\n\nThe value of this parameter is used when selecting splits. On every iteration each possible split gets a score (for example, the score indicates how much adding this split will improve the loss function for the training dataset). The split with the highest score is selected.\n\nThe scores have no randomness. A normally distributed random variable is added to the score of the feature. It has a zero mean and a variance that decreases during the training. The value of this parameter is the multiplier of the variance.\n\nThis parameter is not supported for the following loss functions:\n\n- QueryCrossEntropy\n- YetiRankPairwise\n- PairLogitPairwise\n## Bagging temperature\nTry setting different values for the `bagging_temperature` parameter\n\nParameters\n\n**Command-line version parameters:** `--bagging-temperature`\n\n**Python parameters:** `bagging_temperature`\n\n**R parameters:** `bagging_temperature`\n\n#### Description\nDefines the settings of the Bayesian bootstrap. It is used by default in classification and regression modes.\n\nUse the Bayesian bootstrap to assign random weights to objects.\n\nThe weights are sampled from exponential distribution if the value of this parameter is set to \"1\". All weights are equal to 1 if the value of this parameter is set to \"0\".\n\nPossible values are in the range \\[ 0 ; inf ⁡ ) \\[0; \\\\inf). The higher the value the more aggressive the bagging is.\n\nThis parameter can be used if the selected bootstrap type is Bayesian.\n## Border count\nThe number of splits for numerical features.\n\nThe default value depends on the processing unit type and other parameters:\n\n- CPU: 254\n- GPU in PairLogitPairwise and YetiRankPairwise modes: 32\n- GPU in all other modes: 128\n\nThe value of this parameter significantly impacts the speed of training on GPU. The smaller the value, the faster the training is performed (refer to the [Number of splits for numerical features](https:\/\/catboost.ai\/docs\/en\/concepts\/speed-up-training) section for details).\n\n128 splits are enough for many datasets. However, try to set the value of this parameter to 254 when training on GPU if the best possible quality is required.\n\nThe value of this parameter does not significantly impact the speed of training on CPU. Try to set it to 254 for the best possible quality.\n\nParameters\n\n**Command-line version parameters:** `-x`, `--border-count`\n\n**Python parameters:** `border_count`\n\n*Alias:*`max_bin`\n\n**R parameters:** `border_count`\n\n#### Description\nRecommended values are up to 255. Larger values slow down the training.\n\nThe number of splits for numerical features. Allowed values are integers from 1 to 65535 inclusively.\n## Internal dataset order\nUse this option if the objects in your dataset are given in the required order. In this case, random permutations are not performed during the [Transforming categorical features to numerical features](https:\/\/catboost.ai\/docs\/en\/concepts\/algorithm-main-stages_cat-to-numberic) and [Choosing the tree structure](https:\/\/catboost.ai\/docs\/en\/concepts\/algorithm-main-stages_choose-tree-structure) stages.\n\nParameters\n\n**Command-line version parameters:** `--has-time`\n\n**Python parameters:** `--has-time`\n\n**R parameters:** `--has-time`\n\n#### Description\nUse the order of objects in the input data (do not perform random permutations during the [Transforming categorical features to numerical features](https:\/\/catboost.ai\/docs\/en\/concepts\/algorithm-main-stages_cat-to-numberic) and [Choosing the tree structure](https:\/\/catboost.ai\/docs\/en\/concepts\/algorithm-main-stages_choose-tree-structure) stages).\n\nThe Timestamp column type is used to determine the order of objects if specified in the [input data](https:\/\/catboost.ai\/docs\/en\/concepts\/input-data).\n## Tree growing policy\nBy default, CatBoost uses symmetric trees, which are built if the growing policy is set to SymmetricTree.\n\nSuch trees are built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric.\n\nSymmetric trees have a very good prediction speed (roughly 10 times faster than non-symmetric trees) and give better quality in many cases.\n\nHowever, in some cases, other tree growing strategies can give better results than growing symmetric trees.\n\nTry to analyze the results obtained with different growing trees strategies.\n\nSpecifics: Symmetric trees, that are used by default, can be applied much faster (up to 10 times faster).\n\nParameters\n\n**Command-line version parameters:** `--grow-policy`\n\n**Python parameters:** `grow_policy`\n\n**R parameters:** `grow_policy`\n\n#### Description\nThe tree growing policy. Defines how to perform greedy tree construction.\n\nPossible values:\n\n- SymmetricTree — A tree is built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric.\n- Depthwise — A tree is built level by level until the specified depth is reached. On each iteration, all non-terminal leaves from the last tree level are split. Each leaf is split by condition with the best loss improvement.\n  Note\n  \n  Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm.\n- Lossguide — A tree is built leaf by leaf until the specified maximum number of leaves is reached. On each iteration, non-terminal leaf with the best loss improvement is split.\n  Note\n  \n  Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm.\n\n**Command-line version parameters:** `--min-data-in-leaf`\n\n**Python parameters:** `min_data_in_leaf`\n\n*Alias:*`min_child_samples`\n\n**R parameters:** `min_data_in_leaf`\n\n#### Description\nThe minimum number of training samples in a leaf. CatBoost does not search for new splits in leaves with samples count less than the specified value.  \n Can be used only with the Lossguide and Depthwise growing policies.\n\n**Command-line version parameters:** `--max-leaves`\n\n**Python parameters:** `max_leaves`\n\n*Alias:*`num_leaves`\n\n**R parameters:** `max_leaves`\n\n#### Description\nThe maximum number of leafs in the resulting tree. Can be used only with the Lossguide growing policy.\n\nNote\n\nIt is not recommended to use values greater than 64, since it can significantly slow down the training process.\n## Golden features\nIf the dataset has a feature, which is a strong predictor of the result, the pre-quantisation of this feature may decrease the information that the model can get from it. It is recommended to use an increased number of borders (1024) for this feature.\n\nNote\n\nAn increased number of borders should not be set for all features. It is recommended to set it for one or two golden features.\n\nCommand-line\n\nPython\n\nR\n\n| Parameter | Description |\n|---|---|\n| `--per-float-feature-quantization` | A semicolon separated list of quantization descriptions.  Format:   `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` |\n\nExamples:\n\n- ```\n    --per-float-feature-quantization 0:border_count=1024\n  ```\n  In this example, the feature indexed 0 has 1024 borders.\n- ```\n    --per-float-feature-quantization 0:border_count=1024;1:border_count=1024\n  ```\n  In this example, features indexed 0 and 1 have 1024 borders.\n\n| Parameter | Description |\n|---|---|\n| `per_float_feature_quantization` | The quantization description for the specified feature or list of features.  Description format for a single feature:  `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` |\n\nExamples:\n\n- ```\n    per_float_feature_quantization='0:border_count=1024'\n  ```\n  In this example, the feature indexed 0 has 1024 borders.\n- ```\n    per_float_feature_quantization=['0:border_count=1024', '1:border_count=1024']\n  ```\n  In this example, features indexed 0 and 1 have 1024 borders.\n\n| Parameter | Description |\n|---|---|\n| `per_float_feature_quantization` | The quantization description for the specified feature or list of features.  Description format for a single feature:  `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` |\n\nExamples:\n\n- ```\n    per_float_feature_quantization = '0:border_count=1024')\n  ```\n  In this example, the feature indexed 0 has 1024 borders.\n- ```\n    per_float_feature_quantization = c('0:border_count=1024', '1:border_count=1024'\n  ```\n  In this example, features indexed 0 and 1 have 1024 borders.\n## Methods for hyperparameter search\nThe Python package provides Grid and Randomized search methods for searching optimal parameter values for training the model with the given dataset.\n\nParameters\n\n| Class | Method | Description |\n|---|---|---|\n| [CatBoost](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboost) | [grid\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboost_grid_search) | A simple grid search over specified parameter values for a model. |\n| [CatBoost](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboost) | [randomized\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboost_randomized_search) | A simple randomized search on hyperparameters. |\n| [CatBoostClassifier](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboostclassifier) | [grid\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboostclassifier_grid_search) | A simple grid search over specified parameter values for a model. |\n| [CatBoostClassifier](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboostclassifier) | [randomized\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboostclassifier_randomized_search) | A simple randomized search on hyperparameters. |\n| [CatBoostRegressor](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboostregressor) | [grid\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboostregressor_grid_search) | A simple grid search over specified parameter values for a model. |\n| [CatBoostRegressor](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboostregressor) | [randomized\\_search](https:\/\/catboost.ai\/docs\/en\/concepts\/python-reference_catboostregressor_randomized_search) | A simple randomized search on hyperparameters. |\n## Methods for hyperparameter search by optuna\nOptuna is a famous hyperparameter optimization framework.  \n Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials.  \n Catboost supports to stop unpromising trial of hyperparameter by callbacking after iteration functionality. [Pull Request](https:\/\/github.com\/catboost\/catboost\/pull\/1697\/files#diff-ccca44461ac6b094190f29fec157a227996e226ea483213680dd0a152cd412eaR9679)\n\nThe following is an optuna example that demonstrates a pruner for CatBoost. [Example](https:\/\/github.com\/optuna\/optuna-examples\/blob\/main\/catboost\/catboost_pruning.py)","meta_canonical":null,"ml_categories_json":"{\"\/Computers_and_Electronics\":969,\"\/Computers_and_Electronics\/Software\":880,\"\/Computers_and_Electronics\/Software\/Software_Utilities\":760}","ml_types_json":"{\"\/Article\":501,\"\/Article\/Tutorial_or_Guide\":486}","ml_intent_types_json":"{\"Informational\":985}","meta_language":"en","attrs_author":null,"attrs_publish_time":0,"attrs_original_publish_time":1731946335,"attrs_is_republished":0,"attrs_nr_words":"2527","attrs_boilerpipe_nr_words":"2354","body_ext_links_number":2,"body_int_links_number":27,"meta_nofollow":0,"meta_noarchive":0,"props_was_rendered":0,"src_redirect":"","download_time_msec":947,"download_ttfb_msec":723,"download_size":266835}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

1 day ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property

Value

URL

https://catboost.ai/docs/en/concepts/parameter-tuning

Last Crawled

2026-04-22 07:43:37 (1 day ago)

First Indexed

2024-11-18 16:12:15 (1 year ago)

HTTP Status Code

200

Content

Meta Title

Parameter tuning | CatBoost

Meta Description

CatBoost provides a flexible interface for parameter tuning and can be configured to suit different tasks.

Meta Canonical

null

Boilerpipe Text

CatBoost provides a flexible interface for parameter tuning and can be configured to suit different tasks. This section contains some tips on the possible parameter settings. One-hot encoding Warning Do not use one-hot encoding during preprocessing. This affects both the training speed and the resulting quality. Sometimes when categorical features don't have a lot of values, one-hot encoding works well. Usually one-hot encoding does not significantly improve the quality of the model. But if it is required, use the inbuilt parameters instead of preprocessing the dataset. Parameters Command-line version parameters: --one-hot-max-size Python parameters: one_hot_max_size R parameters: one_hot_max_size Description Use one-hot encoding for all categorical features with a number of different values less than or equal to the given parameter value. Ctrs are not calculated for such features. Default value The default value depends on various conditions: N/A if training is performed on CPU in Pairwise scoring mode Read more about Pairwise scoring The following loss functions use Pairwise scoring: YetiRankPairwise PairLogitPairwise QueryCrossEntropy Pairwise scoring is slightly different from regular training on pairs, since pairs are generated only internally during the training for the corresponding metrics. One-hot encoding is not available for these loss functions. 255 if training is performed on GPU and the selected Ctr types require target data that is not available during the training 10 if training is performed in Ranking mode 2 if none of the conditions above is met Number of trees It is recommended to check that there is no obvious underfitting or overfitting before tuning any other parameters. In order to do this it is necessary to analyze the metric value on the validation dataset and select the appropriate number of iterations. This can be done by setting the number of iterations to a large value, using the overfitting detector parameters and turning the use best model options on. In this case the resulting model contains only the first k best iterations, where k is the iteration with the best loss value on the validation dataset. Also, the metric for choosing the best model may differ from the one used for optimizing the objective value. For example, it is possible to set the optimized function to Logloss and use the AUC function for the overfitting detector. To do so, use the evaluation metric parameter. Parameters Command-line version parameters: -i , --iterations Python parameters: --iterations R parameters: --iterations Description The maximum number of trees that can be built when solving machine learning problems. When using other parameters that limit the number of iterations, the final number of trees may be less than the number specified in this parameter. Command-line version parameters: --use-best-model Python parameters: --use-best-model R parameters: --use-best-model Description If this parameter is set, the number of trees that are saved in the resulting model is defined as follows: Build the number of trees defined by the training parameters. Use the validation dataset to identify the iteration with the optimal value of the metric specified in --eval-metric ( --eval-metric ). No trees are saved after this iteration. This option requires a validation dataset to be provided. Command-line version parameters: --eval-metric Python parameters: --eval-metric R parameters: --eval-metric Description The metric used for overfitting detection (if enabled) and best model selection (if enabled). Some metrics support optional parameters (see the Objectives and metrics section for details on each metric). Format: <Metric>[:<parameter 1>=<value>;..;<parameter N>=<value>] Supported metrics Examples: R2 Quantile:alpha=0.3 Command-line version parameters: Overfitting detection settings Command-line version parameters: --od-type Python parameters: od_type R parameters: od_type Description The type of the overfitting detector to use. Possible values: IncToDec Iter Command-line version parameters: --od-pval Python parameters: od_pval R parameters: od_pval Description The threshold for the IncToDec overfitting detector type. The training is stopped when the specified value is reached. Requires that a validation dataset was input. For best results, it is recommended to set a value in the range [ 1 0 – 10 ; 1 0 − 2 ] [10^{–10}; 10^{-2}] . The larger the value, the earlier overfitting is detected. Alert Do not use this parameter with the Iter overfitting detector type. Command-line version parameters: --od-wait Python parameters: od_wait R parameters: od_wait Description The number of iterations to continue the training after the iteration with the optimal metric value. The purpose of this parameter differs depending on the selected overfitting detector type: IncToDec — Ignore the overfitting detector when the threshold is reached and continue learning for the specified number of iterations after the iteration with the optimal metric value. Iter — Consider the model overfitted and stop training after the specified number of iterations since the iteration with the optimal metric value. Learning rate This setting is used for reducing the gradient step. It affects the overall time of training: the smaller the value, the more iterations are required for training. Choose the value based on the performance expectations. By default, the learning rate is defined automatically based on the dataset properties and the number of iterations. The automatically defined value should be close to the optimal one. Possible ways of adjusting the learning rate depending on the overfitting results: There is no overfitting on the last iterations of training (the training does not converge) — increase the learning rate. Overfitting is detected — decrease the learning rate. Parameters Command-line version parameters: -w , --learning-rate Python parameters: learning_rate R parameters: learning_rate Description The learning rate. Used for reducing the gradient step. Tree depth In most cases, the optimal depth ranges from 4 to 10. Values in the range from 6 to 10 are recommended. Note The maximum depth of the trees is limited to 8 for pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) when the training is performed on GPU. Parameters Command-line version parameters: -n , --depth Python parameters: depth R parameters: depth Description Depth of the trees. The range of supported values depends on the processing unit type and the type of the selected loss function: CPU — Any integer up to 16. GPU — Any integer up to 8 pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) and up to 16 for all other loss functions. L2 regularization Try different values for the regularizer to find the best possible. Parameters Command-line version parameters: --l2-leaf-reg Python parameters: l2_leaf_reg R parameters: l2_leaf_reg Description Coefficient at the L2 regularization term of the cost function. Any positive value is allowed. Random strength Try setting different values for the random_strength parameter. Parameters Command-line version parameters: --random-strength Python parameters: random_strength R parameters: random_strength Description The amount of randomness to use for scoring splits when the tree structure is selected. Use this parameter to avoid overfitting the model. The value of this parameter is used when selecting splits. On every iteration each possible split gets a score (for example, the score indicates how much adding this split will improve the loss function for the training dataset). The split with the highest score is selected. The scores have no randomness. A normally distributed random variable is added to the score of the feature. It has a zero mean and a variance that decreases during the training. The value of this parameter is the multiplier of the variance. This parameter is not supported for the following loss functions: QueryCrossEntropy YetiRankPairwise PairLogitPairwise Bagging temperature Try setting different values for the bagging_temperature parameter Parameters Command-line version parameters: --bagging-temperature Python parameters: bagging_temperature R parameters: bagging_temperature Description Defines the settings of the Bayesian bootstrap. It is used by default in classification and regression modes. Use the Bayesian bootstrap to assign random weights to objects. The weights are sampled from exponential distribution if the value of this parameter is set to 1 . All weights are equal to 1 if the value of this parameter is set to 0 . Possible values are in the range [ 0 ; inf ⁡ ) [0; \inf) . The higher the value the more aggressive the bagging is. This parameter can be used if the selected bootstrap type is Bayesian. Border count The number of splits for numerical features. The default value depends on the processing unit type and other parameters: CPU: 254 GPU in PairLogitPairwise and YetiRankPairwise modes: 32 GPU in all other modes: 128 The value of this parameter significantly impacts the speed of training on GPU. The smaller the value, the faster the training is performed (refer to the Number of splits for numerical features section for details). 128 splits are enough for many datasets. However, try to set the value of this parameter to 254 when training on GPU if the best possible quality is required. The value of this parameter does not significantly impact the speed of training on CPU. Try to set it to 254 for the best possible quality. Parameters Command-line version parameters: -x , --border-count Python parameters: border_count Alias: max_bin R parameters: border_count Description Recommended values are up to 255. Larger values slow down the training. The number of splits for numerical features. Allowed values are integers from 1 to 65535 inclusively. Internal dataset order Use this option if the objects in your dataset are given in the required order. In this case, random permutations are not performed during the Transforming categorical features to numerical features and Choosing the tree structure stages. Parameters Command-line version parameters: --has-time Python parameters: --has-time R parameters: --has-time Description Use the order of objects in the input data (do not perform random permutations during the Transforming categorical features to numerical features and Choosing the tree structure stages). The Timestamp column type is used to determine the order of objects if specified in the input data . Tree growing policy By default, CatBoost uses symmetric trees, which are built if the growing policy is set to SymmetricTree. Such trees are built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric. Symmetric trees have a very good prediction speed (roughly 10 times faster than non-symmetric trees) and give better quality in many cases. However, in some cases, other tree growing strategies can give better results than growing symmetric trees. Try to analyze the results obtained with different growing trees strategies. Specifics: Symmetric trees, that are used by default, can be applied much faster (up to 10 times faster). Parameters Command-line version parameters: --grow-policy Python parameters: grow_policy R parameters: grow_policy Description The tree growing policy. Defines how to perform greedy tree construction. Possible values: SymmetricTree — A tree is built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric. Depthwise — A tree is built level by level until the specified depth is reached. On each iteration, all non-terminal leaves from the last tree level are split. Each leaf is split by condition with the best loss improvement. Note Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm. Lossguide — A tree is built leaf by leaf until the specified maximum number of leaves is reached. On each iteration, non-terminal leaf with the best loss improvement is split. Note Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm. Command-line version parameters: --min-data-in-leaf Python parameters: min_data_in_leaf Alias: min_child_samples R parameters: min_data_in_leaf Description The minimum number of training samples in a leaf. CatBoost does not search for new splits in leaves with samples count less than the specified value. Can be used only with the Lossguide and Depthwise growing policies. Command-line version parameters: --max-leaves Python parameters: max_leaves Alias: num_leaves R parameters: max_leaves Description The maximum number of leafs in the resulting tree. Can be used only with the Lossguide growing policy. Note It is not recommended to use values greater than 64, since it can significantly slow down the training process. Golden features If the dataset has a feature, which is a strong predictor of the result, the pre-quantisation of this feature may decrease the information that the model can get from it. It is recommended to use an increased number of borders (1024) for this feature. Note An increased number of borders should not be set for all features. It is recommended to set it for one or two golden features. Command-line Python R Parameter Description --per-float-feature-quantization A semicolon separated list of quantization descriptions. Format: FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method] Examples: --per-float-feature-quantization 0:border_count=1024 In this example, the feature indexed 0 has 1024 borders. --per-float-feature-quantization 0:border_count=1024;1:border_count=1024 In this example, features indexed 0 and 1 have 1024 borders. Parameter Description per_float_feature_quantization The quantization description for the specified feature or list of features. Description format for a single feature: FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method] Examples: per_float_feature_quantization='0:border_count=1024' In this example, the feature indexed 0 has 1024 borders. per_float_feature_quantization=[ '0:border_count=1024' , '1:border_count=1024' ] In this example, features indexed 0 and 1 have 1024 borders. Parameter Description per_float_feature_quantization The quantization description for the specified feature or list of features. Description format for a single feature: FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method] Examples: per_float_feature_quantization = '0:border_count=1024' ) In this example, the feature indexed 0 has 1024 borders. per_float_feature_quantization = c ( '0:border_count=1024' , '1:border_count=1024' In this example, features indexed 0 and 1 have 1024 borders. Methods for hyperparameter search The Python package provides Grid and Randomized search methods for searching optimal parameter values for training the model with the given dataset. Parameters Class Method Description CatBoost grid_search A simple grid search over specified parameter values for a model. CatBoost randomized_search A simple randomized search on hyperparameters. CatBoostClassifier grid_search A simple grid search over specified parameter values for a model. CatBoostClassifier randomized_search A simple randomized search on hyperparameters. CatBoostRegressor grid_search A simple grid search over specified parameter values for a model. CatBoostRegressor randomized_search A simple randomized search on hyperparameters. Methods for hyperparameter search by optuna Optuna is a famous hyperparameter optimization framework. Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Catboost supports to stop unpromising trial of hyperparameter by callbacking after iteration functionality. Pull Request The following is an optuna example that demonstrates a pruner for CatBoost. Example

Markdown

[![Logo icon](https://yastatic.net/s3/locdoc/daas-static/catboost/71b237a322eec6f2889af0dae2a9c549.svg)](https://catboost.ai/ "CatBoost") - Installation - [Overview](https://catboost.ai/docs/en/concepts/en/concepts/installation) - Python package installation - CatBoost for Apache Spark installation - R package installation - Command-line version binary - Build from source - Key Features - Training parameters - Python package - CatBoost for Apache Spark - R package - Command-line version - Applying models - Objectives and metrics - Model analysis - Data format description - [Parameter tuning](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning) - [Speeding up the training](https://catboost.ai/docs/en/concepts/en/concepts/speed-up-training) - Data visualization - Algorithm details - [FAQ](https://catboost.ai/docs/en/concepts/en/concepts/faq) - Educational materials - [Development and contributions](https://catboost.ai/docs/en/concepts/en/concepts/development-and-contributions) - [Contacts](https://catboost.ai/docs/en/concepts/en/concepts/contacts) Parameter tuning ## In this article: - [One-hot encoding](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#one-hot-enc) - [Number of trees](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#trees-number) - [Learning rate](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#learning-rate) - [Tree depth](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#tree-depth) - [L2 regularization](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#l2-reg) - [Random strength](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#rand-str) - [Bagging temperature](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#bagg-temp) - [Border count](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#border-count) - [Internal dataset order](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#internal-dataset-order) - [Tree growing policy](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#tree-growing-policy) - [Golden features](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#golden-features) - [Methods for hyperparameter search](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#defining-optimal-parameter-values) - [Methods for hyperparameter search by optuna](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#methods-for-hyperparameter-search-by-optuna) # Parameter tuning - [One-hot encoding](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#one-hot-enc) - [Number of trees](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#trees-number) - [Learning rate](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#learning-rate) - [Tree depth](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#tree-depth) - [L2 regularization](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#l2-reg) - [Random strength](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#rand-str) - [Bagging temperature](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#bagg-temp) - [Border count](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#border-count) - [Internal dataset order](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#internal-dataset-order) - [Tree growing policy](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#tree-growing-policy) - [Golden features](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#golden-features) - [Methods for hyperparameter search](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#defining-optimal-parameter-values) - [Methods for hyperparameter search by optuna](https://catboost.ai/docs/en/concepts/en/concepts/parameter-tuning#methods-for-hyperparameter-search-by-optuna) CatBoost provides a flexible interface for parameter tuning and can be configured to suit different tasks. This section contains some tips on the possible parameter settings. ## One-hot encoding Warning Do not use one-hot encoding during preprocessing. This affects both the training speed and the resulting quality. Sometimes when categorical features don't have a lot of values, one-hot encoding works well. Usually one-hot encoding does not significantly improve the quality of the model. But if it is required, use the inbuilt parameters instead of preprocessing the dataset. Parameters **Command-line version parameters:** `--one-hot-max-size` **Python parameters:** `one_hot_max_size` **R parameters:** `one_hot_max_size` #### Description Use one-hot encoding for all categorical features with a number of different values less than or equal to the given parameter value. Ctrs are not calculated for such features. **Default value** The default value depends on various conditions: - N/A if training is performed on CPU in Pairwise scoring mode Read more about Pairwise scoring The following loss functions use Pairwise scoring: - YetiRankPairwise - PairLogitPairwise - QueryCrossEntropy Pairwise scoring is slightly different from regular training on pairs, since pairs are generated only internally during the training for the corresponding metrics. One-hot encoding is not available for these loss functions. - 255 if training is performed on GPU and the selected Ctr types require target data that is not available during the training - 10 if training is performed in [Ranking](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions-ranking) mode - 2 if none of the conditions above is met ## Number of trees It is recommended to check that there is no obvious underfitting or overfitting before tuning any other parameters. In order to do this it is necessary to analyze the metric value on the validation dataset and select the appropriate number of iterations. This can be done by setting the number of [iterations](https://catboost.ai/docs/en/concepts/en/references/training-parameters/common#iterations) to a large value, using the [overfitting detector](https://catboost.ai/docs/en/concepts/en/concepts/overfitting-detector) parameters and turning the [use best model](https://catboost.ai/docs/en/concepts/en/references/training-parameters/common#use_best_model) options on. In this case the resulting model contains only the first `k` best iterations, where `k` is the iteration with the best loss value on the validation dataset. Also, the metric for choosing the best model may differ from the one used for optimizing the objective value. For example, it is possible to set the optimized function to Logloss and use the AUC function for the overfitting detector. To do so, use the [evaluation metric](https://catboost.ai/docs/en/concepts/en/references/training-parameters/common#eval_metric) parameter. Parameters **Command-line version parameters:** `-i`, `--iterations` **Python parameters:** `--iterations` **R parameters:** `--iterations` #### Description The maximum number of trees that can be built when solving machine learning problems. When using other parameters that limit the number of iterations, the final number of trees may be less than the number specified in this parameter. **Command-line version parameters:** `--use-best-model` **Python parameters:** `--use-best-model` **R parameters:** `--use-best-model` #### Description If this parameter is set, the number of trees that are saved in the resulting model is defined as follows: 1. Build the number of trees defined by the training parameters. 2. Use the validation dataset to identify the iteration with the optimal value of the metric specified in `--eval-metric` (`--eval-metric`). No trees are saved after this iteration. This option requires a validation dataset to be provided. **Command-line version parameters:** `--eval-metric` **Python parameters:** `--eval-metric` **R parameters:** `--eval-metric` #### Description The metric used for overfitting detection (if enabled) and best model selection (if enabled). Some metrics support optional parameters (see the [Objectives and metrics](https://catboost.ai/docs/en/concepts/en/concepts/loss-functions) section for details on each metric). Format: ``` <Metric>[:<parameter 1>=<value>;..;<parameter N>=<value>] ``` [Supported metrics](https://catboost.ai/docs/en/concepts/en/references/eval-metric__supported-metrics) Examples: ``` R2 ``` ``` Quantile:alpha=0.3 ``` **Command-line version parameters:** **Overfitting detection settings** **Command-line version parameters:** `--od-type` **Python parameters:** `od_type` **R parameters:** `od_type` #### Description The type of the overfitting detector to use. Possible values: - IncToDec - Iter **Command-line version parameters:** `--od-pval` **Python parameters:** `od_pval` **R parameters:** `od_pval` #### Description The threshold for the IncToDec [overfitting detector](https://catboost.ai/docs/en/concepts/en/concepts/overfitting-detector) type. The training is stopped when the specified value is reached. Requires that a validation dataset was input. For best results, it is recommended to set a value in the range \[ 1 0 – 10 ; 1 0 − 2 \] \[10^{–10}; 10^{-2}\] \[10–10;10−2\]. The larger the value, the earlier overfitting is detected. Alert Do not use this parameter with the Iter overfitting detector type. **Command-line version parameters:** `--od-wait` **Python parameters:** `od_wait` **R parameters:** `od_wait` #### Description The number of iterations to continue the training after the iteration with the optimal metric value. The purpose of this parameter differs depending on the selected overfitting detector type: - IncToDec — Ignore the overfitting detector when the threshold is reached and continue learning for the specified number of iterations after the iteration with the optimal metric value. - Iter — Consider the model overfitted and stop training after the specified number of iterations since the iteration with the optimal metric value. ## Learning rate This setting is used for reducing the gradient step. It affects the overall time of training: the smaller the value, the more iterations are required for training. Choose the value based on the performance expectations. By default, the learning rate is defined automatically based on the dataset properties and the number of iterations. The automatically defined value should be close to the optimal one. Possible ways of adjusting the learning rate depending on the overfitting results: - There is no overfitting on the last iterations of training (the training does not converge) — increase the learning rate. - Overfitting is detected — decrease the learning rate. Parameters **Command-line version parameters:** `-w`, `--learning-rate` **Python parameters:** `learning_rate` **R parameters:** `learning_rate` #### Description The learning rate. Used for reducing the gradient step. ## Tree depth In most cases, the optimal depth ranges from 4 to 10. Values in the range from 6 to 10 are recommended. Note The maximum depth of the trees is limited to 8 for pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) when the training is performed on GPU. Parameters **Command-line version parameters:** `-n`, `--depth` **Python parameters:** `depth` **R parameters:** `depth` #### Description Depth of the trees. The range of supported values depends on the processing unit type and the type of the selected loss function: - CPU — Any integer up to 16. - GPU — Any integer up to 8 pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) and up to 16 for all other loss functions. ## L2 regularization Try different values for the regularizer to find the best possible. Parameters **Command-line version parameters:** `--l2-leaf-reg` **Python parameters:** `l2_leaf_reg` **R parameters:** `l2_leaf_reg` #### Description Coefficient at the L2 regularization term of the cost function. Any positive value is allowed. ## Random strength Try setting different values for the `random_strength` parameter. Parameters **Command-line version parameters:** `--random-strength` **Python parameters:** `random_strength` **R parameters:** `random_strength` #### Description The amount of randomness to use for scoring splits when the tree structure is selected. Use this parameter to avoid overfitting the model. The value of this parameter is used when selecting splits. On every iteration each possible split gets a score (for example, the score indicates how much adding this split will improve the loss function for the training dataset). The split with the highest score is selected. The scores have no randomness. A normally distributed random variable is added to the score of the feature. It has a zero mean and a variance that decreases during the training. The value of this parameter is the multiplier of the variance. This parameter is not supported for the following loss functions: - QueryCrossEntropy - YetiRankPairwise - PairLogitPairwise ## Bagging temperature Try setting different values for the `bagging_temperature` parameter Parameters **Command-line version parameters:** `--bagging-temperature` **Python parameters:** `bagging_temperature` **R parameters:** `bagging_temperature` #### Description Defines the settings of the Bayesian bootstrap. It is used by default in classification and regression modes. Use the Bayesian bootstrap to assign random weights to objects. The weights are sampled from exponential distribution if the value of this parameter is set to "1". All weights are equal to 1 if the value of this parameter is set to "0". Possible values are in the range \[ 0 ; inf ⁡ ) \[0; \\inf) \[0;inf). The higher the value the more aggressive the bagging is. This parameter can be used if the selected bootstrap type is Bayesian. ## Border count The number of splits for numerical features. The default value depends on the processing unit type and other parameters: - CPU: 254 - GPU in PairLogitPairwise and YetiRankPairwise modes: 32 - GPU in all other modes: 128 The value of this parameter significantly impacts the speed of training on GPU. The smaller the value, the faster the training is performed (refer to the [Number of splits for numerical features](https://catboost.ai/docs/en/concepts/en/concepts/speed-up-training) section for details). 128 splits are enough for many datasets. However, try to set the value of this parameter to 254 when training on GPU if the best possible quality is required. The value of this parameter does not significantly impact the speed of training on CPU. Try to set it to 254 for the best possible quality. Parameters **Command-line version parameters:** `-x`, `--border-count` **Python parameters:** `border_count` *Alias:*`max_bin` **R parameters:** `border_count` #### Description Recommended values are up to 255. Larger values slow down the training. The number of splits for numerical features. Allowed values are integers from 1 to 65535 inclusively. ## Internal dataset order Use this option if the objects in your dataset are given in the required order. In this case, random permutations are not performed during the [Transforming categorical features to numerical features](https://catboost.ai/docs/en/concepts/en/concepts/algorithm-main-stages_cat-to-numberic) and [Choosing the tree structure](https://catboost.ai/docs/en/concepts/en/concepts/algorithm-main-stages_choose-tree-structure) stages. Parameters **Command-line version parameters:** `--has-time` **Python parameters:** `--has-time` **R parameters:** `--has-time` #### Description Use the order of objects in the input data (do not perform random permutations during the [Transforming categorical features to numerical features](https://catboost.ai/docs/en/concepts/en/concepts/algorithm-main-stages_cat-to-numberic) and [Choosing the tree structure](https://catboost.ai/docs/en/concepts/en/concepts/algorithm-main-stages_choose-tree-structure) stages). The Timestamp column type is used to determine the order of objects if specified in the [input data](https://catboost.ai/docs/en/concepts/en/concepts/input-data). ## Tree growing policy By default, CatBoost uses symmetric trees, which are built if the growing policy is set to SymmetricTree. Such trees are built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric. Symmetric trees have a very good prediction speed (roughly 10 times faster than non-symmetric trees) and give better quality in many cases. However, in some cases, other tree growing strategies can give better results than growing symmetric trees. Try to analyze the results obtained with different growing trees strategies. Specifics: Symmetric trees, that are used by default, can be applied much faster (up to 10 times faster). Parameters **Command-line version parameters:** `--grow-policy` **Python parameters:** `grow_policy` **R parameters:** `grow_policy` #### Description The tree growing policy. Defines how to perform greedy tree construction. Possible values: - SymmetricTree — A tree is built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric. - Depthwise — A tree is built level by level until the specified depth is reached. On each iteration, all non-terminal leaves from the last tree level are split. Each leaf is split by condition with the best loss improvement. Note Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm. - Lossguide — A tree is built leaf by leaf until the specified maximum number of leaves is reached. On each iteration, non-terminal leaf with the best loss improvement is split. Note Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm. **Command-line version parameters:** `--min-data-in-leaf` **Python parameters:** `min_data_in_leaf` *Alias:*`min_child_samples` **R parameters:** `min_data_in_leaf` #### Description The minimum number of training samples in a leaf. CatBoost does not search for new splits in leaves with samples count less than the specified value. Can be used only with the Lossguide and Depthwise growing policies. **Command-line version parameters:** `--max-leaves` **Python parameters:** `max_leaves` *Alias:*`num_leaves` **R parameters:** `max_leaves` #### Description The maximum number of leafs in the resulting tree. Can be used only with the Lossguide growing policy. Note It is not recommended to use values greater than 64, since it can significantly slow down the training process. ## Golden features If the dataset has a feature, which is a strong predictor of the result, the pre-quantisation of this feature may decrease the information that the model can get from it. It is recommended to use an increased number of borders (1024) for this feature. Note An increased number of borders should not be set for all features. It is recommended to set it for one or two golden features. Command-line Python R | Parameter | Description | |---|---| | `--per-float-feature-quantization` | A semicolon separated list of quantization descriptions. Format: `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` | Examples: - ``` --per-float-feature-quantization 0:border_count=1024 ``` In this example, the feature indexed 0 has 1024 borders. - ``` --per-float-feature-quantization 0:border_count=1024;1:border_count=1024 ``` In this example, features indexed 0 and 1 have 1024 borders. | Parameter | Description | |---|---| | `per_float_feature_quantization` | The quantization description for the specified feature or list of features. Description format for a single feature: `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` | Examples: - ``` per_float_feature_quantization='0:border_count=1024' ``` In this example, the feature indexed 0 has 1024 borders. - ``` per_float_feature_quantization=['0:border_count=1024', '1:border_count=1024'] ``` In this example, features indexed 0 and 1 have 1024 borders. | Parameter | Description | |---|---| | `per_float_feature_quantization` | The quantization description for the specified feature or list of features. Description format for a single feature: `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` | Examples: - ``` per_float_feature_quantization = '0:border_count=1024') ``` In this example, the feature indexed 0 has 1024 borders. - ``` per_float_feature_quantization = c('0:border_count=1024', '1:border_count=1024' ``` In this example, features indexed 0 and 1 have 1024 borders. ## Methods for hyperparameter search The Python package provides Grid and Randomized search methods for searching optimal parameter values for training the model with the given dataset. Parameters | Class | Method | Description | |---|---|---| | [CatBoost](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboost) | [grid\_search](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboost_grid_search) | A simple grid search over specified parameter values for a model. | | [CatBoost](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboost) | [randomized\_search](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboost_randomized_search) | A simple randomized search on hyperparameters. | | [CatBoostClassifier](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboostclassifier) | [grid\_search](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboostclassifier_grid_search) | A simple grid search over specified parameter values for a model. | | [CatBoostClassifier](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboostclassifier) | [randomized\_search](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboostclassifier_randomized_search) | A simple randomized search on hyperparameters. | | [CatBoostRegressor](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboostregressor) | [grid\_search](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboostregressor_grid_search) | A simple grid search over specified parameter values for a model. | | [CatBoostRegressor](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboostregressor) | [randomized\_search](https://catboost.ai/docs/en/concepts/en/concepts/python-reference_catboostregressor_randomized_search) | A simple randomized search on hyperparameters. | ## Methods for hyperparameter search by optuna Optuna is a famous hyperparameter optimization framework. Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Catboost supports to stop unpromising trial of hyperparameter by callbacking after iteration functionality. [Pull Request](https://github.com/catboost/catboost/pull/1697/files#diff-ccca44461ac6b094190f29fec157a227996e226ea483213680dd0a152cd412eaR9679) The following is an optuna example that demonstrates a pruner for CatBoost. [Example](https://github.com/optuna/optuna-examples/blob/main/catboost/catboost_pruning.py) ### Was the article helpful? Yes No Previous [ROC curve points](https://catboost.ai/docs/en/concepts/en/concepts/output-data_roc-curve-points) Next [Speeding up the training](https://catboost.ai/docs/en/concepts/en/concepts/speed-up-training) ![](https://mc.yandex.ru/watch/60763294)

Readable Markdown

CatBoost provides a flexible interface for parameter tuning and can be configured to suit different tasks. This section contains some tips on the possible parameter settings. ## One-hot encoding Warning Do not use one-hot encoding during preprocessing. This affects both the training speed and the resulting quality. Sometimes when categorical features don't have a lot of values, one-hot encoding works well. Usually one-hot encoding does not significantly improve the quality of the model. But if it is required, use the inbuilt parameters instead of preprocessing the dataset. Parameters **Command-line version parameters:** `--one-hot-max-size` **Python parameters:** `one_hot_max_size` **R parameters:** `one_hot_max_size` #### Description Use one-hot encoding for all categorical features with a number of different values less than or equal to the given parameter value. Ctrs are not calculated for such features. **Default value** The default value depends on various conditions: - N/A if training is performed on CPU in Pairwise scoring mode Read more about Pairwise scoring The following loss functions use Pairwise scoring: - YetiRankPairwise - PairLogitPairwise - QueryCrossEntropy Pairwise scoring is slightly different from regular training on pairs, since pairs are generated only internally during the training for the corresponding metrics. One-hot encoding is not available for these loss functions. - 255 if training is performed on GPU and the selected Ctr types require target data that is not available during the training - 10 if training is performed in [Ranking](https://catboost.ai/docs/en/concepts/loss-functions-ranking) mode - 2 if none of the conditions above is met ## Number of trees It is recommended to check that there is no obvious underfitting or overfitting before tuning any other parameters. In order to do this it is necessary to analyze the metric value on the validation dataset and select the appropriate number of iterations. This can be done by setting the number of [iterations](https://catboost.ai/docs/en/references/training-parameters/common#iterations) to a large value, using the [overfitting detector](https://catboost.ai/docs/en/concepts/overfitting-detector) parameters and turning the [use best model](https://catboost.ai/docs/en/references/training-parameters/common#use_best_model) options on. In this case the resulting model contains only the first `k` best iterations, where `k` is the iteration with the best loss value on the validation dataset. Also, the metric for choosing the best model may differ from the one used for optimizing the objective value. For example, it is possible to set the optimized function to Logloss and use the AUC function for the overfitting detector. To do so, use the [evaluation metric](https://catboost.ai/docs/en/references/training-parameters/common#eval_metric) parameter. Parameters **Command-line version parameters:** `-i`, `--iterations` **Python parameters:** `--iterations` **R parameters:** `--iterations` #### Description The maximum number of trees that can be built when solving machine learning problems. When using other parameters that limit the number of iterations, the final number of trees may be less than the number specified in this parameter. **Command-line version parameters:** `--use-best-model` **Python parameters:** `--use-best-model` **R parameters:** `--use-best-model` #### Description If this parameter is set, the number of trees that are saved in the resulting model is defined as follows: 1. Build the number of trees defined by the training parameters. 2. Use the validation dataset to identify the iteration with the optimal value of the metric specified in `--eval-metric` (`--eval-metric`). No trees are saved after this iteration. This option requires a validation dataset to be provided. **Command-line version parameters:** `--eval-metric` **Python parameters:** `--eval-metric` **R parameters:** `--eval-metric` #### Description The metric used for overfitting detection (if enabled) and best model selection (if enabled). Some metrics support optional parameters (see the [Objectives and metrics](https://catboost.ai/docs/en/concepts/loss-functions) section for details on each metric). Format: ``` <Metric>[:<parameter 1>=<value>;..;<parameter N>=<value>] ``` [Supported metrics](https://catboost.ai/docs/en/references/eval-metric__supported-metrics) Examples: ``` R2 ``` ``` Quantile:alpha=0.3 ``` **Command-line version parameters:** **Overfitting detection settings** **Command-line version parameters:** `--od-type` **Python parameters:** `od_type` **R parameters:** `od_type` #### Description The type of the overfitting detector to use. Possible values: - IncToDec - Iter **Command-line version parameters:** `--od-pval` **Python parameters:** `od_pval` **R parameters:** `od_pval` #### Description The threshold for the IncToDec [overfitting detector](https://catboost.ai/docs/en/concepts/overfitting-detector) type. The training is stopped when the specified value is reached. Requires that a validation dataset was input. For best results, it is recommended to set a value in the range \[ 1 0 – 10 ; 1 0 − 2 \] \[10^{–10}; 10^{-2}\]. The larger the value, the earlier overfitting is detected. Alert Do not use this parameter with the Iter overfitting detector type. **Command-line version parameters:** `--od-wait` **Python parameters:** `od_wait` **R parameters:** `od_wait` #### Description The number of iterations to continue the training after the iteration with the optimal metric value. The purpose of this parameter differs depending on the selected overfitting detector type: - IncToDec — Ignore the overfitting detector when the threshold is reached and continue learning for the specified number of iterations after the iteration with the optimal metric value. - Iter — Consider the model overfitted and stop training after the specified number of iterations since the iteration with the optimal metric value. ## Learning rate This setting is used for reducing the gradient step. It affects the overall time of training: the smaller the value, the more iterations are required for training. Choose the value based on the performance expectations. By default, the learning rate is defined automatically based on the dataset properties and the number of iterations. The automatically defined value should be close to the optimal one. Possible ways of adjusting the learning rate depending on the overfitting results: - There is no overfitting on the last iterations of training (the training does not converge) — increase the learning rate. - Overfitting is detected — decrease the learning rate. Parameters **Command-line version parameters:** `-w`, `--learning-rate` **Python parameters:** `learning_rate` **R parameters:** `learning_rate` #### Description The learning rate. Used for reducing the gradient step. ## Tree depth In most cases, the optimal depth ranges from 4 to 10. Values in the range from 6 to 10 are recommended. Note The maximum depth of the trees is limited to 8 for pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) when the training is performed on GPU. Parameters **Command-line version parameters:** `-n`, `--depth` **Python parameters:** `depth` **R parameters:** `depth` #### Description Depth of the trees. The range of supported values depends on the processing unit type and the type of the selected loss function: - CPU — Any integer up to 16. - GPU — Any integer up to 8 pairwise modes (YetiRank, PairLogitPairwise and QueryCrossEntropy) and up to 16 for all other loss functions. ## L2 regularization Try different values for the regularizer to find the best possible. Parameters **Command-line version parameters:** `--l2-leaf-reg` **Python parameters:** `l2_leaf_reg` **R parameters:** `l2_leaf_reg` #### Description Coefficient at the L2 regularization term of the cost function. Any positive value is allowed. ## Random strength Try setting different values for the `random_strength` parameter. Parameters **Command-line version parameters:** `--random-strength` **Python parameters:** `random_strength` **R parameters:** `random_strength` #### Description The amount of randomness to use for scoring splits when the tree structure is selected. Use this parameter to avoid overfitting the model. The value of this parameter is used when selecting splits. On every iteration each possible split gets a score (for example, the score indicates how much adding this split will improve the loss function for the training dataset). The split with the highest score is selected. The scores have no randomness. A normally distributed random variable is added to the score of the feature. It has a zero mean and a variance that decreases during the training. The value of this parameter is the multiplier of the variance. This parameter is not supported for the following loss functions: - QueryCrossEntropy - YetiRankPairwise - PairLogitPairwise ## Bagging temperature Try setting different values for the `bagging_temperature` parameter Parameters **Command-line version parameters:** `--bagging-temperature` **Python parameters:** `bagging_temperature` **R parameters:** `bagging_temperature` #### Description Defines the settings of the Bayesian bootstrap. It is used by default in classification and regression modes. Use the Bayesian bootstrap to assign random weights to objects. The weights are sampled from exponential distribution if the value of this parameter is set to "1". All weights are equal to 1 if the value of this parameter is set to "0". Possible values are in the range \[ 0 ; inf ⁡ ) \[0; \\inf). The higher the value the more aggressive the bagging is. This parameter can be used if the selected bootstrap type is Bayesian. ## Border count The number of splits for numerical features. The default value depends on the processing unit type and other parameters: - CPU: 254 - GPU in PairLogitPairwise and YetiRankPairwise modes: 32 - GPU in all other modes: 128 The value of this parameter significantly impacts the speed of training on GPU. The smaller the value, the faster the training is performed (refer to the [Number of splits for numerical features](https://catboost.ai/docs/en/concepts/speed-up-training) section for details). 128 splits are enough for many datasets. However, try to set the value of this parameter to 254 when training on GPU if the best possible quality is required. The value of this parameter does not significantly impact the speed of training on CPU. Try to set it to 254 for the best possible quality. Parameters **Command-line version parameters:** `-x`, `--border-count` **Python parameters:** `border_count` *Alias:*`max_bin` **R parameters:** `border_count` #### Description Recommended values are up to 255. Larger values slow down the training. The number of splits for numerical features. Allowed values are integers from 1 to 65535 inclusively. ## Internal dataset order Use this option if the objects in your dataset are given in the required order. In this case, random permutations are not performed during the [Transforming categorical features to numerical features](https://catboost.ai/docs/en/concepts/algorithm-main-stages_cat-to-numberic) and [Choosing the tree structure](https://catboost.ai/docs/en/concepts/algorithm-main-stages_choose-tree-structure) stages. Parameters **Command-line version parameters:** `--has-time` **Python parameters:** `--has-time` **R parameters:** `--has-time` #### Description Use the order of objects in the input data (do not perform random permutations during the [Transforming categorical features to numerical features](https://catboost.ai/docs/en/concepts/algorithm-main-stages_cat-to-numberic) and [Choosing the tree structure](https://catboost.ai/docs/en/concepts/algorithm-main-stages_choose-tree-structure) stages). The Timestamp column type is used to determine the order of objects if specified in the [input data](https://catboost.ai/docs/en/concepts/input-data). ## Tree growing policy By default, CatBoost uses symmetric trees, which are built if the growing policy is set to SymmetricTree. Such trees are built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric. Symmetric trees have a very good prediction speed (roughly 10 times faster than non-symmetric trees) and give better quality in many cases. However, in some cases, other tree growing strategies can give better results than growing symmetric trees. Try to analyze the results obtained with different growing trees strategies. Specifics: Symmetric trees, that are used by default, can be applied much faster (up to 10 times faster). Parameters **Command-line version parameters:** `--grow-policy` **Python parameters:** `grow_policy` **R parameters:** `grow_policy` #### Description The tree growing policy. Defines how to perform greedy tree construction. Possible values: - SymmetricTree — A tree is built level by level until the specified depth is reached. On each iteration, all leaves from the last tree level are split with the same condition. The resulting tree structure is always symmetric. - Depthwise — A tree is built level by level until the specified depth is reached. On each iteration, all non-terminal leaves from the last tree level are split. Each leaf is split by condition with the best loss improvement. Note Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm. - Lossguide — A tree is built leaf by leaf until the specified maximum number of leaves is reached. On each iteration, non-terminal leaf with the best loss improvement is split. Note Models with this growing policy can not be analyzed using the PredictionDiff feature importance and can be exported only to json and cbm. **Command-line version parameters:** `--min-data-in-leaf` **Python parameters:** `min_data_in_leaf` *Alias:*`min_child_samples` **R parameters:** `min_data_in_leaf` #### Description The minimum number of training samples in a leaf. CatBoost does not search for new splits in leaves with samples count less than the specified value. Can be used only with the Lossguide and Depthwise growing policies. **Command-line version parameters:** `--max-leaves` **Python parameters:** `max_leaves` *Alias:*`num_leaves` **R parameters:** `max_leaves` #### Description The maximum number of leafs in the resulting tree. Can be used only with the Lossguide growing policy. Note It is not recommended to use values greater than 64, since it can significantly slow down the training process. ## Golden features If the dataset has a feature, which is a strong predictor of the result, the pre-quantisation of this feature may decrease the information that the model can get from it. It is recommended to use an increased number of borders (1024) for this feature. Note An increased number of borders should not be set for all features. It is recommended to set it for one or two golden features. Command-line Python R | Parameter | Description | |---|---| | `--per-float-feature-quantization` | A semicolon separated list of quantization descriptions. Format: `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` | Examples: - ``` --per-float-feature-quantization 0:border_count=1024 ``` In this example, the feature indexed 0 has 1024 borders. - ``` --per-float-feature-quantization 0:border_count=1024;1:border_count=1024 ``` In this example, features indexed 0 and 1 have 1024 borders. | Parameter | Description | |---|---| | `per_float_feature_quantization` | The quantization description for the specified feature or list of features. Description format for a single feature: `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` | Examples: - ``` per_float_feature_quantization='0:border_count=1024' ``` In this example, the feature indexed 0 has 1024 borders. - ``` per_float_feature_quantization=['0:border_count=1024', '1:border_count=1024'] ``` In this example, features indexed 0 and 1 have 1024 borders. | Parameter | Description | |---|---| | `per_float_feature_quantization` | The quantization description for the specified feature or list of features. Description format for a single feature: `FeatureId[:border_count=BorderCount][:nan_mode=BorderType][:border_type=border_selection_method]` | Examples: - ``` per_float_feature_quantization = '0:border_count=1024') ``` In this example, the feature indexed 0 has 1024 borders. - ``` per_float_feature_quantization = c('0:border_count=1024', '1:border_count=1024' ``` In this example, features indexed 0 and 1 have 1024 borders. ## Methods for hyperparameter search The Python package provides Grid and Randomized search methods for searching optimal parameter values for training the model with the given dataset. Parameters | Class | Method | Description | |---|---|---| | [CatBoost](https://catboost.ai/docs/en/concepts/python-reference_catboost) | [grid\_search](https://catboost.ai/docs/en/concepts/python-reference_catboost_grid_search) | A simple grid search over specified parameter values for a model. | | [CatBoost](https://catboost.ai/docs/en/concepts/python-reference_catboost) | [randomized\_search](https://catboost.ai/docs/en/concepts/python-reference_catboost_randomized_search) | A simple randomized search on hyperparameters. | | [CatBoostClassifier](https://catboost.ai/docs/en/concepts/python-reference_catboostclassifier) | [grid\_search](https://catboost.ai/docs/en/concepts/python-reference_catboostclassifier_grid_search) | A simple grid search over specified parameter values for a model. | | [CatBoostClassifier](https://catboost.ai/docs/en/concepts/python-reference_catboostclassifier) | [randomized\_search](https://catboost.ai/docs/en/concepts/python-reference_catboostclassifier_randomized_search) | A simple randomized search on hyperparameters. | | [CatBoostRegressor](https://catboost.ai/docs/en/concepts/python-reference_catboostregressor) | [grid\_search](https://catboost.ai/docs/en/concepts/python-reference_catboostregressor_grid_search) | A simple grid search over specified parameter values for a model. | | [CatBoostRegressor](https://catboost.ai/docs/en/concepts/python-reference_catboostregressor) | [randomized\_search](https://catboost.ai/docs/en/concepts/python-reference_catboostregressor_randomized_search) | A simple randomized search on hyperparameters. | ## Methods for hyperparameter search by optuna Optuna is a famous hyperparameter optimization framework. Optuna enables efficient hyperparameter optimization by adopting state-of-the-art algorithms for sampling hyperparameters and pruning efficiently unpromising trials. Catboost supports to stop unpromising trial of hyperparameter by callbacking after iteration functionality. [Pull Request](https://github.com/catboost/catboost/pull/1697/files#diff-ccca44461ac6b094190f29fec157a227996e226ea483213680dd0a152cd412eaR9679) The following is an optuna example that demonstrates a pruner for CatBoost. [Example](https://github.com/optuna/optuna-examples/blob/main/catboost/catboost_pruning.py)

ML Classification

ML Categories

/Computers_and_Electronics		96.9%
/Computers_and_Electronics/Software		88.0%
/Computers_and_Electronics/Software/Software_Utilities		76.0%

Raw JSON

{
    "/Computers_and_Electronics": 969,
    "/Computers_and_Electronics/Software": 880,
    "/Computers_and_Electronics/Software/Software_Utilities": 760
}

ML Page Types

/Article		50.1%
/Article/Tutorial_or_Guide		48.6%

Raw JSON

{
    "/Article": 501,
    "/Article/Tutorial_or_Guide": 486
}

ML Intent Types

Informational

98.5%

Raw JSON

{
    "Informational": 985
}

Content Metadata

Language

Author

null

Publish Time

not set

Original Publish Time

2024-11-18 16:12:15 (1 year ago)

Republished

Word Count (Total)

2,527

Word Count (Content)

2,354

Links

External Links

Internal Links

Technical SEO

Meta Nofollow

Meta Noarchive

JS Rendered

Redirect Target

null

Performance

Download Time (ms)

947

TTFB (ms)

723

Download Size (bytes)

266,835

Shard

169 (laksa)

Root Hash

17435841955170310369

Unparsed URL

ai,catboost!/docs/en/concepts/parameter-tuning s443