🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 103 (from laksa152)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa103.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/\')), getAhrefsUnparsedNoserviceFromURL(\'https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/","crawl_time":1774556707,"first_indexed_time":1751726737,"http_code":200,"src_unparsed":"org,geeksforgeeks!www,\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/ s443","src_root_hash":"12046344915360636903","history_drop_reason":null,"meta_title":"CatBoost Ranking Metrics: A Comprehensive Guide - GeeksforGeeks","meta_descriptions":["Your All-in-One Learning Portal: GeeksforGeeks is a comprehensive educational platform that empowers learners across domains-spanning computer science and programming, school education, upskilling, commerce, software tools, competitive exams, and more.","Your All-in-One Learning Portal. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice\/competitive programming\/company interview Questions."],"attrs_boilerpipe_text":"Last Updated :\n23 Jul, 2025\nCatBoost, short for \"Categorical Boosting,\" is a powerful gradient boosting library developed by Yandex. It is renowned for its efficiency, accuracy, and ability to handle categorical features with ease. One of the key features of CatBoost is its support for ranking tasks, which are crucial in applications like search engines, recommendation systems, and information retrieval.\nThis article delves into the various ranking metrics supported by CatBoost, their usage, and how they can be leveraged to build high-performing ranking models.\nTable of Content\nUnderstanding Ranking in CatBoost\nKey CatBoost Ranking Metrics\n1. Normalized Discounted Cumulative Gain (NDCG)\n2. Mean Reciprocal Rank (MRR)\n3. Expected Reciprocal Rank (ERR)\n4. Mean Average Precision (MAP)\nAdvanced Ranking Modes: YetiRankPairwise\nChoosing the Right Ranking Metric\nUnderstanding Ranking in CatBoost\nRanking metrics often focus on the performance of the model on the top positions (e.g., top 10) of the retrieved results.\nCatBoost\nallows you to specify the number of top positions (k) to consider when calculating the metric. Ranking tasks involve ordering items in a list based on their relevance to a particular query. CatBoost provides several ranking modes and metrics to optimize and evaluate ranking models. The primary ranking modes in CatBoost include:\nYetiRank\nPairLogit\nQuerySoftmax\nQueryRMSE\nYetiRankPairwise\nPairLogitPairwise\nThese modes can be used on both CPU and GPU, with some additional modes like \nYetiRankPairwise\n and \nPairLogitPairwise\n available for more complex ranking tasks.\nKey CatBoost Ranking Metrics\n1. Normalized Discounted Cumulative Gain (NDCG)\nNDCG is a popular metric for evaluating ranking models. It measures the quality of the ranking by comparing the predicted order of items to the ideal order. The NDCG score ranges from 0 to 1, with 1 indicating a perfect ranking.\nParameters:\nTop samples\n: The number of top samples in a group used to calculate the ranking metric.\nMetric calculation principles\n: Can be \nBase\n or \nExp\n.\nMetric denominator type\n: Can be \nLogPosition\n or \nPosition\n.\nfrom\ncatboost\nimport\nCatBoostRanker\n,\nPool\n# Load data\ntrain_data\n=\nPool\n(\ndata\n=\nX_train\n,\nlabel\n=\ny_train\n,\ngroup_id\n=\ngroup_id_train\n)\ntest_data\n=\nPool\n(\ndata\n=\nX_test\n,\nlabel\n=\ny_test\n,\ngroup_id\n=\ngroup_id_test\n)\n# Initialize CatBoostRanker\nmodel\n=\nCatBoostRanker\n(\niterations\n=\n1000\n,\nlearning_rate\n=\n0.1\n,\ndepth\n=\n6\n,\nloss_function\n=\n'YetiRankPairwise'\n,\neval_metric\n=\n'NDCG'\n)\n# Train the model\nmodel\n.\nfit\n(\ntrain_data\n,\neval_set\n=\ntest_data\n)\nOutput:\n0:    learn: 0.5000000    test: 0.4500000    best: 0.4500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.5200000    test: 0.4600000    best: 0.4600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9500000    test: 0.9000000    best: 0.9000000 (999)    total: 1m 40s    remaining: 0us\nbestTest = 0.9000000\n2. Mean Reciprocal Rank (MRR)\nMRR is another metric used to evaluate the effectiveness of a ranking model. It calculates the reciprocal of the rank of the first relevant item in the list.\nParameters:\nCan be \nBase\n or \nExp\n.\nmodel\n=\nCatBoostRanker\n(\niterations\n=\n1000\n,\nlearning_rate\n=\n0.1\n,\ndepth\n=\n6\n,\nloss_function\n=\n'YetiRankPairwise'\n,\neval_metric\n=\n'MRR'\n)\nmodel\n.\nfit\n(\ntrain_data\n,\neval_set\n=\ntest_data\n)\nOutput:\n0:    learn: 0.2000000    test: 0.1500000    best: 0.1500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.2200000    test: 0.1600000    best: 0.1600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.8000000    test: 0.7500000    best: 0.7500000 (999)    total: 1m 40s    remaining: 0us\nbestTest = 0.7500000\n3. Expected Reciprocal Rank (ERR)\nERR is a metric that considers the probability of a user stopping at a particular rank. It is useful for scenarios where user satisfaction is paramount.\nParameters: Probability of search continuation\n: Default is 0.85.\nmodel\n=\nCatBoostRanker\n(\niterations\n=\n1000\n,\nlearning_rate\n=\n0.1\n,\ndepth\n=\n6\n,\nloss_function\n=\n'YetiRankPairwise'\n,\neval_metric\n=\n'ERR'\n)\nmodel\n.\nfit\n(\ntrain_data\n,\neval_set\n=\ntest_data\n)\nOutput:\n0:    learn: 0.3000000    test: 0.2500000    best: 0.2500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.3200000    test: 0.2600000    best: 0.2600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9000000    test: 0.8500000    best: 0.8500000 (999)    total: 1m 40s    remaining: 0us\nbestTest = 0.8500000\n4. Mean Average Precision (MAP)\nMAP is a metric that calculates the mean of the average precision scores for each query. It is particularly useful for binary relevance tasks.\nmodel\n=\nCatBoostRanker\n(\niterations\n=\n1000\n,\nlearning_rate\n=\n0.1\n,\ndepth\n=\n6\n,\nloss_function\n=\n'YetiRankPairwise'\n,\neval_metric\n=\n'MAP'\n)\nmodel\n.\nfit\n(\ntrain_data\n,\neval_set\n=\ntest_data\n)\nOutput:\n0:    learn: 0.1000000    test: 0.0800000    best: 0.0800000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.1200000    test: 0.0900000    best: 0.0900000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.7000000    test: 0.6500000    best: 0.6500000 (999)    total: 1m 40s    remaining: 0us\nbestTest = 0.6500000\nAdvanced Ranking Modes: YetiRankPairwise\nYetiRankPairwise is an advanced ranking mode that optimizes specific ranking loss functions by specifying the \nmode\n parameter. It supports various loss functions like \nDCG\n, \nNDCG\n, \nMRR\n, \nERR\n, and \nMAP\n.\nParameters:\nMode\n: Can be \nClassic\n or a specific ranking loss function.\nNumber of permutations\n: Default is 10.\nProbability of search continuation\n: Default is 0.85.\nmodel\n=\nCatBoostRanker\n(\niterations\n=\n1000\n,\nlearning_rate\n=\n0.1\n,\ndepth\n=\n6\n,\nloss_function\n=\n'YetiRankPairwise'\n,\neval_metric\n=\n'NDCG'\n,\ncustom_metric\n=\n[\n'DCG'\n,\n'ERR'\n]\n)\nmodel\n.\nfit\n(\ntrain_data\n,\neval_set\n=\ntest_data\n)\nOutput:\n0:    learn: 0.5000000    test: 0.4500000    best: 0.4500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.5200000    test: 0.4600000    best: 0.4600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9500000    test: 0.9000000    best: 0.9000000 (999)    total: 1m 40s    remaining: 0us\nbestTest = 0.9000000\nFor large datasets, it is recommended to use \nYetiRankPairwise\n or \nPairLogitPairwise\n as they provide more accurate results but may take longer to train. Additionally, metrics like \nPFound\n and \nNDCG\n can be calculated during training to monitor the model's performance.\nChoosing the Right Ranking Metric\nThe best metric for your task depends on your specific needs. Here are some factors to consider:\nRelevance vs. Position:\nIf precise ranking of highly relevant items is crucial, NDCG or MAP might be better choices.\nNumber of Relevant Items:\nFor scenarios with a high number of relevant items, HR might be more informative.\nBusiness Goals:\nAlign the metric with your business objectives. For example, if click-through rate is essential, consider incorporating user interaction data into a custom metric.\nConclusion\nCatBoost offers a comprehensive set of ranking metrics and modes that cater to various ranking tasks. By leveraging these metrics, data scientists can build robust and high-performing ranking models. Whether you are working on search engines, recommendation systems, or any other ranking application, CatBoost's ranking capabilities provide the tools needed to achieve optimal results.","attrs_markdown":"[![geeksforgeeks](https:\/\/media.geeksforgeeks.org\/gfg-gg-logo.svg)](https:\/\/www.geeksforgeeks.org\/)\n\n![search icon](https:\/\/media.geeksforgeeks.org\/auth-dashboard-uploads\/Property=Light---Default.svg)\n\n- Sign In\n- [Courses]()\n- [Tutorials]()\n- [Interview Prep]()\n\n- [Python for Machine Learning](https:\/\/www.geeksforgeeks.org\/machine-learning\/python-for-machine-learning\/)\n- [Machine Learning with R](https:\/\/www.geeksforgeeks.org\/r-machine-learning\/introduction-to-machine-learning-in-r\/)\n- [Machine Learning Algorithms](https:\/\/www.geeksforgeeks.org\/machine-learning\/machine-learning-algorithms\/)\n- [EDA](https:\/\/www.geeksforgeeks.org\/data-analysis\/what-is-exploratory-data-analysis\/)\n- [Math for Machine Learning](https:\/\/www.geeksforgeeks.org\/machine-learning\/machine-learning-mathematics\/)\n- [Machine Learning Interview Questions](https:\/\/www.geeksforgeeks.org\/machine-learning\/machine-learning-interview-questions\/)\n- [ML Projects](https:\/\/www.geeksforgeeks.org\/machine-learning\/machine-learning-projects\/)\n- [Deep Learning](https:\/\/www.geeksforgeeks.org\/deep-learning\/deep-learning-tutorial\/)\n- [NLP](https:\/\/www.geeksforgeeks.org\/nlp\/natural-language-processing-nlp-tutorial\/)\n- [Computer vision](https:\/\/www.geeksforgeeks.org\/computer-vision\/computer-vision\/)\n\n# CatBoost Ranking Metrics: A Comprehensive Guide\nLast Updated : 23 Jul, 2025\n\nCatBoost, short for \"Categorical Boosting,\" is a powerful gradient boosting library developed by Yandex. It is renowned for its efficiency, accuracy, and ability to handle categorical features with ease. One of the key features of CatBoost is its support for ranking tasks, which are crucial in applications like search engines, recommendation systems, and information retrieval. ****This article delves into the various ranking metrics supported by CatBoost, their usage, and how they can be leveraged to build high-performing ranking models.****\n\nTable of Content\n\n- [Understanding Ranking in CatBoost](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#understanding-ranking-in-catboost)\n- [Key CatBoost Ranking Metrics](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#key-catboost-ranking-metrics)\n  - [1\\. Normalized Discounted Cumulative Gain (NDCG)](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#1-normalized-discounted-cumulative-gain-ndcg)\n  - [2\\. Mean Reciprocal Rank (MRR)](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#2-mean-reciprocal-rank-mrr)\n  - [3\\. Expected Reciprocal Rank (ERR)](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#3-expected-reciprocal-rank-err)\n  - [4\\. Mean Average Precision (MAP)](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#4-mean-average-precision-map)\n- [Advanced Ranking Modes: YetiRankPairwise](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#advanced-ranking-modes-yetirankpairwise)\n- [Choosing the Right Ranking Metric](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#choosing-the-right-ranking-metric)\n## Understanding Ranking in CatBoost\nRanking metrics often focus on the performance of the model on the top positions (e.g., top 10) of the retrieved results. [CatBoost](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ml\/) allows you to specify the number of top positions (k) to consider when calculating the metric. Ranking tasks involve ordering items in a list based on their relevance to a particular query. CatBoost provides several ranking modes and metrics to optimize and evaluate ranking models. The primary ranking modes in CatBoost include:\n\n- ****YetiRank****\n- ****PairLogit****\n- ****QuerySoftmax****\n- ****QueryRMSE****\n- ****YetiRankPairwise****\n- ****PairLogitPairwise****\n\nThese modes can be used on both CPU and GPU, with some additional modes like `YetiRankPairwise` and `PairLogitPairwise` available for more complex ranking tasks.\n\n## Key CatBoost Ranking Metrics\n### 1\\. Normalized Discounted Cumulative Gain (NDCG)\nNDCG is a popular metric for evaluating ranking models. It measures the quality of the ranking by comparing the predicted order of items to the ideal order. The NDCG score ranges from 0 to 1, with 1 indicating a perfect ranking.****Parameters:****\n\n- ****Top samples****: The number of top samples in a group used to calculate the ranking metric.\n- ****Metric calculation principles****: Can be `Base` or `Exp`.\n- ****Metric denominator type****: Can be `LogPosition` or `Position`.\n\nPython\n\n``\n\n****Output:****\n```\n0:    learn: 0.5000000    test: 0.4500000    best: 0.4500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.5200000    test: 0.4600000    best: 0.4600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9500000    test: 0.9000000    best: 0.9000000 (999)    total: 1m 40s    remaining: 0us\n\nbestTest = 0.9000000\n```\n### 2\\. Mean Reciprocal Rank (MRR)\nMRR is another metric used to evaluate the effectiveness of a ranking model. It calculates the reciprocal of the rank of the first relevant item in the list.\n\n****Parameters:**** Can be `Base` or `Exp`.\n\nPython\n\n``\n\n****Output:****\n```\n0:    learn: 0.2000000    test: 0.1500000    best: 0.1500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.2200000    test: 0.1600000    best: 0.1600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.8000000    test: 0.7500000    best: 0.7500000 (999)    total: 1m 40s    remaining: 0us\n\nbestTest = 0.7500000\n```\n### 3\\. Expected Reciprocal Rank (ERR)\nERR is a metric that considers the probability of a user stopping at a particular rank. It is useful for scenarios where user satisfaction is paramount. ****Parameters: Probability of search continuation****: Default is 0.85.\n\nPython\n\n``\n\n****Output:****\n```\n0:    learn: 0.3000000    test: 0.2500000    best: 0.2500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.3200000    test: 0.2600000    best: 0.2600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9000000    test: 0.8500000    best: 0.8500000 (999)    total: 1m 40s    remaining: 0us\n\nbestTest = 0.8500000\n```\n### 4\\. Mean Average Precision (MAP)\nMAP is a metric that calculates the mean of the average precision scores for each query. It is particularly useful for binary relevance tasks.\n\nPython\n\n``\n\n****Output:****\n```\n0:    learn: 0.1000000    test: 0.0800000    best: 0.0800000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.1200000    test: 0.0900000    best: 0.0900000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.7000000    test: 0.6500000    best: 0.6500000 (999)    total: 1m 40s    remaining: 0us\n\nbestTest = 0.6500000\n```\n## Advanced Ranking Modes: YetiRankPairwise\nYetiRankPairwise is an advanced ranking mode that optimizes specific ranking loss functions by specifying the `mode` parameter. It supports various loss functions like `DCG`, `NDCG`, `MRR`, `ERR`, and `MAP`. ****Parameters:****\n\n- ****Mode****: Can be `Classic` or a specific ranking loss function.\n- ****Number of permutations****: Default is 10.\n- ****Probability of search continuation****: Default is 0.85.\n\nPython\n\n``\n\n****Output:****\n```\n0:    learn: 0.5000000    test: 0.4500000    best: 0.4500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.5200000    test: 0.4600000    best: 0.4600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9500000    test: 0.9000000    best: 0.9000000 (999)    total: 1m 40s    remaining: 0us\n\nbestTest = 0.9000000\n```\nFor large datasets, it is recommended to use `YetiRankPairwise` or `PairLogitPairwise` as they provide more accurate results but may take longer to train. Additionally, metrics like `PFound` and `NDCG` can be calculated during training to monitor the model's performance.\n\n## Choosing the Right Ranking Metric\nThe best metric for your task depends on your specific needs. Here are some factors to consider:\n\n- ****Relevance vs. Position:**** If precise ranking of highly relevant items is crucial, NDCG or MAP might be better choices.\n- ****Number of Relevant Items:**** For scenarios with a high number of relevant items, HR might be more informative.\n- ****Business Goals:**** Align the metric with your business objectives. For example, if click-through rate is essential, consider incorporating user interaction data into a custom metric.\n## Conclusion\nCatBoost offers a comprehensive set of ranking metrics and modes that cater to various ranking tasks. By leveraging these metrics, data scientists can build robust and high-performing ranking models. Whether you are working on search engines, recommendation systems, or any other ranking application, CatBoost's ranking capabilities provide the tools needed to achieve optimal results.\n\nComment\n\n[A](https:\/\/www.geeksforgeeks.org\/user\/aakritibugax2\/)\n\n[aakritibugax2](https:\/\/www.geeksforgeeks.org\/user\/aakritibugax2\/)\n\n1\n\nArticle Tags:\n\nArticle Tags:\n\n[Machine Learning](https:\/\/www.geeksforgeeks.org\/category\/ai-ml-ds\/machine-learning\/)\n\n[Blogathon](https:\/\/www.geeksforgeeks.org\/category\/blogathon\/)\n\n[AI-ML-DS](https:\/\/www.geeksforgeeks.org\/category\/ai-ml-ds\/)\n\n[CatBoost](https:\/\/www.geeksforgeeks.org\/tag\/catboost\/)\n\n[Data Science Blogathon 2024](https:\/\/www.geeksforgeeks.org\/tag\/data-science-blogathon-2024\/)\n\n[\\+1 More]()\n### Explore\n\n[![GeeksforGeeks](https:\/\/media.geeksforgeeks.org\/auth-dashboard-uploads\/gfgFooterLogo.png)](https:\/\/www.geeksforgeeks.org\/)\n\n![location](https:\/\/media.geeksforgeeks.org\/img-practice\/Location-1685004904.svg)\n\nCorporate & Communications Address:\n\nA-143, 7th Floor, Sovereign Corporate Tower, Sector- 136, Noida, Uttar Pradesh (201305)\n\n![location](https:\/\/media.geeksforgeeks.org\/img-practice\/Location-1685004904.svg)\n\nRegistered Address:\n\nK 061, Tower K, Gulshan Vivante Apartment, Sector 137, Noida, Gautam Buddh Nagar, Uttar Pradesh, 201305\n\n[![GFG App on Play Store](https:\/\/media.geeksforgeeks.org\/auth-dashboard-uploads\/googleplay-%281%29.png)](https:\/\/geeksforgeeksapp.page.link\/gfg-app)[![GFG App on App Store](https:\/\/media.geeksforgeeks.org\/auth-dashboard-uploads\/appstore-%281%29.png)](https:\/\/geeksforgeeksapp.page.link\/gfg-app)\n\n- Company\n- [About Us](https:\/\/www.geeksforgeeks.org\/about\/)\n- [Legal](https:\/\/www.geeksforgeeks.org\/legal\/)\n- [Privacy Policy](https:\/\/www.geeksforgeeks.org\/legal\/privacy-policy\/)\n- [Contact Us](https:\/\/www.geeksforgeeks.org\/about\/contact-us\/)\n- [Advertise with us](https:\/\/www.geeksforgeeks.org\/advertise-with-us\/)\n- [GFG Corporate Solution](https:\/\/www.geeksforgeeks.org\/gfg-corporate-solution\/)\n- [Campus Training Program](https:\/\/www.geeksforgeeks.org\/campus-training-program\/)\n\n- Explore\n- [POTD](https:\/\/www.geeksforgeeks.org\/problem-of-the-day)\n- [Job-A-Thon](https:\/\/practice.geeksforgeeks.org\/events\/rec\/job-a-thon\/)\n- [Blogs](https:\/\/www.geeksforgeeks.org\/category\/blogs\/?type=recent)\n- [Nation Skill Up](https:\/\/www.geeksforgeeks.org\/nation-skill-up\/)\n\n- Tutorials\n- [Programming Languages](https:\/\/www.geeksforgeeks.org\/computer-science-fundamentals\/programming-language-tutorials\/)\n- [DSA](https:\/\/www.geeksforgeeks.org\/dsa\/dsa-tutorial-learn-data-structures-and-algorithms\/)\n- [Web Technology](https:\/\/www.geeksforgeeks.org\/web-tech\/web-technology\/)\n- [AI, ML & Data Science](https:\/\/www.geeksforgeeks.org\/machine-learning\/ai-ml-and-data-science-tutorial-learn-ai-ml-and-data-science\/)\n- [DevOps](https:\/\/www.geeksforgeeks.org\/devops\/devops-tutorial\/)\n- [CS Core Subjects](https:\/\/www.geeksforgeeks.org\/gate\/gate-exam-tutorial\/)\n- [Interview Preparation](https:\/\/www.geeksforgeeks.org\/aptitude\/interview-corner\/)\n- [Software and Tools](https:\/\/www.geeksforgeeks.org\/websites-apps\/software-and-tools-a-to-z-list\/)\n\n- Courses\n- [ML and Data Science](https:\/\/www.geeksforgeeks.org\/courses\/category\/machine-learning-data-science)\n- [DSA and Placements](https:\/\/www.geeksforgeeks.org\/courses\/category\/dsa-placements)\n- [Web Development](https:\/\/www.geeksforgeeks.org\/courses\/category\/development-testing)\n- [Programming Languages](https:\/\/www.geeksforgeeks.org\/courses\/category\/programming-languages)\n- [DevOps & Cloud](https:\/\/www.geeksforgeeks.org\/courses\/category\/cloud-devops)\n- [GATE](https:\/\/www.geeksforgeeks.org\/courses\/category\/gate)\n- [Trending Technologies](https:\/\/www.geeksforgeeks.org\/courses\/category\/trending-technologies\/)\n\n- Videos\n- [DSA](https:\/\/www.geeksforgeeks.org\/videos\/category\/sde-sheet\/)\n- [Python](https:\/\/www.geeksforgeeks.org\/videos\/category\/python\/)\n- [Java](https:\/\/www.geeksforgeeks.org\/videos\/category\/java-w6y5f4\/)\n- [C++](https:\/\/www.geeksforgeeks.org\/videos\/category\/c\/)\n- [Web Development](https:\/\/www.geeksforgeeks.org\/videos\/category\/web-development\/)\n- [Data Science](https:\/\/www.geeksforgeeks.org\/videos\/category\/data-science\/)\n- [CS Subjects](https:\/\/www.geeksforgeeks.org\/videos\/category\/cs-subjects\/)\n\n- Preparation Corner\n- [Interview Corner](https:\/\/www.geeksforgeeks.org\/interview-prep\/interview-corner\/)\n- [Aptitude](https:\/\/www.geeksforgeeks.org\/aptitude\/aptitude-questions-and-answers\/)\n- [Puzzles](https:\/\/www.geeksforgeeks.org\/aptitude\/puzzles\/)\n- [GfG 160](https:\/\/www.geeksforgeeks.org\/courses\/gfg-160-series)\n- [System Design](https:\/\/www.geeksforgeeks.org\/system-design\/system-design-tutorial\/)\n\n[@GeeksforGeeks, Sanchhaya Education Private Limited](https:\/\/www.geeksforgeeks.org\/), [All rights reserved](https:\/\/www.geeksforgeeks.org\/copyright-information\/)\n\n![]()","attrs_readable_markdown":"Last Updated : 23 Jul, 2025\n\nCatBoost, short for \"Categorical Boosting,\" is a powerful gradient boosting library developed by Yandex. It is renowned for its efficiency, accuracy, and ability to handle categorical features with ease. One of the key features of CatBoost is its support for ranking tasks, which are crucial in applications like search engines, recommendation systems, and information retrieval. ****This article delves into the various ranking metrics supported by CatBoost, their usage, and how they can be leveraged to build high-performing ranking models.****\n\nTable of Content\n\n- [Understanding Ranking in CatBoost](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#understanding-ranking-in-catboost)\n- [Key CatBoost Ranking Metrics](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#key-catboost-ranking-metrics)\n  - [1\\. Normalized Discounted Cumulative Gain (NDCG)](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#1-normalized-discounted-cumulative-gain-ndcg)\n  - [2\\. Mean Reciprocal Rank (MRR)](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#2-mean-reciprocal-rank-mrr)\n  - [3\\. Expected Reciprocal Rank (ERR)](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#3-expected-reciprocal-rank-err)\n  - [4\\. Mean Average Precision (MAP)](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#4-mean-average-precision-map)\n- [Advanced Ranking Modes: YetiRankPairwise](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#advanced-ranking-modes-yetirankpairwise)\n- [Choosing the Right Ranking Metric](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ranking-metrics-a-comprehensive-guide\/#choosing-the-right-ranking-metric)\n## Understanding Ranking in CatBoost\nRanking metrics often focus on the performance of the model on the top positions (e.g., top 10) of the retrieved results. [CatBoost](https:\/\/www.geeksforgeeks.org\/machine-learning\/catboost-ml\/) allows you to specify the number of top positions (k) to consider when calculating the metric. Ranking tasks involve ordering items in a list based on their relevance to a particular query. CatBoost provides several ranking modes and metrics to optimize and evaluate ranking models. The primary ranking modes in CatBoost include:\n\n- ****YetiRank****\n- ****PairLogit****\n- ****QuerySoftmax****\n- ****QueryRMSE****\n- ****YetiRankPairwise****\n- ****PairLogitPairwise****\n\nThese modes can be used on both CPU and GPU, with some additional modes like `YetiRankPairwise` and `PairLogitPairwise` available for more complex ranking tasks.\n\n## Key CatBoost Ranking Metrics\n### 1\\. Normalized Discounted Cumulative Gain (NDCG)\nNDCG is a popular metric for evaluating ranking models. It measures the quality of the ranking by comparing the predicted order of items to the ideal order. The NDCG score ranges from 0 to 1, with 1 indicating a perfect ranking.****Parameters:****\n\n- ****Top samples****: The number of top samples in a group used to calculate the ranking metric.\n- ****Metric calculation principles****: Can be `Base` or `Exp`.\n- ****Metric denominator type****: Can be `LogPosition` or `Position`.\n\n``\n\n****Output:****\n```\n0:    learn: 0.5000000    test: 0.4500000    best: 0.4500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.5200000    test: 0.4600000    best: 0.4600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9500000    test: 0.9000000    best: 0.9000000 (999)    total: 1m 40s    remaining: 0usbestTest = 0.9000000\n```\n### 2\\. Mean Reciprocal Rank (MRR)\nMRR is another metric used to evaluate the effectiveness of a ranking model. It calculates the reciprocal of the rank of the first relevant item in the list.\n\n****Parameters:**** Can be `Base` or `Exp`.\n\n``\n\n****Output:****\n```\n0:    learn: 0.2000000    test: 0.1500000    best: 0.1500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.2200000    test: 0.1600000    best: 0.1600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.8000000    test: 0.7500000    best: 0.7500000 (999)    total: 1m 40s    remaining: 0usbestTest = 0.7500000\n```\n### 3\\. Expected Reciprocal Rank (ERR)\nERR is a metric that considers the probability of a user stopping at a particular rank. It is useful for scenarios where user satisfaction is paramount. ****Parameters: Probability of search continuation****: Default is 0.85.\n\n``\n\n****Output:****\n```\n0:    learn: 0.3000000    test: 0.2500000    best: 0.2500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.3200000    test: 0.2600000    best: 0.2600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9000000    test: 0.8500000    best: 0.8500000 (999)    total: 1m 40s    remaining: 0usbestTest = 0.8500000\n```\n### 4\\. Mean Average Precision (MAP)\nMAP is a metric that calculates the mean of the average precision scores for each query. It is particularly useful for binary relevance tasks.\n\n``\n\n****Output:****\n```\n0:    learn: 0.1000000    test: 0.0800000    best: 0.0800000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.1200000    test: 0.0900000    best: 0.0900000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.7000000    test: 0.6500000    best: 0.6500000 (999)    total: 1m 40s    remaining: 0usbestTest = 0.6500000\n```\n## Advanced Ranking Modes: YetiRankPairwise\nYetiRankPairwise is an advanced ranking mode that optimizes specific ranking loss functions by specifying the `mode` parameter. It supports various loss functions like `DCG`, `NDCG`, `MRR`, `ERR`, and `MAP`. ****Parameters:****\n\n- ****Mode****: Can be `Classic` or a specific ranking loss function.\n- ****Number of permutations****: Default is 10.\n- ****Probability of search continuation****: Default is 0.85.\n\n``\n\n****Output:****\n```\n0:    learn: 0.5000000    test: 0.4500000    best: 0.4500000 (0)    total: 0.1s    remaining: 1m 40s\n1:    learn: 0.5200000    test: 0.4600000    best: 0.4600000 (1)    total: 0.2s    remaining: 1m 40s\n...\n999:    learn: 0.9500000    test: 0.9000000    best: 0.9000000 (999)    total: 1m 40s    remaining: 0usbestTest = 0.9000000\n```\nFor large datasets, it is recommended to use `YetiRankPairwise` or `PairLogitPairwise` as they provide more accurate results but may take longer to train. Additionally, metrics like `PFound` and `NDCG` can be calculated during training to monitor the model's performance.\n\n## Choosing the Right Ranking Metric\nThe best metric for your task depends on your specific needs. Here are some factors to consider:\n\n- ****Relevance vs. Position:**** If precise ranking of highly relevant items is crucial, NDCG or MAP might be better choices.\n- ****Number of Relevant Items:**** For scenarios with a high number of relevant items, HR might be more informative.\n- ****Business Goals:**** Align the metric with your business objectives. For example, if click-through rate is essential, consider incorporating user interaction data into a custom metric.\n## Conclusion\nCatBoost offers a comprehensive set of ranking metrics and modes that cater to various ranking tasks. By leveraging these metrics, data scientists can build robust and high-performing ranking models. Whether you are working on search engines, recommendation systems, or any other ranking application, CatBoost's ranking capabilities provide the tools needed to achieve optimal results.","meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

11 days ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0.4 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/
Last Crawled	2026-03-26 20:25:07 (11 days ago)
First Indexed	2025-07-05 14:45:37 (9 months ago)
HTTP Status Code	200
Meta Title	CatBoost Ranking Metrics: A Comprehensive Guide - GeeksforGeeks
Meta Description	Your All-in-One Learning Portal: GeeksforGeeks is a comprehensive educational platform that empowers learners across domains-spanning computer science and programming, school education, upskilling, commerce, software tools, competitive exams, and more., Your All-in-One Learning Portal. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions.
Meta Canonical	null
Boilerpipe Text	Last Updated : 23 Jul, 2025 CatBoost, short for "Categorical Boosting," is a powerful gradient boosting library developed by Yandex. It is renowned for its efficiency, accuracy, and ability to handle categorical features with ease. One of the key features of CatBoost is its support for ranking tasks, which are crucial in applications like search engines, recommendation systems, and information retrieval. This article delves into the various ranking metrics supported by CatBoost, their usage, and how they can be leveraged to build high-performing ranking models. Table of Content Understanding Ranking in CatBoost Key CatBoost Ranking Metrics 1. Normalized Discounted Cumulative Gain (NDCG) 2. Mean Reciprocal Rank (MRR) 3. Expected Reciprocal Rank (ERR) 4. Mean Average Precision (MAP) Advanced Ranking Modes: YetiRankPairwise Choosing the Right Ranking Metric Understanding Ranking in CatBoost Ranking metrics often focus on the performance of the model on the top positions (e.g., top 10) of the retrieved results. CatBoost allows you to specify the number of top positions (k) to consider when calculating the metric. Ranking tasks involve ordering items in a list based on their relevance to a particular query. CatBoost provides several ranking modes and metrics to optimize and evaluate ranking models. The primary ranking modes in CatBoost include: YetiRank PairLogit QuerySoftmax QueryRMSE YetiRankPairwise PairLogitPairwise These modes can be used on both CPU and GPU, with some additional modes like YetiRankPairwise and PairLogitPairwise available for more complex ranking tasks. Key CatBoost Ranking Metrics 1. Normalized Discounted Cumulative Gain (NDCG) NDCG is a popular metric for evaluating ranking models. It measures the quality of the ranking by comparing the predicted order of items to the ideal order. The NDCG score ranges from 0 to 1, with 1 indicating a perfect ranking. Parameters: Top samples : The number of top samples in a group used to calculate the ranking metric. Metric calculation principles : Can be Base or Exp . Metric denominator type : Can be LogPosition or Position . from catboost import CatBoostRanker , Pool # Load data train_data = Pool ( data = X_train , label = y_train , group_id = group_id_train ) test_data = Pool ( data = X_test , label = y_test , group_id = group_id_test ) # Initialize CatBoostRanker model = CatBoostRanker ( iterations = 1000 , learning_rate = 0.1 , depth = 6 , loss_function = 'YetiRankPairwise' , eval_metric = 'NDCG' ) # Train the model model . fit ( train_data , eval_set = test_data ) Output: 0: learn: 0.5000000 test: 0.4500000 best: 0.4500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.5200000 test: 0.4600000 best: 0.4600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9500000 test: 0.9000000 best: 0.9000000 (999) total: 1m 40s remaining: 0us bestTest = 0.9000000 2. Mean Reciprocal Rank (MRR) MRR is another metric used to evaluate the effectiveness of a ranking model. It calculates the reciprocal of the rank of the first relevant item in the list. Parameters: Can be Base or Exp . model = CatBoostRanker ( iterations = 1000 , learning_rate = 0.1 , depth = 6 , loss_function = 'YetiRankPairwise' , eval_metric = 'MRR' ) model . fit ( train_data , eval_set = test_data ) Output: 0: learn: 0.2000000 test: 0.1500000 best: 0.1500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.2200000 test: 0.1600000 best: 0.1600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.8000000 test: 0.7500000 best: 0.7500000 (999) total: 1m 40s remaining: 0us bestTest = 0.7500000 3. Expected Reciprocal Rank (ERR) ERR is a metric that considers the probability of a user stopping at a particular rank. It is useful for scenarios where user satisfaction is paramount. Parameters: Probability of search continuation : Default is 0.85. model = CatBoostRanker ( iterations = 1000 , learning_rate = 0.1 , depth = 6 , loss_function = 'YetiRankPairwise' , eval_metric = 'ERR' ) model . fit ( train_data , eval_set = test_data ) Output: 0: learn: 0.3000000 test: 0.2500000 best: 0.2500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.3200000 test: 0.2600000 best: 0.2600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9000000 test: 0.8500000 best: 0.8500000 (999) total: 1m 40s remaining: 0us bestTest = 0.8500000 4. Mean Average Precision (MAP) MAP is a metric that calculates the mean of the average precision scores for each query. It is particularly useful for binary relevance tasks. model = CatBoostRanker ( iterations = 1000 , learning_rate = 0.1 , depth = 6 , loss_function = 'YetiRankPairwise' , eval_metric = 'MAP' ) model . fit ( train_data , eval_set = test_data ) Output: 0: learn: 0.1000000 test: 0.0800000 best: 0.0800000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.1200000 test: 0.0900000 best: 0.0900000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.7000000 test: 0.6500000 best: 0.6500000 (999) total: 1m 40s remaining: 0us bestTest = 0.6500000 Advanced Ranking Modes: YetiRankPairwise YetiRankPairwise is an advanced ranking mode that optimizes specific ranking loss functions by specifying the mode parameter. It supports various loss functions like DCG , NDCG , MRR , ERR , and MAP . Parameters: Mode : Can be Classic or a specific ranking loss function. Number of permutations : Default is 10. Probability of search continuation : Default is 0.85. model = CatBoostRanker ( iterations = 1000 , learning_rate = 0.1 , depth = 6 , loss_function = 'YetiRankPairwise' , eval_metric = 'NDCG' , custom_metric = [ 'DCG' , 'ERR' ] ) model . fit ( train_data , eval_set = test_data ) Output: 0: learn: 0.5000000 test: 0.4500000 best: 0.4500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.5200000 test: 0.4600000 best: 0.4600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9500000 test: 0.9000000 best: 0.9000000 (999) total: 1m 40s remaining: 0us bestTest = 0.9000000 For large datasets, it is recommended to use YetiRankPairwise or PairLogitPairwise as they provide more accurate results but may take longer to train. Additionally, metrics like PFound and NDCG can be calculated during training to monitor the model's performance. Choosing the Right Ranking Metric The best metric for your task depends on your specific needs. Here are some factors to consider: Relevance vs. Position: If precise ranking of highly relevant items is crucial, NDCG or MAP might be better choices. Number of Relevant Items: For scenarios with a high number of relevant items, HR might be more informative. Business Goals: Align the metric with your business objectives. For example, if click-through rate is essential, consider incorporating user interaction data into a custom metric. Conclusion CatBoost offers a comprehensive set of ranking metrics and modes that cater to various ranking tasks. By leveraging these metrics, data scientists can build robust and high-performing ranking models. Whether you are working on search engines, recommendation systems, or any other ranking application, CatBoost's ranking capabilities provide the tools needed to achieve optimal results.
Markdown	[![geeksforgeeks](https://media.geeksforgeeks.org/gfg-gg-logo.svg)](https://www.geeksforgeeks.org/) ![search icon](https://media.geeksforgeeks.org/auth-dashboard-uploads/Property=Light---Default.svg) - Sign In - [Courses]() - [Tutorials]() - [Interview Prep]() - [Python for Machine Learning](https://www.geeksforgeeks.org/machine-learning/python-for-machine-learning/) - [Machine Learning with R](https://www.geeksforgeeks.org/r-machine-learning/introduction-to-machine-learning-in-r/) - [Machine Learning Algorithms](https://www.geeksforgeeks.org/machine-learning/machine-learning-algorithms/) - [EDA](https://www.geeksforgeeks.org/data-analysis/what-is-exploratory-data-analysis/) - [Math for Machine Learning](https://www.geeksforgeeks.org/machine-learning/machine-learning-mathematics/) - [Machine Learning Interview Questions](https://www.geeksforgeeks.org/machine-learning/machine-learning-interview-questions/) - [ML Projects](https://www.geeksforgeeks.org/machine-learning/machine-learning-projects/) - [Deep Learning](https://www.geeksforgeeks.org/deep-learning/deep-learning-tutorial/) - [NLP](https://www.geeksforgeeks.org/nlp/natural-language-processing-nlp-tutorial/) - [Computer vision](https://www.geeksforgeeks.org/computer-vision/computer-vision/) # CatBoost Ranking Metrics: A Comprehensive Guide Last Updated : 23 Jul, 2025 CatBoost, short for "Categorical Boosting," is a powerful gradient boosting library developed by Yandex. It is renowned for its efficiency, accuracy, and ability to handle categorical features with ease. One of the key features of CatBoost is its support for ranking tasks, which are crucial in applications like search engines, recommendation systems, and information retrieval. **This article delves into the various ranking metrics supported by CatBoost, their usage, and how they can be leveraged to build high-performing ranking models. Table of Content - [Understanding Ranking in CatBoost](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#understanding-ranking-in-catboost) - [Key CatBoost Ranking Metrics](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#key-catboost-ranking-metrics) - [1\. Normalized Discounted Cumulative Gain (NDCG)](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#1-normalized-discounted-cumulative-gain-ndcg) - [2\. Mean Reciprocal Rank (MRR)](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#2-mean-reciprocal-rank-mrr) - [3\. Expected Reciprocal Rank (ERR)](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#3-expected-reciprocal-rank-err) - [4\. Mean Average Precision (MAP)](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#4-mean-average-precision-map) - [Advanced Ranking Modes: YetiRankPairwise](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#advanced-ranking-modes-yetirankpairwise) - [Choosing the Right Ranking Metric](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#choosing-the-right-ranking-metric) ## Understanding Ranking in CatBoost Ranking metrics often focus on the performance of the model on the top positions (e.g., top 10) of the retrieved results. [CatBoost](https://www.geeksforgeeks.org/machine-learning/catboost-ml/) allows you to specify the number of top positions (k) to consider when calculating the metric. Ranking tasks involve ordering items in a list based on their relevance to a particular query. CatBoost provides several ranking modes and metrics to optimize and evaluate ranking models. The primary ranking modes in CatBoost include: - YetiRank - PairLogit - QuerySoftmax - QueryRMSE - YetiRankPairwise - PairLogitPairwise These modes can be used on both CPU and GPU, with some additional modes like `YetiRankPairwise` and `PairLogitPairwise` available for more complex ranking tasks. ## Key CatBoost Ranking Metrics ### 1\. Normalized Discounted Cumulative Gain (NDCG) NDCG is a popular metric for evaluating ranking models. It measures the quality of the ranking by comparing the predicted order of items to the ideal order. The NDCG score ranges from 0 to 1, with 1 indicating a perfect ranking.Parameters: - Top samples: The number of top samples in a group used to calculate the ranking metric. - Metric calculation principles: Can be `Base` or `Exp`. - Metric denominator type: Can be `LogPosition` or `Position`. Python `` Output: ``` 0: learn: 0.5000000 test: 0.4500000 best: 0.4500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.5200000 test: 0.4600000 best: 0.4600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9500000 test: 0.9000000 best: 0.9000000 (999) total: 1m 40s remaining: 0us bestTest = 0.9000000 ``` ### 2\. Mean Reciprocal Rank (MRR) MRR is another metric used to evaluate the effectiveness of a ranking model. It calculates the reciprocal of the rank of the first relevant item in the list. Parameters: Can be `Base` or `Exp`. Python `` Output: ``` 0: learn: 0.2000000 test: 0.1500000 best: 0.1500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.2200000 test: 0.1600000 best: 0.1600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.8000000 test: 0.7500000 best: 0.7500000 (999) total: 1m 40s remaining: 0us bestTest = 0.7500000 ``` ### 3\. Expected Reciprocal Rank (ERR) ERR is a metric that considers the probability of a user stopping at a particular rank. It is useful for scenarios where user satisfaction is paramount. Parameters: Probability of search continuation: Default is 0.85. Python `` Output: ``` 0: learn: 0.3000000 test: 0.2500000 best: 0.2500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.3200000 test: 0.2600000 best: 0.2600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9000000 test: 0.8500000 best: 0.8500000 (999) total: 1m 40s remaining: 0us bestTest = 0.8500000 ``` ### 4\. Mean Average Precision (MAP) MAP is a metric that calculates the mean of the average precision scores for each query. It is particularly useful for binary relevance tasks. Python `` Output: ``` 0: learn: 0.1000000 test: 0.0800000 best: 0.0800000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.1200000 test: 0.0900000 best: 0.0900000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.7000000 test: 0.6500000 best: 0.6500000 (999) total: 1m 40s remaining: 0us bestTest = 0.6500000 ``` ## Advanced Ranking Modes: YetiRankPairwise YetiRankPairwise is an advanced ranking mode that optimizes specific ranking loss functions by specifying the `mode` parameter. It supports various loss functions like `DCG`, `NDCG`, `MRR`, `ERR`, and `MAP`. Parameters: - Mode: Can be `Classic` or a specific ranking loss function. - Number of permutations: Default is 10. - Probability of search continuation: Default is 0.85. Python `` Output: ``` 0: learn: 0.5000000 test: 0.4500000 best: 0.4500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.5200000 test: 0.4600000 best: 0.4600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9500000 test: 0.9000000 best: 0.9000000 (999) total: 1m 40s remaining: 0us bestTest = 0.9000000 ``` For large datasets, it is recommended to use `YetiRankPairwise` or `PairLogitPairwise` as they provide more accurate results but may take longer to train. Additionally, metrics like `PFound` and `NDCG` can be calculated during training to monitor the model's performance. ## Choosing the Right Ranking Metric The best metric for your task depends on your specific needs. Here are some factors to consider: - Relevance vs. Position: If precise ranking of highly relevant items is crucial, NDCG or MAP might be better choices. - Number of Relevant Items: For scenarios with a high number of relevant items, HR might be more informative. - Business Goals:** Align the metric with your business objectives. For example, if click-through rate is essential, consider incorporating user interaction data into a custom metric. ## Conclusion CatBoost offers a comprehensive set of ranking metrics and modes that cater to various ranking tasks. By leveraging these metrics, data scientists can build robust and high-performing ranking models. Whether you are working on search engines, recommendation systems, or any other ranking application, CatBoost's ranking capabilities provide the tools needed to achieve optimal results. Comment [A](https://www.geeksforgeeks.org/user/aakritibugax2/) [aakritibugax2](https://www.geeksforgeeks.org/user/aakritibugax2/) 1 Article Tags: Article Tags: [Machine Learning](https://www.geeksforgeeks.org/category/ai-ml-ds/machine-learning/) [Blogathon](https://www.geeksforgeeks.org/category/blogathon/) [AI-ML-DS](https://www.geeksforgeeks.org/category/ai-ml-ds/) [CatBoost](https://www.geeksforgeeks.org/tag/catboost/) [Data Science Blogathon 2024](https://www.geeksforgeeks.org/tag/data-science-blogathon-2024/) [\+1 More]() ### Explore [![GeeksforGeeks](https://media.geeksforgeeks.org/auth-dashboard-uploads/gfgFooterLogo.png)](https://www.geeksforgeeks.org/) ![location](https://media.geeksforgeeks.org/img-practice/Location-1685004904.svg) Corporate & Communications Address: A-143, 7th Floor, Sovereign Corporate Tower, Sector- 136, Noida, Uttar Pradesh (201305) ![location](https://media.geeksforgeeks.org/img-practice/Location-1685004904.svg) Registered Address: K 061, Tower K, Gulshan Vivante Apartment, Sector 137, Noida, Gautam Buddh Nagar, Uttar Pradesh, 201305 [![GFG App on Play Store](https://media.geeksforgeeks.org/auth-dashboard-uploads/googleplay-%281%29.png)](https://geeksforgeeksapp.page.link/gfg-app)[![GFG App on App Store](https://media.geeksforgeeks.org/auth-dashboard-uploads/appstore-%281%29.png)](https://geeksforgeeksapp.page.link/gfg-app) - Company - [About Us](https://www.geeksforgeeks.org/about/) - [Legal](https://www.geeksforgeeks.org/legal/) - [Privacy Policy](https://www.geeksforgeeks.org/legal/privacy-policy/) - [Contact Us](https://www.geeksforgeeks.org/about/contact-us/) - [Advertise with us](https://www.geeksforgeeks.org/advertise-with-us/) - [GFG Corporate Solution](https://www.geeksforgeeks.org/gfg-corporate-solution/) - [Campus Training Program](https://www.geeksforgeeks.org/campus-training-program/) - Explore - [POTD](https://www.geeksforgeeks.org/problem-of-the-day) - [Job-A-Thon](https://practice.geeksforgeeks.org/events/rec/job-a-thon/) - [Blogs](https://www.geeksforgeeks.org/category/blogs/?type=recent) - [Nation Skill Up](https://www.geeksforgeeks.org/nation-skill-up/) - Tutorials - [Programming Languages](https://www.geeksforgeeks.org/computer-science-fundamentals/programming-language-tutorials/) - [DSA](https://www.geeksforgeeks.org/dsa/dsa-tutorial-learn-data-structures-and-algorithms/) - [Web Technology](https://www.geeksforgeeks.org/web-tech/web-technology/) - [AI, ML & Data Science](https://www.geeksforgeeks.org/machine-learning/ai-ml-and-data-science-tutorial-learn-ai-ml-and-data-science/) - [DevOps](https://www.geeksforgeeks.org/devops/devops-tutorial/) - [CS Core Subjects](https://www.geeksforgeeks.org/gate/gate-exam-tutorial/) - [Interview Preparation](https://www.geeksforgeeks.org/aptitude/interview-corner/) - [Software and Tools](https://www.geeksforgeeks.org/websites-apps/software-and-tools-a-to-z-list/) - Courses - [ML and Data Science](https://www.geeksforgeeks.org/courses/category/machine-learning-data-science) - [DSA and Placements](https://www.geeksforgeeks.org/courses/category/dsa-placements) - [Web Development](https://www.geeksforgeeks.org/courses/category/development-testing) - [Programming Languages](https://www.geeksforgeeks.org/courses/category/programming-languages) - [DevOps & Cloud](https://www.geeksforgeeks.org/courses/category/cloud-devops) - [GATE](https://www.geeksforgeeks.org/courses/category/gate) - [Trending Technologies](https://www.geeksforgeeks.org/courses/category/trending-technologies/) - Videos - [DSA](https://www.geeksforgeeks.org/videos/category/sde-sheet/) - [Python](https://www.geeksforgeeks.org/videos/category/python/) - [Java](https://www.geeksforgeeks.org/videos/category/java-w6y5f4/) - [C++](https://www.geeksforgeeks.org/videos/category/c/) - [Web Development](https://www.geeksforgeeks.org/videos/category/web-development/) - [Data Science](https://www.geeksforgeeks.org/videos/category/data-science/) - [CS Subjects](https://www.geeksforgeeks.org/videos/category/cs-subjects/) - Preparation Corner - [Interview Corner](https://www.geeksforgeeks.org/interview-prep/interview-corner/) - [Aptitude](https://www.geeksforgeeks.org/aptitude/aptitude-questions-and-answers/) - [Puzzles](https://www.geeksforgeeks.org/aptitude/puzzles/) - [GfG 160](https://www.geeksforgeeks.org/courses/gfg-160-series) - [System Design](https://www.geeksforgeeks.org/system-design/system-design-tutorial/) [@GeeksforGeeks, Sanchhaya Education Private Limited](https://www.geeksforgeeks.org/), [All rights reserved](https://www.geeksforgeeks.org/copyright-information/) ![]()
Readable Markdown	Last Updated : 23 Jul, 2025 CatBoost, short for "Categorical Boosting," is a powerful gradient boosting library developed by Yandex. It is renowned for its efficiency, accuracy, and ability to handle categorical features with ease. One of the key features of CatBoost is its support for ranking tasks, which are crucial in applications like search engines, recommendation systems, and information retrieval. **This article delves into the various ranking metrics supported by CatBoost, their usage, and how they can be leveraged to build high-performing ranking models. Table of Content - [Understanding Ranking in CatBoost](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#understanding-ranking-in-catboost) - [Key CatBoost Ranking Metrics](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#key-catboost-ranking-metrics) - [1\. Normalized Discounted Cumulative Gain (NDCG)](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#1-normalized-discounted-cumulative-gain-ndcg) - [2\. Mean Reciprocal Rank (MRR)](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#2-mean-reciprocal-rank-mrr) - [3\. Expected Reciprocal Rank (ERR)](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#3-expected-reciprocal-rank-err) - [4\. Mean Average Precision (MAP)](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#4-mean-average-precision-map) - [Advanced Ranking Modes: YetiRankPairwise](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#advanced-ranking-modes-yetirankpairwise) - [Choosing the Right Ranking Metric](https://www.geeksforgeeks.org/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/#choosing-the-right-ranking-metric) ## Understanding Ranking in CatBoost Ranking metrics often focus on the performance of the model on the top positions (e.g., top 10) of the retrieved results. [CatBoost](https://www.geeksforgeeks.org/machine-learning/catboost-ml/) allows you to specify the number of top positions (k) to consider when calculating the metric. Ranking tasks involve ordering items in a list based on their relevance to a particular query. CatBoost provides several ranking modes and metrics to optimize and evaluate ranking models. The primary ranking modes in CatBoost include: - YetiRank - PairLogit - QuerySoftmax - QueryRMSE - YetiRankPairwise - PairLogitPairwise These modes can be used on both CPU and GPU, with some additional modes like `YetiRankPairwise` and `PairLogitPairwise` available for more complex ranking tasks. ## Key CatBoost Ranking Metrics ### 1\. Normalized Discounted Cumulative Gain (NDCG) NDCG is a popular metric for evaluating ranking models. It measures the quality of the ranking by comparing the predicted order of items to the ideal order. The NDCG score ranges from 0 to 1, with 1 indicating a perfect ranking.Parameters: - Top samples: The number of top samples in a group used to calculate the ranking metric. - Metric calculation principles: Can be `Base` or `Exp`. - Metric denominator type: Can be `LogPosition` or `Position`. `` Output: ``` 0: learn: 0.5000000 test: 0.4500000 best: 0.4500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.5200000 test: 0.4600000 best: 0.4600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9500000 test: 0.9000000 best: 0.9000000 (999) total: 1m 40s remaining: 0usbestTest = 0.9000000 ``` ### 2\. Mean Reciprocal Rank (MRR) MRR is another metric used to evaluate the effectiveness of a ranking model. It calculates the reciprocal of the rank of the first relevant item in the list. Parameters: Can be `Base` or `Exp`. `` Output: ``` 0: learn: 0.2000000 test: 0.1500000 best: 0.1500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.2200000 test: 0.1600000 best: 0.1600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.8000000 test: 0.7500000 best: 0.7500000 (999) total: 1m 40s remaining: 0usbestTest = 0.7500000 ``` ### 3\. Expected Reciprocal Rank (ERR) ERR is a metric that considers the probability of a user stopping at a particular rank. It is useful for scenarios where user satisfaction is paramount. Parameters: Probability of search continuation: Default is 0.85. `` Output: ``` 0: learn: 0.3000000 test: 0.2500000 best: 0.2500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.3200000 test: 0.2600000 best: 0.2600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9000000 test: 0.8500000 best: 0.8500000 (999) total: 1m 40s remaining: 0usbestTest = 0.8500000 ``` ### 4\. Mean Average Precision (MAP) MAP is a metric that calculates the mean of the average precision scores for each query. It is particularly useful for binary relevance tasks. `` Output: ``` 0: learn: 0.1000000 test: 0.0800000 best: 0.0800000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.1200000 test: 0.0900000 best: 0.0900000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.7000000 test: 0.6500000 best: 0.6500000 (999) total: 1m 40s remaining: 0usbestTest = 0.6500000 ``` ## Advanced Ranking Modes: YetiRankPairwise YetiRankPairwise is an advanced ranking mode that optimizes specific ranking loss functions by specifying the `mode` parameter. It supports various loss functions like `DCG`, `NDCG`, `MRR`, `ERR`, and `MAP`. Parameters: - Mode: Can be `Classic` or a specific ranking loss function. - Number of permutations: Default is 10. - Probability of search continuation: Default is 0.85. `` Output: ``` 0: learn: 0.5000000 test: 0.4500000 best: 0.4500000 (0) total: 0.1s remaining: 1m 40s 1: learn: 0.5200000 test: 0.4600000 best: 0.4600000 (1) total: 0.2s remaining: 1m 40s ... 999: learn: 0.9500000 test: 0.9000000 best: 0.9000000 (999) total: 1m 40s remaining: 0usbestTest = 0.9000000 ``` For large datasets, it is recommended to use `YetiRankPairwise` or `PairLogitPairwise` as they provide more accurate results but may take longer to train. Additionally, metrics like `PFound` and `NDCG` can be calculated during training to monitor the model's performance. ## Choosing the Right Ranking Metric The best metric for your task depends on your specific needs. Here are some factors to consider: - Relevance vs. Position: If precise ranking of highly relevant items is crucial, NDCG or MAP might be better choices. - Number of Relevant Items: For scenarios with a high number of relevant items, HR might be more informative. - Business Goals:** Align the metric with your business objectives. For example, if click-through rate is essential, consider incorporating user interaction data into a custom metric. ## Conclusion CatBoost offers a comprehensive set of ranking metrics and modes that cater to various ranking tasks. By leveraging these metrics, data scientists can build robust and high-performing ranking models. Whether you are working on search engines, recommendation systems, or any other ranking application, CatBoost's ranking capabilities provide the tools needed to achieve optimal results.
Shard	103 (laksa)
Root Hash	12046344915360636903
Unparsed URL	org,geeksforgeeks!www,/machine-learning/catboost-ranking-metrics-a-comprehensive-guide/ s443