🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 84 (from laksa070)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa084.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://www.nytimes.com/video/multimedia/1247465085049/obama-surprised-humbled-by-nobel.html\')), getAhrefsUnparsedNoserviceFromURL(\'https://www.nytimes.com/video/multimedia/1247465085049/obama-surprised-humbled-by-nobel.html\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/www.nytimes.com\/video\/multimedia\/1247465085049\/obama-surprised-humbled-by-nobel.html","crawl_time":1756239706,"first_indexed_time":0,"http_code":200,"src_unparsed":"com,nytimes!www,\/video\/multimedia\/1247465085049\/obama-surprised-humbled-by-nobel.html s443","src_root_hash":"4566504020376537684","history_drop_reason":null,"meta_title":"Obama: Surprised, Humbled by Nobel - The New York Times","meta_descriptions":["U.S. President Barack Obama expressed surprise over winning the 2009 Nobel Peace Prize but said he accepted the honor as a call to action"],"attrs_boilerpipe_text":"James A. Lovell Jr., Commander of Apollo 13, Dies at 97","attrs_markdown":"[Skip to content](https:\/\/www.nytimes.com\/video\/multimedia\/1247465085049\/obama-surprised-humbled-by-nobel.html#site-content)[Skip to site index](https:\/\/www.nytimes.com\/video\/multimedia\/1247465085049\/obama-surprised-humbled-by-nobel.html#site-index)\n\nnew video loaded: Obama: Surprised, Humbled by Nobel\n\n## Breaking News\n# Obama: Surprised, Humbled by Nobel\nReuters •\n\nOctober 9, 2009\n## U.S. President Barack Obama expressed surprise over winning the 2009 Nobel Peace Prize but said he accepted the honor as a call to action\n\nAdvertisement\n\n[SKIP ADVERTISEMENT](https:\/\/www.nytimes.com\/video\/multimedia\/1247465085049\/obama-surprised-humbled-by-nobel.html#after-mid1)\n\nRecent episodes in Breaking News\n\nShow more videos from Breaking News\n\n[Latest Video](https:\/\/www.nytimes.com\/video\/latest-video)\n\n[Visual Investigations](https:\/\/www.nytimes.com\/video\/investigations)\n\n[Diary of a Song](https:\/\/www.nytimes.com\/video\/diaryofasong)\n\n[Magazine](https:\/\/www.nytimes.com\/video\/magazine)\n\n[T Magazine](https:\/\/www.nytimes.com\/video\/t-magazine)\n\n[Op-Docs](https:\/\/www.nytimes.com\/video\/op-docs)\n\n[Opinion](https:\/\/www.nytimes.com\/video\/opinion)\n\n[Middle East Crisis](https:\/\/www.nytimes.com\/video\/middle-east-crisis)\n\nAdvertisement\n\n[SKIP ADVERTISEMENT](https:\/\/www.nytimes.com\/video\/multimedia\/1247465085049\/obama-surprised-humbled-by-nobel.html#after-bottom)\n## Site Index\n## Site Information Navigation\n- [© 2025 The New York Times Company](https:\/\/help.nytimes.com\/hc\/en-us\/articles\/115014792127-Copyright-Notice)\n\n- [NYTCo](https:\/\/www.nytco.com\/)\n- [Contact Us](https:\/\/help.nytimes.com\/hc\/en-us\/articles\/115015385887-Contact-The-New-York-Times)\n- [Accessibility](https:\/\/help.nytimes.com\/hc\/en-us\/articles\/115015727108-Accessibility)\n- [Work with us](https:\/\/www.nytco.com\/careers\/)\n- [Advertise](https:\/\/advertising.nytimes.com\/)\n- [T Brand Studio](https:\/\/advertising.nytimes.com\/custom-content\/)\n- [Your Ad Choices](https:\/\/www.nytimes.com\/privacy\/cookie-policy#how-do-i-manage-trackers)\n- [Privacy Policy](https:\/\/help.nytimes.com\/hc\/en-us\/articles\/10940941449492-The-New-York-Times-Company-Privacy-Policy)\n- [Terms of Service](https:\/\/help.nytimes.com\/hc\/en-us\/articles\/115014893428-Terms-of-Service)\n- [Terms of Sale](https:\/\/help.nytimes.com\/hc\/en-us\/articles\/115014893968-Terms-of-Sale)\n- [Site Map](https:\/\/www.nytimes.com\/sitemap\/)\n- [Canada](https:\/\/www.nytimes.com\/ca\/)\n- [International](https:\/\/www.nytimes.com\/international\/)\n- [Help](https:\/\/help.nytimes.com\/hc\/en-us)\n- [Subscriptions](https:\/\/www.nytimes.com\/subscription?campaignId=37WXW)\n\n- [Manage Privacy Preferences](https:\/\/www.nytimes.com\/privacy\/manage-settings)","attrs_readable_markdown":null,"meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

🚫

NOT INDEXABLE

✅

CRAWLED

7 months ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	FAIL	`download_stamp > now() - 6 MONTH`	7.7 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://www.nytimes.com/video/multimedia/1247465085049/obama-surprised-humbled-by-nobel.html
Last Crawled	2025-08-26 20:21:46 (7 months ago)
First Indexed	not set
HTTP Status Code	200
Meta Title	Obama: Surprised, Humbled by Nobel - The New York Times
Meta Description	U.S. President Barack Obama expressed surprise over winning the 2009 Nobel Peace Prize but said he accepted the honor as a call to action
Meta Canonical	null
Boilerpipe Text	James A. Lovell Jr., Commander of Apollo 13, Dies at 97
Markdown	[Skip to content](https://www.nytimes.com/video/multimedia/1247465085049/obama-surprised-humbled-by-nobel.html#site-content)[Skip to site index](https://www.nytimes.com/video/multimedia/1247465085049/obama-surprised-humbled-by-nobel.html#site-index) new video loaded: Obama: Surprised, Humbled by Nobel ## Breaking News # Obama: Surprised, Humbled by Nobel Reuters • October 9, 2009 ## U.S. President Barack Obama expressed surprise over winning the 2009 Nobel Peace Prize but said he accepted the honor as a call to action Advertisement [SKIP ADVERTISEMENT](https://www.nytimes.com/video/multimedia/1247465085049/obama-surprised-humbled-by-nobel.html#after-mid1) Recent episodes in Breaking News Show more videos from Breaking News [Latest Video](https://www.nytimes.com/video/latest-video) [Visual Investigations](https://www.nytimes.com/video/investigations) [Diary of a Song](https://www.nytimes.com/video/diaryofasong) [Magazine](https://www.nytimes.com/video/magazine) [T Magazine](https://www.nytimes.com/video/t-magazine) [Op-Docs](https://www.nytimes.com/video/op-docs) [Opinion](https://www.nytimes.com/video/opinion) [Middle East Crisis](https://www.nytimes.com/video/middle-east-crisis) Advertisement [SKIP ADVERTISEMENT](https://www.nytimes.com/video/multimedia/1247465085049/obama-surprised-humbled-by-nobel.html#after-bottom) ## Site Index ## Site Information Navigation - [© 2025 The New York Times Company](https://help.nytimes.com/hc/en-us/articles/115014792127-Copyright-Notice) - [NYTCo](https://www.nytco.com/) - [Contact Us](https://help.nytimes.com/hc/en-us/articles/115015385887-Contact-The-New-York-Times) - [Accessibility](https://help.nytimes.com/hc/en-us/articles/115015727108-Accessibility) - [Work with us](https://www.nytco.com/careers/) - [Advertise](https://advertising.nytimes.com/) - [T Brand Studio](https://advertising.nytimes.com/custom-content/) - [Your Ad Choices](https://www.nytimes.com/privacy/cookie-policy#how-do-i-manage-trackers) - [Privacy Policy](https://help.nytimes.com/hc/en-us/articles/10940941449492-The-New-York-Times-Company-Privacy-Policy) - [Terms of Service](https://help.nytimes.com/hc/en-us/articles/115014893428-Terms-of-Service) - [Terms of Sale](https://help.nytimes.com/hc/en-us/articles/115014893968-Terms-of-Sale) - [Site Map](https://www.nytimes.com/sitemap/) - [Canada](https://www.nytimes.com/ca/) - [International](https://www.nytimes.com/international/) - [Help](https://help.nytimes.com/hc/en-us) - [Subscriptions](https://www.nytimes.com/subscription?campaignId=37WXW) - [Manage Privacy Preferences](https://www.nytimes.com/privacy/manage-settings)
Readable Markdown	null
Shard	84 (laksa)
Root Hash	4566504020376537684
Unparsed URL	com,nytimes!www,/video/multimedia/1247465085049/obama-surprised-humbled-by-nobel.html s443