🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 56 (from laksa075)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa056.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://www.wbur.org/npr/131351976/prince-william-to-marry-long-time-girlfriend-kate-middleton\')), getAhrefsUnparsedNoserviceFromURL(\'https://www.wbur.org/npr/131351976/prince-william-to-marry-long-time-girlfriend-kate-middleton\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/www.wbur.org\/npr\/131351976\/prince-william-to-marry-long-time-girlfriend-kate-middleton","crawl_time":1776227991,"first_indexed_time":1544027729,"http_code":200,"src_unparsed":"org,wbur!www,\/npr\/131351976\/prince-william-to-marry-long-time-girlfriend-kate-middleton s443","src_root_hash":"15087967046536411256","history_drop_reason":null,"meta_title":"Prince William To Marry Longtime Girlfriend Kate Middleton | WBUR","meta_descriptions":["Royal officials say the prince and Kate Middleton got engaged last month during a vacation in Kenya. They are due to marry next spring or summer in London."],"attrs_boilerpipe_text":"Support WBUR\nBritain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP)\nSupport WBUR\nSupport WBUR\nListen Live","attrs_markdown":"[Skip to main content](https:\/\/www.wbur.org\/npr\/131351976\/prince-william-to-marry-long-time-girlfriend-kate-middleton#main)\n\n[WBUR90.0 WBUR - Boston's NPR News Station](https:\/\/www.wbur.org\/)\n\nListen Live: Loading...\n\n[Donate](https:\/\/donate.wbur.org\/secure\/heart-wbur-homepage?utm_source=wbur.org&utm_medium=heart)\n\n[WBUR90.0 WBUR - Boston's NPR News Station](https:\/\/www.wbur.org\/)\n\nListen Live: Loading...\n\n[Donate](https:\/\/donate.wbur.org\/secure\/heart-wbur-homepage?utm_source=wbur.org&utm_medium=heart)\n\nListen Live\n\n#### Sections\n- [Local Coverage](https:\/\/www.wbur.org\/news)\n- [Arts & Culture](https:\/\/www.wbur.org\/news\/section\/arts-culture)\n- [Business](https:\/\/www.wbur.org\/news\/section\/business)\n- [Education](https:\/\/www.wbur.org\/news\/section\/education)\n- [Environment](https:\/\/www.wbur.org\/news\/section\/environment)\n- [Field Guide To Boston](https:\/\/www.wbur.org\/news\/section\/field-guide-to-boston)\n- [Health](https:\/\/www.wbur.org\/news\/section\/health)\n- [Investigations](https:\/\/www.wbur.org\/news\/section\/investigations)\n- [Cognoscenti](https:\/\/www.wbur.org\/opinion\/section\/cognoscenti)\n- [Boston News Quiz](https:\/\/www.wbur.org\/quiz)\n\n#### Radio\n- [All Radio Programs](https:\/\/www.wbur.org\/radio)\n- [All Things Considered](https:\/\/www.wbur.org\/radio\/programs\/all-things-considered)\n- [Here & Now](https:\/\/www.wbur.org\/radio\/programs\/hereandnow)\n- [Morning Edition](https:\/\/www.wbur.org\/radio\/programs\/morningedition)\n- [On Point](https:\/\/www.wbur.org\/radio\/programs\/onpoint)\n- [On-Air Schedule](https:\/\/www.wbur.org\/radio\/schedule)\n- [Ways To Listen](https:\/\/www.wbur.org\/ways-to-listen)\n\n#### Podcasts\n- [All Podcasts](https:\/\/www.wbur.org\/podcasts)\n- [Beyond All Repair](https:\/\/www.wbur.org\/podcasts\/violation)\n- [Circle Round](https:\/\/www.wbur.org\/podcasts\/circleround)\n- [Endless Thread](https:\/\/www.wbur.org\/podcasts\/endlessthread)\n- [Last Seen](https:\/\/www.wbur.org\/podcasts\/lastseen)\n- [The Midnight Rebellion](https:\/\/podcasts.wbur.org\/the-midnight-rebellion)\n- [Violation](https:\/\/www.wbur.org\/podcasts\/violation\/the-marshall-project-wbur-violation)\n\n#### Events\n- [CitySpace Calendar](https:\/\/www.wbur.org\/events)\n- [Watch Past Events](https:\/\/www.wbur.org\/events\/browse?archive)\n- [Events Newsletter](https:\/\/www.wbur.org\/events\/newsletter)\n- [Rent CitySpace](https:\/\/www.rentcityspace.com\/)\n- [The WBUR Festival](https:\/\/www.wburfestival.org\/)\n\n#### Support\n- [Member Services](https:\/\/www.wbur.org\/membership)\n- [Become A Member](https:\/\/www.wbur.org\/membership\/605748\/members)\n- [Donate Your Car](https:\/\/www.wbur.org\/membership\/605928\/donate-your-car)\n- [Join The Murrow Society](https:\/\/www.wbur.org\/murrowsociety)\n- [Make A Donation](https:\/\/donate.wbur.org\/secure\/give?utm_source=wbur.org&utm_medium=web&utm_term=hamburger_menu)\n- [Become A Sponsor](https:\/\/www.wbur.org\/sponsorship\/about)\n- [Create A Custom Podcast](https:\/\/www.wbur.org\/inside\/cityspaceproductions)\n\nSupport WBUR\n\n# Prince William To Marry Longtime Girlfriend Kate Middleton\n- [JJ Sutherland](https:\/\/npr.org\/people\/5239626\/jj-sutherland)\n\nNovember 16, 2010\n\nShare\n\n![Britain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP)](https:\/\/static.wbur.org\/images\/npr-logo.svg)\n\nBritain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP)\n\nSupport WBUR\n\nSupport WBUR\n\n*Resume*\n\nListen Live\n\n[Loading...](https:\/\/www.wbur.org\/)\n\n#### Contact Us\n(617) 353-0909\n\n[info@wbur.org](mailto:info@wbur.org)\n\n890 Commonwealth Ave.  \nBoston, MA 02215\n\n[More ways to get in touch.](https:\/\/www.wbur.org\/inside\/contact)\n\n#### About WBUR\n- [Who We Are](https:\/\/www.wbur.org\/about)\n- [Inside WBUR](https:\/\/www.wbur.org\/inside)\n- [Diversity, Equity And Inclusion](https:\/\/www.wbur.org\/inside\/2021\/11\/15\/diversity-equity-and-inclusion)\n- [Board Of Directors](https:\/\/www.wbur.org\/inside\/board-of-directors)\n- [Community Advisory Board](https:\/\/www.wbur.org\/inside\/community-advisory-board)\n- [CPB Transparency](https:\/\/www.wbur.org\/inside\/cpb-transparency)\n- [Careers](https:\/\/www.wbur.org\/inside\/jobs)\n- [WBUR Staff](https:\/\/www.wbur.org\/inside\/staff)\n- [Ethics Guide](https:\/\/www.wbur.org\/inside\/2022\/03\/10\/ethics-guide)\n- [Licensing WBUR Content](https:\/\/www.wbur.org\/inside\/licensing-wbur-content)\n\n#### Support WBUR\n- [Make A Donation](https:\/\/donate.wbur.org\/secure\/give?utm_source=wbur.org&utm_medium=web&utm_term=web_footer)\n- [Volunteer](https:\/\/www.wbur.org\/inside\/volunteer-opportunities)\n- [Member Services](https:\/\/www.wbur.org\/membership)\n- [Become A Member](https:\/\/www.wbur.org\/membership\/605748\/members)\n- [Donate Your Car](https:\/\/www.wbur.org\/membership\/605928\/donate-your-car)\n- [Join The Murrow Society](https:\/\/www.wbur.org\/murrowsociety)\n- [Become A Sponsor](https:\/\/www.wbur.org\/sponsorship\/about)\n\n#### Follow\n- [Facebook](https:\/\/www.facebook.com\/wburnews)\n- [Instagram](https:\/\/www.instagram.com\/wbur\/)\n- [YouTube](https:\/\/www.youtube.com\/user\/WBUR)\n- [Linkedin](https:\/\/www.linkedin.com\/company\/wbur)\n\n- [© Copyright WBUR 2026](https:\/\/www.bu.edu\/tech\/services\/security\/cyber-security\/dmca\/copyright\/)\n- [WBUR Privacy Policy](https:\/\/www.wbur.org\/inside\/privacy-policy)\n- [Syndication](https:\/\/www.wbur.org\/inside\/syndication)\n- [FCC Applications](https:\/\/media.wbur.org\/wp\/2021\/12\/WBUR-FM-FCC-Applications-Tab-01568791xB3D1E.pdf)\n- [FCC Public File](https:\/\/publicfiles.fcc.gov\/fm-profile\/WBUR-FM)\n- [WBUR EEO Reports](https:\/\/publicfiles.fcc.gov\/fm-profile\/WBUR-FM\/equal-employment-opportunity-records\/additional-documents\/eeo-public-file-reports\/137e0968-7b62-c93a-d484-71c68fd6373e)\n- [Public File Assistance](https:\/\/www.wbur.org\/inside\/fcc-public-file-information)\n\nThis site is protected by reCAPTCHA and the Google [Privacy Policy](https:\/\/policies.google.com\/privacy) and [Terms of Service](https:\/\/policies.google.com\/terms) apply.","attrs_readable_markdown":"Support WBUR\n\n![Britain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP)](https:\/\/static.wbur.org\/images\/npr-logo.svg)\n\nBritain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP)\n\nSupport WBUR\n\nSupport WBUR\n\nListen Live","meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

55 minutes ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://www.wbur.org/npr/131351976/prince-william-to-marry-long-time-girlfriend-kate-middleton
Last Crawled	2026-04-15 04:39:51 (55 minutes ago)
First Indexed	2018-12-05 16:35:29 (7 years ago)
HTTP Status Code	200
Meta Title	Prince William To Marry Longtime Girlfriend Kate Middleton \| WBUR
Meta Description	Royal officials say the prince and Kate Middleton got engaged last month during a vacation in Kenya. They are due to marry next spring or summer in London.
Meta Canonical	null
Boilerpipe Text	Support WBUR Britain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP) Support WBUR Support WBUR Listen Live
Markdown	[Skip to main content](https://www.wbur.org/npr/131351976/prince-william-to-marry-long-time-girlfriend-kate-middleton#main) [WBUR90.0 WBUR - Boston's NPR News Station](https://www.wbur.org/) Listen Live: Loading... [Donate](https://donate.wbur.org/secure/heart-wbur-homepage?utm_source=wbur.org&utm_medium=heart) [WBUR90.0 WBUR - Boston's NPR News Station](https://www.wbur.org/) Listen Live: Loading... [Donate](https://donate.wbur.org/secure/heart-wbur-homepage?utm_source=wbur.org&utm_medium=heart) Listen Live #### Sections - [Local Coverage](https://www.wbur.org/news) - [Arts & Culture](https://www.wbur.org/news/section/arts-culture) - [Business](https://www.wbur.org/news/section/business) - [Education](https://www.wbur.org/news/section/education) - [Environment](https://www.wbur.org/news/section/environment) - [Field Guide To Boston](https://www.wbur.org/news/section/field-guide-to-boston) - [Health](https://www.wbur.org/news/section/health) - [Investigations](https://www.wbur.org/news/section/investigations) - [Cognoscenti](https://www.wbur.org/opinion/section/cognoscenti) - [Boston News Quiz](https://www.wbur.org/quiz) #### Radio - [All Radio Programs](https://www.wbur.org/radio) - [All Things Considered](https://www.wbur.org/radio/programs/all-things-considered) - [Here & Now](https://www.wbur.org/radio/programs/hereandnow) - [Morning Edition](https://www.wbur.org/radio/programs/morningedition) - [On Point](https://www.wbur.org/radio/programs/onpoint) - [On-Air Schedule](https://www.wbur.org/radio/schedule) - [Ways To Listen](https://www.wbur.org/ways-to-listen) #### Podcasts - [All Podcasts](https://www.wbur.org/podcasts) - [Beyond All Repair](https://www.wbur.org/podcasts/violation) - [Circle Round](https://www.wbur.org/podcasts/circleround) - [Endless Thread](https://www.wbur.org/podcasts/endlessthread) - [Last Seen](https://www.wbur.org/podcasts/lastseen) - [The Midnight Rebellion](https://podcasts.wbur.org/the-midnight-rebellion) - [Violation](https://www.wbur.org/podcasts/violation/the-marshall-project-wbur-violation) #### Events - [CitySpace Calendar](https://www.wbur.org/events) - [Watch Past Events](https://www.wbur.org/events/browse?archive) - [Events Newsletter](https://www.wbur.org/events/newsletter) - [Rent CitySpace](https://www.rentcityspace.com/) - [The WBUR Festival](https://www.wburfestival.org/) #### Support - [Member Services](https://www.wbur.org/membership) - [Become A Member](https://www.wbur.org/membership/605748/members) - [Donate Your Car](https://www.wbur.org/membership/605928/donate-your-car) - [Join The Murrow Society](https://www.wbur.org/murrowsociety) - [Make A Donation](https://donate.wbur.org/secure/give?utm_source=wbur.org&utm_medium=web&utm_term=hamburger_menu) - [Become A Sponsor](https://www.wbur.org/sponsorship/about) - [Create A Custom Podcast](https://www.wbur.org/inside/cityspaceproductions) Support WBUR # Prince William To Marry Longtime Girlfriend Kate Middleton - [JJ Sutherland](https://npr.org/people/5239626/jj-sutherland) November 16, 2010 Share ![Britain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP)](https://static.wbur.org/images/npr-logo.svg) Britain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP) Support WBUR Support WBUR Resume Listen Live [Loading...](https://www.wbur.org/) #### Contact Us (617) 353-0909 [info@wbur.org](mailto:info@wbur.org) 890 Commonwealth Ave. Boston, MA 02215 [More ways to get in touch.](https://www.wbur.org/inside/contact) #### About WBUR - [Who We Are](https://www.wbur.org/about) - [Inside WBUR](https://www.wbur.org/inside) - [Diversity, Equity And Inclusion](https://www.wbur.org/inside/2021/11/15/diversity-equity-and-inclusion) - [Board Of Directors](https://www.wbur.org/inside/board-of-directors) - [Community Advisory Board](https://www.wbur.org/inside/community-advisory-board) - [CPB Transparency](https://www.wbur.org/inside/cpb-transparency) - [Careers](https://www.wbur.org/inside/jobs) - [WBUR Staff](https://www.wbur.org/inside/staff) - [Ethics Guide](https://www.wbur.org/inside/2022/03/10/ethics-guide) - [Licensing WBUR Content](https://www.wbur.org/inside/licensing-wbur-content) #### Support WBUR - [Make A Donation](https://donate.wbur.org/secure/give?utm_source=wbur.org&utm_medium=web&utm_term=web_footer) - [Volunteer](https://www.wbur.org/inside/volunteer-opportunities) - [Member Services](https://www.wbur.org/membership) - [Become A Member](https://www.wbur.org/membership/605748/members) - [Donate Your Car](https://www.wbur.org/membership/605928/donate-your-car) - [Join The Murrow Society](https://www.wbur.org/murrowsociety) - [Become A Sponsor](https://www.wbur.org/sponsorship/about) #### Follow - [Facebook](https://www.facebook.com/wburnews) - [Instagram](https://www.instagram.com/wbur/) - [YouTube](https://www.youtube.com/user/WBUR) - [Linkedin](https://www.linkedin.com/company/wbur) - [© Copyright WBUR 2026](https://www.bu.edu/tech/services/security/cyber-security/dmca/copyright/) - [WBUR Privacy Policy](https://www.wbur.org/inside/privacy-policy) - [Syndication](https://www.wbur.org/inside/syndication) - [FCC Applications](https://media.wbur.org/wp/2021/12/WBUR-FM-FCC-Applications-Tab-01568791xB3D1E.pdf) - [FCC Public File](https://publicfiles.fcc.gov/fm-profile/WBUR-FM) - [WBUR EEO Reports](https://publicfiles.fcc.gov/fm-profile/WBUR-FM/equal-employment-opportunity-records/additional-documents/eeo-public-file-reports/137e0968-7b62-c93a-d484-71c68fd6373e) - [Public File Assistance](https://www.wbur.org/inside/fcc-public-file-information) This site is protected by reCAPTCHA and the Google [Privacy Policy](https://policies.google.com/privacy) and [Terms of Service](https://policies.google.com/terms) apply.
Readable Markdown	Support WBUR ![Britain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP)](https://static.wbur.org/images/npr-logo.svg) Britain's Prince William and Kate Middleton, pictured in 2008 in Cranwell, England, met more than eight years ago while both were students at the University of St. Andrews in Scotland. (AP) Support WBUR Support WBUR Listen Live
Shard	56 (laksa)
Root Hash	15087967046536411256
Unparsed URL	org,wbur!www,/npr/131351976/prince-william-to-marry-long-time-girlfriend-kate-middleton s443