Crawler Inspector

1. Shard Calculation

Query:

Response:

Calculated Shard: 45 (from laksa109)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa045.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://www.fdc.myflorida.com/institutions/visiting-information\')), getAhrefsUnparsedNoserviceFromURL(\'https://www.fdc.myflorida.com/institutions/visiting-information\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/www.fdc.myflorida.com\/institutions\/visiting-information","crawl_time":1766970796,"first_indexed_time":1718096061,"http_code":200,"src_unparsed":"com,myflorida!fdc,www,\/institutions\/visiting-information s443","src_root_hash":"1074884992542252045","history_drop_reason":null,"meta_title":"Florida Department of Corrections","meta_descriptions":["Florida Department of Corrections - Florida Department of Corrections"],"attrs_boilerpipe_text":"Florida Department of Corrections","attrs_markdown":"[Skip to Main Content](https:\/\/www.fdc.myflorida.com\/institutions\/visiting-information#main-content)\n\nSearch:\n\n![Florida Department of Corrections logo](https:\/\/www.fdc.myflorida.com\/_next\/static\/media\/FDC-logo_bw.0d299bc8.png)\n\nQUICK LINKS\n***\n\nCONTACT\n***\n[![Facebook icon](https:\/\/www.fdc.myflorida.com\/_next\/static\/media\/facebook.a2058c13.svg)](https:\/\/www.facebook.com\/FLCorrections)[![Instagram icon](https:\/\/www.fdc.myflorida.com\/_next\/static\/media\/instagram.806e811f.svg)](https:\/\/www.instagram.com\/fl_corrections\/)[![Youtube icon](https:\/\/www.fdc.myflorida.com\/_next\/static\/media\/youtube.07d420ea.svg)](https:\/\/www.youtube.com\/myfloridadoc)[![X (Formerly Twitter) icon](https:\/\/www.fdc.myflorida.com\/_next\/static\/media\/x-twitter.702ac2a0.svg)](https:\/\/twitter.com\/FL_Corrections)[![LinkedIn icon](https:\/\/www.fdc.myflorida.com\/_next\/static\/media\/linkedin.380b8d9e.svg)](https:\/\/www.linkedin.com\/company\/florida-department-of-corrections\/)\n\nFlorida Department of Corrections\n\n- [Home](https:\/\/www.fdc.myflorida.com\/)","attrs_readable_markdown":null,"meta_canonical":"com,myflorida!fdc,www,\/fdc\/institutions\/visiting-information s443"}

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	3.7 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	FAIL	`meta_canonical IS NULL OR = '' OR = src_unparsed`	com,myflorida!fdc,www,/fdc/institutions/visiting-information s443

Property	Value
URL	https://www.fdc.myflorida.com/institutions/visiting-information
Last Crawled	2025-12-29 01:13:16 (3 months ago)
First Indexed	2024-06-11 08:54:21 (1 year ago)
HTTP Status Code	200
Meta Title	Florida Department of Corrections
Meta Description	Florida Department of Corrections - Florida Department of Corrections
Meta Canonical	com,myflorida!fdc,www,/fdc/institutions/visiting-information s443
Boilerpipe Text	Florida Department of Corrections
Markdown	[Skip to Main Content](https://www.fdc.myflorida.com/institutions/visiting-information#main-content) Search: ![Florida Department of Corrections logo](https://www.fdc.myflorida.com/_next/static/media/FDC-logo_bw.0d299bc8.png) QUICK LINKS * CONTACT * [![Facebook icon](https://www.fdc.myflorida.com/_next/static/media/facebook.a2058c13.svg)](https://www.facebook.com/FLCorrections)[![Instagram icon](https://www.fdc.myflorida.com/_next/static/media/instagram.806e811f.svg)](https://www.instagram.com/fl_corrections/)[![Youtube icon](https://www.fdc.myflorida.com/_next/static/media/youtube.07d420ea.svg)](https://www.youtube.com/myfloridadoc)[![X (Formerly Twitter) icon](https://www.fdc.myflorida.com/_next/static/media/x-twitter.702ac2a0.svg)](https://twitter.com/FL_Corrections)[![LinkedIn icon](https://www.fdc.myflorida.com/_next/static/media/linkedin.380b8d9e.svg)](https://www.linkedin.com/company/florida-department-of-corrections/) Florida Department of Corrections - [Home](https://www.fdc.myflorida.com/)
Readable Markdown	null
Shard	45 (laksa)
Root Hash	1074884992542252045
Unparsed URL	com,myflorida!fdc,www,/institutions/visiting-information s443

🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

1. Shard Calculation

Query:

Response:

2. Crawled Status Check

Query:

Response:

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

Page Info Filters

Page Details