Crawler Inspector

1. Shard Calculation

Query:

Response:

Calculated Shard: 137 (from laksa066)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa137.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://www.golflink.com/instruction/golf-for-beginners\')), getAhrefsUnparsedNoserviceFromURL(\'https://www.golflink.com/instruction/golf-for-beginners\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/www.golflink.com\/instruction\/golf-for-beginners","crawl_time":1729661472,"first_indexed_time":1706747037,"http_code":200,"src_unparsed":"com,golflink!www,\/instruction\/golf-for-beginners s443","src_root_hash":"4752384300683870537","history_drop_reason":"tooold","meta_title":"Golf For Beginners: 10 Tips For Newbie Golfers Golflink.com","meta_descriptions":["Learn how to start playing golf with these tips for beginners. You’ll be strutting to the first tee with confidence in no time!"],"attrs_boilerpipe_text":"I have been involved in golf for over four decades, first as a player, then as an industry professional for 28 years, and as a PGA Professional for 15 years. As a PGA Professional, it’s my job to introduce people to golf, grow their love for it, and guide them on their golf journey. It’s normal for new golfers to feel inadequate, apprehensive, and nervous. These are natural feelings when starting something new. Golf is a complex game that involves many principles, equipment, and rules. Golfers must master basic things such as posture, grip, and alignment; and advanced concepts like the ball flight laws. It can be daunting, but also very rewarding. To pile on, golf has a complex set of rules plus unofficial do's and don'ts that most golfers pick up along the way. When you add all that up, it’s no surprise that learning golf as a beginner can be overwhelming. As a professional who has been helping people enjoy and get better at golf for 28 years, I would like to share 10 valuable tips for beginner golfers. These tips are based on my own experience, and they will be helpful for anyone starting their golf journey.","attrs_markdown":null,"attrs_readable_markdown":null,"meta_canonical":null}

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	FAIL	`download_stamp > now() - 6 MONTH`	17.8 months ago
History drop	FAIL	`isNull(history_drop_reason)`	tooold
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Property	Value
URL	https://www.golflink.com/instruction/golf-for-beginners
Last Crawled	2024-10-23 05:31:12 (1 year ago)
First Indexed	2024-02-01 00:23:57 (2 years ago)
HTTP Status Code	200
Meta Title	Golf For Beginners: 10 Tips For Newbie Golfers Golflink.com
Meta Description	Learn how to start playing golf with these tips for beginners. You’ll be strutting to the first tee with confidence in no time!
Meta Canonical	null
Boilerpipe Text	I have been involved in golf for over four decades, first as a player, then as an industry professional for 28 years, and as a PGA Professional for 15 years. As a PGA Professional, it’s my job to introduce people to golf, grow their love for it, and guide them on their golf journey. It’s normal for new golfers to feel inadequate, apprehensive, and nervous. These are natural feelings when starting something new. Golf is a complex game that involves many principles, equipment, and rules. Golfers must master basic things such as posture, grip, and alignment; and advanced concepts like the ball flight laws. It can be daunting, but also very rewarding. To pile on, golf has a complex set of rules plus unofficial do's and don'ts that most golfers pick up along the way. When you add all that up, it’s no surprise that learning golf as a beginner can be overwhelming. As a professional who has been helping people enjoy and get better at golf for 28 years, I would like to share 10 valuable tips for beginner golfers. These tips are based on my own experience, and they will be helpful for anyone starting their golf journey.
Markdown	null
Readable Markdown	null
Shard	137 (laksa)
Root Hash	4752384300683870537
Unparsed URL	com,golflink!www,/instruction/golf-for-beginners s443

🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

1. Shard Calculation

Query:

Response:

2. Crawled Status Check

Query:

Response:

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

Page Info Filters

Page Details