🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:
Response:
Calculated Shard: 25 (from laksa136)

2. Crawled Status Check

Query:
Response:

3. Robots.txt Check

Query:
Response:

4. Spam/Ban Check

Query:
Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄
INDEXABLE
CRAWLED
3 days ago
🤖
ROBOTS ALLOWED

Page Info Filters

FilterStatusConditionDetails
HTTP statusPASSdownload_http_code = 200HTTP 200
Age cutoffPASSdownload_stamp > now() - 6 MONTH0.1 months ago
History dropPASSisNull(history_drop_reason)No drop reason
Spam/banPASSfh_dont_index != 1 AND ml_spam_score = 0ml_spam_score=0
CanonicalPASSmeta_canonical IS NULL OR = '' OR = src_unparsedNot set

Page Details

PropertyValue
URLhttps://beanrated.com/beans
Last Crawled2026-04-07 20:00:59 (3 days ago)
First Indexed2025-09-28 22:31:57 (6 months ago)
HTTP Status Code200
Meta TitleBean Rated - Coffee Reviews & Ratings
Meta DescriptionDiscover and rate the best coffee beans from around the world. Join our community of coffee enthusiasts.
Meta Canonicalnull
Boilerpipe Text
Bean Rated Coffee Reviews & Community
Markdown
[![Bean Rated Logo](https://beanrated.com/lovable-uploads/3e363821-0375-4778-8f0f-d14fbb190606.png)Bean RatedCoffee Reviews & Community](https://beanrated.com/) Sign In [![Home](https://beanrated.com/lovable-uploads/ca4f2ff9-ae6c-4206-bd96-73be8dc10d06.png)Home](https://beanrated.com/)[![Coffee Bean](https://beanrated.com/lovable-uploads/efcfb5c0-e10f-4d7d-b56a-ed7b3e86db82.png)Beans](https://beanrated.com/beans)[![Coffee Roaster](https://beanrated.com/lovable-uploads/7ec4a9c4-1836-471e-94e7-9610bf75f852.png)Roasters](https://beanrated.com/roasters)[![Café](https://beanrated.com/lovable-uploads/02b26320-cc90-40db-8b9b-b0aea153909d.png)Cafés](https://beanrated.com/cafes)[![Leaderboard](https://beanrated.com/lovable-uploads/67bdabbc-6d71-4bf5-9b3e-670fc6c27b98.png)Leaderboard](https://beanrated.com/leaderboard)
Readable Markdown
![Bean Rated Logo](https://beanrated.com/lovable-uploads/3e363821-0375-4778-8f0f-d14fbb190606.png) Bean RatedCoffee Reviews & Community
Shard25 (laksa)
Root Hash16961294494851972625
Unparsed URLcom,beanrated!/beans s443