🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:
Response:
Calculated Shard: 81 (from laksa127)

2. Crawled Status Check

Query:
Response:

3. Robots.txt Check

Query:
Response:

4. Spam/Ban Check

Query:
Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📍
LOCATION
Host 81 · Partition 2
laksa081
3090160618261780481
📄
INDEXABLE
CRAWLED
7 days ago
🤖
ROBOTS ALLOWED

Page Info Filters

FilterStatusConditionDetails
HTTP statusPASSdownload_http_code = 200HTTP 200
Age cutoffPASSdownload_stamp > now() - 6 MONTH0.3 months ago
History dropPASSisNull(history_drop_reason)No drop reason
Spam/banPASSfh_dont_index != 1 AND ml_spam_score = 0ml_spam_score=0
CanonicalPASSmeta_canonical IS NULL OR = '' OR = src_unparsedNot set

Page Details

PropertyValue
URLhttps://www.edx.org/school/harvardx
Last Crawled2026-05-26 14:06:33 (7 days ago)
First Indexed2013-12-03 19:18:58 (12 years ago)
HTTP Status Code200
Content
Meta TitleOnline Courses from Harvard University
Meta Description<p>Harvard University is devoted to excellence in teaching, learning, and research, and to developing leaders in many disciplines who make a difference globally. Harvard faculty are engaged with teaching and research to push the boundaries of human knowledge. The University has twelve degree-granting Schools in addition to the Radcliffe Institute for Advanced Study.</p><p>Established in 1636, Harvard is the oldest institution of higher education in the United States. The University, which is based in Cambridge and Boston, Massachusetts, has an enrollment of over 20,000 degree candidates, including undergraduate, graduate, and professional students. Harvard has more than 360,000 alumni around the world.</p>
Meta Canonicalnull
Boilerpipe Text
heavy column, fetched on demand
Markdown
heavy column, fetched on demand
Readable Markdown
heavy column, fetched on demand
ML Classification
ML Categories
/Jobs_and_Education
99.2%
/Jobs_and_Education/Education
99.2%
/Jobs_and_Education/Education/Distance_Learning
97.7%
Raw JSON
{
    "/Jobs_and_Education": 992,
    "/Jobs_and_Education/Education": 992,
    "/Jobs_and_Education/Education/Distance_Learning": 977
}
ML Page Types
/Listing_Collection
78.2%
/Listing_Collection/Product
58.1%
Raw JSON
{
    "/Listing_Collection": 782,
    "/Listing_Collection/Product": 581
}
ML Intent Types
Commercial
67.8%
Transactional
49.3%
Informational
11.8%
Raw JSON
{
    "Commercial": 678,
    "Transactional": 493,
    "Informational": 118
}
Content Metadata
Languageen
Authornull
Publish Timenot set
Original Publish Time2013-12-03 19:18:58 (12 years ago)
RepublishedNo
Word Count (Total)390
Word Count (Content)101
Links
External Links12
Internal Links72
Technical SEO
Meta NofollowNo
Meta NoarchiveNo
JS RenderedNo
Redirect Targetnull
Performance
Download Time (ms)1,890
TTFB (ms)1,861
Download Size (bytes)321,863
Location
Host ID81 (laksa081)
Partition ID2
Root Hash3090160618261780481
Unparsed URLorg,edx!www,/school/harvardx s443