🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 143 (from laksa003)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa143.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical, ml_categories_json AS ml_categories_json, ml_types_json AS ml_types_json, ml_intent_types_json AS ml_intent_types_json, meta_language AS meta_language, attrs_author AS attrs_author, ifNull(toUnixTimestamp(attrs_publish_time), 0) AS attrs_publish_time, ifNull(toUnixTimestamp(attrs_original_publish_time), 0) AS attrs_original_publish_time, ifNull(attrs_is_republished, 0) AS attrs_is_republished, ifNull(attrs_nr_words, 0) AS attrs_nr_words, ifNull(attrs_boilerpipe_nr_words, 0) AS attrs_boilerpipe_nr_words, ifNull(body_ext_links_number, 0) AS body_ext_links_number, ifNull(body_int_links_number, 0) AS body_int_links_number, ifNull(meta_nofollow, 0) AS meta_nofollow, ifNull(meta_noarchive, 0) AS meta_noarchive, ifNull(props_was_rendered, 0) AS props_was_rendered, ifNull(src_redirect, \'\') AS src_redirect, ifNull(download_time_msec, 0) AS download_time_msec, ifNull(download_ttfb_msec, 0) AS download_ttfb_msec, ifNull(download_size, 0) AS download_size FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://srcole.github.io/2016/08/17/olympics/\')), getAhrefsUnparsedNoserviceFromURL(\'https://srcole.github.io/2016/08/17/olympics/\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/srcole.github.io\/2016\/08\/17\/olympics\/","crawl_time":1773927002,"first_indexed_time":1471619599,"http_code":200,"src_unparsed":"io,github!srcole,\/2016\/08\/17\/olympics\/ s443","src_root_hash":"2566890010099092343","history_drop_reason":null,"meta_title":"Which country is winning the 2016 Olympic games?: A Tableau Visualization","meta_descriptions":["Interactive visualization to set weights to each medal category to visualize performance across the globe. Playing around with data visualization in Tableau Public using the Rio Summer 2016 Olympic medals dataset."],"attrs_boilerpipe_text":"August 17, 2016\nPython code for data acquisition\nRelated visualization: Olympics results normalized by sports\nWeighting gold vs. silver vs. bronze medals\nAfter each Olympic games, there’s often a debate of which country “won” the Olympics. This debate is often between\npeople who disagree on whether the total number of medals or the number of gold medals is most important. \nPerhaps for others the answer lies somewhere in between, and using the interactive data visualization below, we can\nweight the value of each medal and see how that affects each country’s standing.\nIn this visualization, you can:\nVary the weights of gold, silver, and bronze medals.\nSelect a country on the map: see in the bar chart how their medal count faired in each sport.\nSelect a sport below the bar chart: see the weighted medal score for each country for only that sport.\nSelect a certain medal type (gold, silver, bronze) in a given sport’s bar: see the worldwide distribution of a certain medal type.\nNote that the ‘Independent Olympic Athletes’ were arbitrarily mapped onto Greenland.","attrs_markdown":"# Scott Cole\n## My personal website\n[Home](https:\/\/srcole.github.io\/) [Burritos of San Diego](https:\/\/srcole.github.io\/100burritos) [Resume](https:\/\/srcole.github.io\/assets\/misc\/resume.pdf) [Data Blog](https:\/\/srcole.github.io\/datablog) [Blog](https:\/\/srcole.github.io\/nondatablog)\n\n# Which country is winning the 2016 Olympic games?: A Tableau Visualization\nAugust 17, 2016\n***\n**[Python code for data acquisition](https:\/\/github.com\/srcole\/qwm\/tree\/master\/olympics)**\n\n**[Related visualization: Olympics results normalized by sports](https:\/\/srcole.github.io\/2016\/08\/20\/olympicssports\/)**\n\n## Weighting gold vs. silver vs. bronze medals\nAfter each Olympic games, there’s often a debate of which country “won” the Olympics. This debate is often between people who disagree on whether the total number of medals or the number of gold medals is most important. Perhaps for others the answer lies somewhere in between, and using the interactive data visualization below, we can weight the value of each medal and see how that affects each country’s standing.\n\n[![Dashboard 1 ](https:\/\/public.tableau.com\/static\/images\/ZR\/ZRMSG847R\/1_rss.png)](https:\/\/srcole.github.io\/2016\/08\/17\/olympics\/)\n\nIn this visualization, you can:\n\n1. Vary the weights of gold, silver, and bronze medals.\n2. Select a country on the map: see in the bar chart how their medal count faired in each sport.\n3. Select a sport below the bar chart: see the weighted medal score for each country for only that sport.\n4. Select a certain medal type (gold, silver, bronze) in a given sport’s bar: see the worldwide distribution of a certain medal type.\n\nNote that the ‘Independent Olympic Athletes’ were arbitrarily mapped onto Greenland.","attrs_readable_markdown":"August 17, 2016\n***\n**[Python code for data acquisition](https:\/\/github.com\/srcole\/qwm\/tree\/master\/olympics)**\n\n**[Related visualization: Olympics results normalized by sports](https:\/\/srcole.github.io\/2016\/08\/20\/olympicssports\/)**\n\n## Weighting gold vs. silver vs. bronze medals\nAfter each Olympic games, there’s often a debate of which country “won” the Olympics. This debate is often between people who disagree on whether the total number of medals or the number of gold medals is most important. Perhaps for others the answer lies somewhere in between, and using the interactive data visualization below, we can weight the value of each medal and see how that affects each country’s standing.\n\nIn this visualization, you can:\n\n1. Vary the weights of gold, silver, and bronze medals.\n2. Select a country on the map: see in the bar chart how their medal count faired in each sport.\n3. Select a sport below the bar chart: see the weighted medal score for each country for only that sport.\n4. Select a certain medal type (gold, silver, bronze) in a given sport’s bar: see the worldwide distribution of a certain medal type.\n\nNote that the ‘Independent Olympic Athletes’ were arbitrarily mapped onto Greenland.","meta_canonical":null,"ml_categories_json":"{\"\/Sports\":917,\"\/Sports\/International_Sports_Competitions\":533,\"\/Sports\/International_Sports_Competitions\/Olympics\":411,\"\/Computers_and_Electronics\":315,\"\/Computers_and_Electronics\/Software\":305,\"\/Science\":303,\"\/Science\/Computer_Science\":169,\"\/Computers_and_Electronics\/Software\/Software_Utilities\":129}","ml_types_json":"{\"\/Interactive_Tools\":507,\"\/Interactive_Tools\/Map\":295}","ml_intent_types_json":"{\"Informational\":996}","meta_language":"en-us","attrs_author":null,"attrs_publish_time":1471392000,"attrs_original_publish_time":1471392000,"attrs_is_republished":0,"attrs_nr_words":"231","attrs_boilerpipe_nr_words":"177","body_ext_links_number":5,"body_int_links_number":9,"meta_nofollow":0,"meta_noarchive":0,"props_was_rendered":1,"src_redirect":"","download_time_msec":57,"download_ttfb_msec":56,"download_size":2934}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

1 month ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	1.2 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property

Value

URL

https://srcole.github.io/2016/08/17/olympics/

Last Crawled

2026-03-19 13:30:02 (1 month ago)

First Indexed

2016-08-19 15:13:19 (9 years ago)

HTTP Status Code

200

Content

Meta Title

Which country is winning the 2016 Olympic games?: A Tableau Visualization

Meta Description

Interactive visualization to set weights to each medal category to visualize performance across the globe. Playing around with data visualization in Tableau Public using the Rio Summer 2016 Olympic medals dataset.

Meta Canonical

null

Boilerpipe Text

August 17, 2016 Python code for data acquisition Related visualization: Olympics results normalized by sports Weighting gold vs. silver vs. bronze medals After each Olympic games, there’s often a debate of which country “won” the Olympics. This debate is often between people who disagree on whether the total number of medals or the number of gold medals is most important. Perhaps for others the answer lies somewhere in between, and using the interactive data visualization below, we can weight the value of each medal and see how that affects each country’s standing. In this visualization, you can: Vary the weights of gold, silver, and bronze medals. Select a country on the map: see in the bar chart how their medal count faired in each sport. Select a sport below the bar chart: see the weighted medal score for each country for only that sport. Select a certain medal type (gold, silver, bronze) in a given sport’s bar: see the worldwide distribution of a certain medal type. Note that the ‘Independent Olympic Athletes’ were arbitrarily mapped onto Greenland.

Markdown

# Scott Cole ## My personal website [Home](https://srcole.github.io/) [Burritos of San Diego](https://srcole.github.io/100burritos) [Resume](https://srcole.github.io/assets/misc/resume.pdf) [Data Blog](https://srcole.github.io/datablog) [Blog](https://srcole.github.io/nondatablog) # Which country is winning the 2016 Olympic games?: A Tableau Visualization August 17, 2016 *** **[Python code for data acquisition](https://github.com/srcole/qwm/tree/master/olympics)** **[Related visualization: Olympics results normalized by sports](https://srcole.github.io/2016/08/20/olympicssports/)** ## Weighting gold vs. silver vs. bronze medals After each Olympic games, there’s often a debate of which country “won” the Olympics. This debate is often between people who disagree on whether the total number of medals or the number of gold medals is most important. Perhaps for others the answer lies somewhere in between, and using the interactive data visualization below, we can weight the value of each medal and see how that affects each country’s standing. [![Dashboard 1 ](https://public.tableau.com/static/images/ZR/ZRMSG847R/1_rss.png)](https://srcole.github.io/2016/08/17/olympics/) In this visualization, you can: 1. Vary the weights of gold, silver, and bronze medals. 2. Select a country on the map: see in the bar chart how their medal count faired in each sport. 3. Select a sport below the bar chart: see the weighted medal score for each country for only that sport. 4. Select a certain medal type (gold, silver, bronze) in a given sport’s bar: see the worldwide distribution of a certain medal type. Note that the ‘Independent Olympic Athletes’ were arbitrarily mapped onto Greenland.

Readable Markdown

August 17, 2016 *** **[Python code for data acquisition](https://github.com/srcole/qwm/tree/master/olympics)** **[Related visualization: Olympics results normalized by sports](https://srcole.github.io/2016/08/20/olympicssports/)** ## Weighting gold vs. silver vs. bronze medals After each Olympic games, there’s often a debate of which country “won” the Olympics. This debate is often between people who disagree on whether the total number of medals or the number of gold medals is most important. Perhaps for others the answer lies somewhere in between, and using the interactive data visualization below, we can weight the value of each medal and see how that affects each country’s standing. In this visualization, you can: 1. Vary the weights of gold, silver, and bronze medals. 2. Select a country on the map: see in the bar chart how their medal count faired in each sport. 3. Select a sport below the bar chart: see the weighted medal score for each country for only that sport. 4. Select a certain medal type (gold, silver, bronze) in a given sport’s bar: see the worldwide distribution of a certain medal type. Note that the ‘Independent Olympic Athletes’ were arbitrarily mapped onto Greenland.

ML Classification

ML Categories

/Sports		91.7%
/Sports/International_Sports_Competitions		53.3%
/Sports/International_Sports_Competitions/Olympics		41.1%
/Computers_and_Electronics		31.5%
/Computers_and_Electronics/Software		30.5%
/Science		30.3%
/Science/Computer_Science		16.9%
/Computers_and_Electronics/Software/Software_Utilities		12.9%

Raw JSON

{
    "/Sports": 917,
    "/Sports/International_Sports_Competitions": 533,
    "/Sports/International_Sports_Competitions/Olympics": 411,
    "/Computers_and_Electronics": 315,
    "/Computers_and_Electronics/Software": 305,
    "/Science": 303,
    "/Science/Computer_Science": 169,
    "/Computers_and_Electronics/Software/Software_Utilities": 129
}

ML Page Types

/Interactive_Tools		50.7%
/Interactive_Tools/Map		29.5%

Raw JSON

{
    "/Interactive_Tools": 507,
    "/Interactive_Tools/Map": 295
}

ML Intent Types

Informational

99.6%

Raw JSON

{
    "Informational": 996
}

Content Metadata

Language

en-us

Author

null

Publish Time

2016-08-17 00:00:00 (9 years ago)

Original Publish Time

2016-08-17 00:00:00 (9 years ago)

Republished

Word Count (Total)

231

Word Count (Content)

177

Links

External Links

Internal Links

Technical SEO

Meta Nofollow

Meta Noarchive

JS Rendered

Yes

Redirect Target

null

Performance

Download Time (ms)

TTFB (ms)

Download Size (bytes)

2,934

Shard

143 (laksa)

Root Hash

2566890010099092343

Unparsed URL

io,github!srcole,/2016/08/17/olympics/ s443