🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 144 (from laksa042)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa144.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://www.cs.usfca.edu/~galles/visualization/Algorithms.html\')), getAhrefsUnparsedNoserviceFromURL(\'https://www.cs.usfca.edu/~galles/visualization/Algorithms.html\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Algorithms.html","crawl_time":1776237678,"first_indexed_time":1393264663,"http_code":200,"src_unparsed":"edu,usfca!cs,www,\/~galles\/visualization\/Algorithms.html s443","src_root_hash":"10754122321078002144","history_drop_reason":null,"meta_title":"Data Structure Visualization","meta_descriptions":[],"attrs_boilerpipe_text":"Copyright 2011\nDavid Galles","attrs_markdown":"# Data Structure Visualizations\n- [About](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/about.html)\n- [Algorithms](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Algorithms.html)\n- [F.A.Q](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/faq.html)\n- [Known Bugs \/ Feature Requests](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/bugfeature.html)\n- [Java Version](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/java\/visualization.html)\n- [Flash Version](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/flash.html)\n- [Create Your Own \/  Source Code](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/source.html)\n- [Contact](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/contact.html)\n\n[David Galles](http:\/\/www.cs.usfca.edu\/galles)    \n  [Computer Science](http:\/\/www.cs.usfca.edu\/)    \n  [University of San Francisco](http:\/\/www.usfca.edu\/)\n\nCurrently, we have visualizations for the following data structures and algorithms:\n\n- Basics\n  - [Stack: Array Implementation](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/StackArray.html)\n  - [Stack: Linked List Implementation](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/StackLL.html)\n  - [Queues: Array Implementation](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/QueueArray.html)\n  - [Queues: Linked List Implementation](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/QueueLL.html)\n  - Lists: Array Implementation (available in [java](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/java\/visualization.html) version)\n  - Lists: Linked List Implementation (available in [java](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/java\/visualization.html) version)\n- Recursion\n  - [Factorial](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RecFact.html)\n  - [Reversing a String](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RecReverse.html)\n  - [N-Queens Problem](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RecQueens.html)\n- Indexing\n  - [Binary and Linear Search (of sorted list)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Search.html)\n  - [Binary Search Trees](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/BST.html)\n  - [AVL Trees (Balanced binary search trees)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/AVLtree.html)\n  - [Red-Black Trees](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RedBlack.html)\n  - [Splay Trees](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/SplayTree.html)\n  - [Open Hash Tables (Closed Addressing)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/OpenHash.html)\n  - [Closed Hash Tables (Open Addressing)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/ClosedHash.html)\n  - [Closed Hash Tables, using buckets](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/ClosedHashBucket.html)\n  - [Trie (Prefix Tree, 26-ary Tree)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Trie.html)\n  - [Radix Tree (Compact Trie)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RadixTree.html)\n  - [Ternary Search Tree (Trie with BST of children)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/TST.html)\n  - [B Trees](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/BTree.html)\n  - [B+ Trees](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/BPlusTree.html)\n- Sorting\n  - [Comparison Sorting](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/ComparisonSort.html)\n    - Bubble Sort\n    - Selection Sort\n    - Insertion Sort\n    - Shell Sort\n    - Merge Sort\n    - Quck Sort\n  - [Bucket Sort](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/BucketSort.html)\n  - [Counting Sort](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/CountingSort.html)\n  - [Radix Sort](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RadixSort.html)\n  - [Heap Sort](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/HeapSort.html)\n- Heap-like Data Structures\n  - [Heaps](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Heap.html)\n  - [Binomial Queues](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/BinomialQueue.html)\n  - [Fibonacci Heaps](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/FibonacciHeap.html)\n  - [Leftist Heaps](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/LeftistHeap.html)\n  - [Skew Heaps](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/SkewHeap.html)\n- Graph Algorithms\n  - [Breadth-First Search](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/BFS.html)\n  - [Depth-First Search](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/DFS.html)\n  - [Connected Components](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/ConnectedComponent.html)\n  - [Dijkstra's Shortest Path](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Dijkstra.html)\n  - [Prim's Minimum Cost Spanning Tree](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Prim.html)\n  - [Topological Sort (Using Indegree array)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/TopoSortIndegree.html)\n  - [Topological Sort (Using DFS)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/TopoSortDFS.html)\n  - [Floyd-Warshall (all pairs shortest paths)](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Floyd.html)\n  - [Kruskal Minimum Cost Spanning Tree Algorithm](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/Kruskal.html)\n- Dynamic Programming\n  - [Calculating nth Fibonacci number](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/DPFib.html)\n  - [Making Change](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/DPChange.html)\n  - [Longest Common Subsequence](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/DPLCS.html)\n- Geometric Algorithms\n  - [2D Rotation and Scale Matrices](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RotateScale2D.html)\n  - [2D Rotation and Translation Matrices](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RotateTranslate2D.html)\n  - [2D Changing Coordinate Systems](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/ChangingCoordinates2D.html)\n  - [3D Rotation and Scale Matrices](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/RotateScale3D.html)\n  - [3D Changing Coordinate Systems](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/ChangingCoordinates3D.html)\n- Others ...\n  - [Disjoint Sets](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/DisjointSets.html)\n  - Huffman Coding (available in [java](https:\/\/www.cs.usfca.edu\/~galles\/visualization\/java\/visualization.html) version)\n\nCopyright 2011 [David Galles](http:\/\/www.cs.usfca.edu\/galles)","attrs_readable_markdown":"Copyright 2011 [David Galles](http:\/\/www.cs.usfca.edu\/galles)","meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

1 hour ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://www.cs.usfca.edu/~galles/visualization/Algorithms.html
Last Crawled	2026-04-15 07:21:18 (1 hour ago)
First Indexed	2014-02-24 17:57:43 (12 years ago)
HTTP Status Code	200
Meta Title	Data Structure Visualization
Meta Description	null
Meta Canonical	null
Boilerpipe Text	Copyright 2011 David Galles
Markdown	# Data Structure Visualizations - [About](https://www.cs.usfca.edu/~galles/visualization/about.html) - [Algorithms](https://www.cs.usfca.edu/~galles/visualization/Algorithms.html) - [F.A.Q](https://www.cs.usfca.edu/~galles/visualization/faq.html) - [Known Bugs / Feature Requests](https://www.cs.usfca.edu/~galles/visualization/bugfeature.html) - [Java Version](https://www.cs.usfca.edu/~galles/visualization/java/visualization.html) - [Flash Version](https://www.cs.usfca.edu/~galles/visualization/flash.html) - [Create Your Own / Source Code](https://www.cs.usfca.edu/~galles/visualization/source.html) - [Contact](https://www.cs.usfca.edu/~galles/visualization/contact.html) [David Galles](http://www.cs.usfca.edu/galles) [Computer Science](http://www.cs.usfca.edu/) [University of San Francisco](http://www.usfca.edu/) Currently, we have visualizations for the following data structures and algorithms: - Basics - [Stack: Array Implementation](https://www.cs.usfca.edu/~galles/visualization/StackArray.html) - [Stack: Linked List Implementation](https://www.cs.usfca.edu/~galles/visualization/StackLL.html) - [Queues: Array Implementation](https://www.cs.usfca.edu/~galles/visualization/QueueArray.html) - [Queues: Linked List Implementation](https://www.cs.usfca.edu/~galles/visualization/QueueLL.html) - Lists: Array Implementation (available in [java](https://www.cs.usfca.edu/~galles/visualization/java/visualization.html) version) - Lists: Linked List Implementation (available in [java](https://www.cs.usfca.edu/~galles/visualization/java/visualization.html) version) - Recursion - [Factorial](https://www.cs.usfca.edu/~galles/visualization/RecFact.html) - [Reversing a String](https://www.cs.usfca.edu/~galles/visualization/RecReverse.html) - [N-Queens Problem](https://www.cs.usfca.edu/~galles/visualization/RecQueens.html) - Indexing - [Binary and Linear Search (of sorted list)](https://www.cs.usfca.edu/~galles/visualization/Search.html) - [Binary Search Trees](https://www.cs.usfca.edu/~galles/visualization/BST.html) - [AVL Trees (Balanced binary search trees)](https://www.cs.usfca.edu/~galles/visualization/AVLtree.html) - [Red-Black Trees](https://www.cs.usfca.edu/~galles/visualization/RedBlack.html) - [Splay Trees](https://www.cs.usfca.edu/~galles/visualization/SplayTree.html) - [Open Hash Tables (Closed Addressing)](https://www.cs.usfca.edu/~galles/visualization/OpenHash.html) - [Closed Hash Tables (Open Addressing)](https://www.cs.usfca.edu/~galles/visualization/ClosedHash.html) - [Closed Hash Tables, using buckets](https://www.cs.usfca.edu/~galles/visualization/ClosedHashBucket.html) - [Trie (Prefix Tree, 26-ary Tree)](https://www.cs.usfca.edu/~galles/visualization/Trie.html) - [Radix Tree (Compact Trie)](https://www.cs.usfca.edu/~galles/visualization/RadixTree.html) - [Ternary Search Tree (Trie with BST of children)](https://www.cs.usfca.edu/~galles/visualization/TST.html) - [B Trees](https://www.cs.usfca.edu/~galles/visualization/BTree.html) - [B+ Trees](https://www.cs.usfca.edu/~galles/visualization/BPlusTree.html) - Sorting - [Comparison Sorting](https://www.cs.usfca.edu/~galles/visualization/ComparisonSort.html) - Bubble Sort - Selection Sort - Insertion Sort - Shell Sort - Merge Sort - Quck Sort - [Bucket Sort](https://www.cs.usfca.edu/~galles/visualization/BucketSort.html) - [Counting Sort](https://www.cs.usfca.edu/~galles/visualization/CountingSort.html) - [Radix Sort](https://www.cs.usfca.edu/~galles/visualization/RadixSort.html) - [Heap Sort](https://www.cs.usfca.edu/~galles/visualization/HeapSort.html) - Heap-like Data Structures - [Heaps](https://www.cs.usfca.edu/~galles/visualization/Heap.html) - [Binomial Queues](https://www.cs.usfca.edu/~galles/visualization/BinomialQueue.html) - [Fibonacci Heaps](https://www.cs.usfca.edu/~galles/visualization/FibonacciHeap.html) - [Leftist Heaps](https://www.cs.usfca.edu/~galles/visualization/LeftistHeap.html) - [Skew Heaps](https://www.cs.usfca.edu/~galles/visualization/SkewHeap.html) - Graph Algorithms - [Breadth-First Search](https://www.cs.usfca.edu/~galles/visualization/BFS.html) - [Depth-First Search](https://www.cs.usfca.edu/~galles/visualization/DFS.html) - [Connected Components](https://www.cs.usfca.edu/~galles/visualization/ConnectedComponent.html) - [Dijkstra's Shortest Path](https://www.cs.usfca.edu/~galles/visualization/Dijkstra.html) - [Prim's Minimum Cost Spanning Tree](https://www.cs.usfca.edu/~galles/visualization/Prim.html) - [Topological Sort (Using Indegree array)](https://www.cs.usfca.edu/~galles/visualization/TopoSortIndegree.html) - [Topological Sort (Using DFS)](https://www.cs.usfca.edu/~galles/visualization/TopoSortDFS.html) - [Floyd-Warshall (all pairs shortest paths)](https://www.cs.usfca.edu/~galles/visualization/Floyd.html) - [Kruskal Minimum Cost Spanning Tree Algorithm](https://www.cs.usfca.edu/~galles/visualization/Kruskal.html) - Dynamic Programming - [Calculating nth Fibonacci number](https://www.cs.usfca.edu/~galles/visualization/DPFib.html) - [Making Change](https://www.cs.usfca.edu/~galles/visualization/DPChange.html) - [Longest Common Subsequence](https://www.cs.usfca.edu/~galles/visualization/DPLCS.html) - Geometric Algorithms - [2D Rotation and Scale Matrices](https://www.cs.usfca.edu/~galles/visualization/RotateScale2D.html) - [2D Rotation and Translation Matrices](https://www.cs.usfca.edu/~galles/visualization/RotateTranslate2D.html) - [2D Changing Coordinate Systems](https://www.cs.usfca.edu/~galles/visualization/ChangingCoordinates2D.html) - [3D Rotation and Scale Matrices](https://www.cs.usfca.edu/~galles/visualization/RotateScale3D.html) - [3D Changing Coordinate Systems](https://www.cs.usfca.edu/~galles/visualization/ChangingCoordinates3D.html) - Others ... - [Disjoint Sets](https://www.cs.usfca.edu/~galles/visualization/DisjointSets.html) - Huffman Coding (available in [java](https://www.cs.usfca.edu/~galles/visualization/java/visualization.html) version) Copyright 2011 [David Galles](http://www.cs.usfca.edu/galles)
Readable Markdown	Copyright 2011 [David Galles](http://www.cs.usfca.edu/galles)
Shard	144 (laksa)
Root Hash	10754122321078002144
Unparsed URL	edu,usfca!cs,www,/~galles/visualization/Algorithms.html s443