🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 104 (from laksa005)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa104.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://emg.nysbc.org/issues/67\')), getAhrefsUnparsedNoserviceFromURL(\'https://emg.nysbc.org/issues/67\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/emg.nysbc.org\/issues\/67","crawl_time":1760575287,"first_indexed_time":1714639205,"http_code":200,"src_unparsed":"org,nysbc!emg,\/issues\/67 s443","src_root_hash":"6313026282925610904","history_drop_reason":null,"meta_title":"Bug #67: Python imports, absolute vs. relative - Leginon - Electron Microscopy Group","meta_descriptions":["Redmine"],"attrs_boilerpipe_text":"We should be using absolute imports instead of relative imports when possible.  This means for instance: import leginon.leginondata rather than import leginondata There are a few reasons why explained in  PEP 328 Python 3 will require either absolute imports or relative imports using a new syntax.  This means that absolute imports will be the best way to have something that works now with Python 2 and in the future with Python 3.  It is also a bad idea to customize PYTHONPATH or add path modifiers like Leginon.pth as a shortcut to not using absolute paths.  In all cases, there is risk of conflicting module names between different packages. There are a few parts to resolving this in myami: - Change import statements throughout myami to absolute module names.  I think it should be OK to leave alone relative imports if it is a module importing another module in the exact same directory, but this will require syntax revision for Python 3.  Scripts in appion\/bin should use \"import appionlib.xxxxx\" - Other references to module names in myami should start using absolute module names.  For instance, the field names in the database like \"REF|leginondata|AcquisitionImageData|image\". sinedon.cfg contains just a simple module name rather than absolute name.  Noderegistry in Leginon had relative modules, but that has been fixed.","attrs_markdown":"### Project\n### General\n### Profile\n\n- [Sign in](https:\/\/emg.nysbc.org\/login)\n- [Register](https:\/\/emg.nysbc.org\/account\/register)\n\n- [Home](https:\/\/emg.nysbc.org\/)\n- [Projects](https:\/\/emg.nysbc.org\/projects)\n- [Help](https:\/\/www.redmine.org\/guide)\n\nLeginon\n\n[All Projects](https:\/\/emg.nysbc.org\/projects?jump=issues)\n# Leginon\n- [Overview](https:\/\/emg.nysbc.org\/projects\/leginon)\n- [Activity](https:\/\/emg.nysbc.org\/projects\/leginon\/activity)\n- [Roadmap](https:\/\/emg.nysbc.org\/projects\/leginon\/roadmap)\n- [Issues](https:\/\/emg.nysbc.org\/projects\/leginon\/issues)\n- [News](https:\/\/emg.nysbc.org\/projects\/leginon\/news)\n- [Documents](https:\/\/emg.nysbc.org\/projects\/leginon\/documents)\n- [Wiki](https:\/\/emg.nysbc.org\/projects\/leginon\/wiki)\n- [Forums](https:\/\/emg.nysbc.org\/projects\/leginon\/boards)\n- [Files](https:\/\/emg.nysbc.org\/projects\/leginon\/files)\n\n### Custom queries\n- [Appion\/Leginon 3.0 bug fixes](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=30)\n- [Leginon 3.2 Bug Fixes](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=47)\n- [Leginon 3.3 Bug Fixes](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=53)\n- [Leginon 3.4 Bug Fixes](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=55)\n- [Leginon 3.4 Completed Features](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=57)\n- [Leginon 3.5 Bug Fixes](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=61)\n- [Leginon 3.5 Completed Features](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=60)\n- [Leginon 3.6 Bug Fixes](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=72)\n- [Leginon 3.6 Completed Features](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=71)\n- [Leginon Known Bugs](https:\/\/emg.nysbc.org\/projects\/leginon\/issues?query_id=48)\n\nActions\n\n[Copy link](https:\/\/emg.nysbc.org\/issues\/67)\n## Bug \\#67\nopen\n\n### Python imports, absolute vs. relative\nAdded by [Jim Pulokas](https:\/\/emg.nysbc.org\/users\/3) [over 15 years](https:\/\/emg.nysbc.org\/projects\/leginon\/activity?from=2010-01-28 \"01\/28\/2010 02:14 PM\") ago. Updated [almost 12 years](https:\/\/emg.nysbc.org\/projects\/leginon\/activity?from=2013-10-21 \"10\/21\/2013 03:17 PM\") ago.\n\nStatus:\n\nAssigned\n\nPriority:\n\nLow\n\nAssignee:\n\n[Anchi Cheng](https:\/\/emg.nysbc.org\/users\/9)\n\nCategory:\n\n\\-\n\nTarget version:\n\n[Appion - Appion\/Leginon Future Version](https:\/\/emg.nysbc.org\/versions\/15)\n\nStart date:\n\n01\/28\/2010\n\nDue date:\n\n% Done:\n\n30%\n\nEstimated time:\n\nAffected Version:\n\nAppion\/Leginon 2.0.1\n\nShow in known bugs:\n\nNo\n\nWorkaround:\n***\n**Description**\n\nWe should be using absolute imports instead of relative imports when possible. This means for instance:\n```\nimport leginon.leginondata\n```\nrather than\n```\nimport leginondata\n```\nThere are a few reasons why explained in [PEP 328](http:\/\/www.python.org\/dev\/peps\/pep-0328\/)  \nPython 3 will require either absolute imports or relative imports using a new syntax. This means that absolute imports will be the best way to have something that works now with Python 2 and in the future with Python 3. It is also a bad idea to customize PYTHONPATH or add path modifiers like Leginon.pth as a shortcut to not using absolute paths. In all cases, there is risk of conflicting module names between different packages.\n\nThere are a few parts to resolving this in myami:\n\n\\- Change import statements throughout myami to absolute module names. I think it should be OK to leave alone relative imports if it is a module importing another module in the exact same directory, but this will require syntax revision for Python 3. Scripts in appion\/bin should use \"import appionlib.xxxxx\"\n\n\\- Other references to module names in myami should start using absolute module names. For instance, the field names in the database like \"REF\\|leginondata\\|AcquisitionImageData\\|image\".  \nsinedon.cfg contains just a simple module name rather than absolute name. Noderegistry in Leginon had relative modules, but that has been fixed.\n***\n**Related issues** [1](https:\/\/emg.nysbc.org\/issues?issue_id=42&set_filter=true&status_id=%2A) (0 open — [1 closed](https:\/\/emg.nysbc.org\/issues?issue_id=42&set_filter=true&status_id=c))\n\n- [History](https:\/\/emg.nysbc.org\/issues\/67?tab=history)\n- [Notes](https:\/\/emg.nysbc.org\/issues\/67?tab=notes)\n- [Property changes](https:\/\/emg.nysbc.org\/issues\/67?tab=properties)\n\nActions\n\n[Copy link](https:\/\/emg.nysbc.org\/issues\/67)\n\n[\\#1](https:\/\/emg.nysbc.org\/issues\/67#note-1)\n#### Updated by [Neil Voss](https:\/\/emg.nysbc.org\/users\/4) [over 15 years](https:\/\/emg.nysbc.org\/projects\/leginon\/activity?from=2010-05-27 \"05\/27\/2010 09:44 AM\") ago\n- **Target version** set to *Appion\/Leginon 2.1.0*\n\nActions\n\n[Copy link](https:\/\/emg.nysbc.org\/issues\/67)\n\n[\\#2](https:\/\/emg.nysbc.org\/issues\/67#note-2)\n#### Updated by Amber Herold [over 15 years](https:\/\/emg.nysbc.org\/projects\/leginon\/activity?from=2010-07-01 \"07\/01\/2010 10:14 AM\") ago\n- **Priority** changed from *Normal* to *Low*\n- **Target version** changed from *Appion\/Leginon 2.1.0* to *Appion\/Leginon Future Version*\n- **Affected Version** set to *Appion\/Leginon 2.0.1*\n\nActions\n\n[Copy link](https:\/\/emg.nysbc.org\/issues\/67)\n\n[\\#3](https:\/\/emg.nysbc.org\/issues\/67#note-3)\n#### Updated by [Jim Pulokas](https:\/\/emg.nysbc.org\/users\/3) [about 14 years](https:\/\/emg.nysbc.org\/projects\/leginon\/activity?from=2011-08-09 \"08\/09\/2011 04:27 PM\") ago\n- **Show in known bugs** set to *No*\n\nAnchi was getting an exception which I traced to the leginondata module getting imported twice. This can happen if you do\n```\nimport leginon\n```\nin one place and\n```\nimport leginon.leginondata\n```\nin another place. Python is pretty good about not importing the same module more than once *if* you import them as the same name. In this case, the module is imported as two different modules, therefore a test such as\n```\nleginon.leginondata.SessionData is leginondata.SessionData\n```\nreturn \"False\". The only solution is to be consistent about how a module is imported. As mentioned previously in this issue, the best practice is to use absolute imports. I solved Anchi's problem by converting all relative imports of the leginondata module to absolute imports. See r15944.\n\nActions\n\n[Copy link](https:\/\/emg.nysbc.org\/issues\/67)\n\n[\\#4](https:\/\/emg.nysbc.org\/issues\/67#note-4)\n#### Updated by [Anchi Cheng](https:\/\/emg.nysbc.org\/users\/9) [almost 12 years](https:\/\/emg.nysbc.org\/projects\/leginon\/activity?from=2013-10-21 \"10\/21\/2013 03:17 PM\") ago\n- **Assignee** changed from *Jim Pulokas* to *Anchi Cheng*\n\nActions\n\n[Copy link](https:\/\/emg.nysbc.org\/issues\/67)\n\nAlso available in: [Atom](https:\/\/emg.nysbc.org\/issues\/67.atom) [PDF](https:\/\/emg.nysbc.org\/issues\/67.pdf)\n\nPowered by [Redmine](https:\/\/www.redmine.org\/) © 2006-2025 Jean-Philippe Lang\n\nLoading...","attrs_readable_markdown":null,"meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

🚫

NOT INDEXABLE

✅

CRAWLED

6 months ago

🤖

ROBOTS SERVER UNREACHABLE

Failed to connect to robots server: Operation timed out after 2002 milliseconds with 0 bytes received

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	FAIL	`download_stamp > now() - 6 MONTH`	6 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://emg.nysbc.org/issues/67
Last Crawled	2025-10-16 00:41:27 (6 months ago)
First Indexed	2024-05-02 08:40:05 (1 year ago)
HTTP Status Code	200
Meta Title	Bug #67: Python imports, absolute vs. relative - Leginon - Electron Microscopy Group
Meta Description	Redmine
Meta Canonical	null
Boilerpipe Text	We should be using absolute imports instead of relative imports when possible. This means for instance: import leginon.leginondata rather than import leginondata There are a few reasons why explained in PEP 328 Python 3 will require either absolute imports or relative imports using a new syntax. This means that absolute imports will be the best way to have something that works now with Python 2 and in the future with Python 3. It is also a bad idea to customize PYTHONPATH or add path modifiers like Leginon.pth as a shortcut to not using absolute paths. In all cases, there is risk of conflicting module names between different packages. There are a few parts to resolving this in myami: - Change import statements throughout myami to absolute module names. I think it should be OK to leave alone relative imports if it is a module importing another module in the exact same directory, but this will require syntax revision for Python 3. Scripts in appion/bin should use "import appionlib.xxxxx" - Other references to module names in myami should start using absolute module names. For instance, the field names in the database like "REF\|leginondata\|AcquisitionImageData\|image". sinedon.cfg contains just a simple module name rather than absolute name. Noderegistry in Leginon had relative modules, but that has been fixed.
Markdown	### Project ### General ### Profile - [Sign in](https://emg.nysbc.org/login) - [Register](https://emg.nysbc.org/account/register) - [Home](https://emg.nysbc.org/) - [Projects](https://emg.nysbc.org/projects) - [Help](https://www.redmine.org/guide) Leginon [All Projects](https://emg.nysbc.org/projects?jump=issues) # Leginon - [Overview](https://emg.nysbc.org/projects/leginon) - [Activity](https://emg.nysbc.org/projects/leginon/activity) - [Roadmap](https://emg.nysbc.org/projects/leginon/roadmap) - [Issues](https://emg.nysbc.org/projects/leginon/issues) - [News](https://emg.nysbc.org/projects/leginon/news) - [Documents](https://emg.nysbc.org/projects/leginon/documents) - [Wiki](https://emg.nysbc.org/projects/leginon/wiki) - [Forums](https://emg.nysbc.org/projects/leginon/boards) - [Files](https://emg.nysbc.org/projects/leginon/files) ### Custom queries - [Appion/Leginon 3.0 bug fixes](https://emg.nysbc.org/projects/leginon/issues?query_id=30) - [Leginon 3.2 Bug Fixes](https://emg.nysbc.org/projects/leginon/issues?query_id=47) - [Leginon 3.3 Bug Fixes](https://emg.nysbc.org/projects/leginon/issues?query_id=53) - [Leginon 3.4 Bug Fixes](https://emg.nysbc.org/projects/leginon/issues?query_id=55) - [Leginon 3.4 Completed Features](https://emg.nysbc.org/projects/leginon/issues?query_id=57) - [Leginon 3.5 Bug Fixes](https://emg.nysbc.org/projects/leginon/issues?query_id=61) - [Leginon 3.5 Completed Features](https://emg.nysbc.org/projects/leginon/issues?query_id=60) - [Leginon 3.6 Bug Fixes](https://emg.nysbc.org/projects/leginon/issues?query_id=72) - [Leginon 3.6 Completed Features](https://emg.nysbc.org/projects/leginon/issues?query_id=71) - [Leginon Known Bugs](https://emg.nysbc.org/projects/leginon/issues?query_id=48) Actions [Copy link](https://emg.nysbc.org/issues/67) ## Bug \#67 open ### Python imports, absolute vs. relative Added by [Jim Pulokas](https://emg.nysbc.org/users/3) [over 15 years](https://emg.nysbc.org/projects/leginon/activity?from=2010-01-28 "01/28/2010 02:14 PM") ago. Updated [almost 12 years](https://emg.nysbc.org/projects/leginon/activity?from=2013-10-21 "10/21/2013 03:17 PM") ago. Status: Assigned Priority: Low Assignee: [Anchi Cheng](https://emg.nysbc.org/users/9) Category: \- Target version: [Appion - Appion/Leginon Future Version](https://emg.nysbc.org/versions/15) Start date: 01/28/2010 Due date: % Done: 30% Estimated time: Affected Version: Appion/Leginon 2.0.1 Show in known bugs: No Workaround: * Description We should be using absolute imports instead of relative imports when possible. This means for instance: ``` import leginon.leginondata ``` rather than ``` import leginondata ``` There are a few reasons why explained in [PEP 328](http://www.python.org/dev/peps/pep-0328/) Python 3 will require either absolute imports or relative imports using a new syntax. This means that absolute imports will be the best way to have something that works now with Python 2 and in the future with Python 3. It is also a bad idea to customize PYTHONPATH or add path modifiers like Leginon.pth as a shortcut to not using absolute paths. In all cases, there is risk of conflicting module names between different packages. There are a few parts to resolving this in myami: \- Change import statements throughout myami to absolute module names. I think it should be OK to leave alone relative imports if it is a module importing another module in the exact same directory, but this will require syntax revision for Python 3. Scripts in appion/bin should use "import appionlib.xxxxx" \- Other references to module names in myami should start using absolute module names. For instance, the field names in the database like "REF\\|leginondata\\|AcquisitionImageData\\|image". sinedon.cfg contains just a simple module name rather than absolute name. Noderegistry in Leginon had relative modules, but that has been fixed. * Related issues [1](https://emg.nysbc.org/issues?issue_id=42&set_filter=true&status_id=%2A) (0 open — [1 closed](https://emg.nysbc.org/issues?issue_id=42&set_filter=true&status_id=c)) - [History](https://emg.nysbc.org/issues/67?tab=history) - [Notes](https://emg.nysbc.org/issues/67?tab=notes) - [Property changes](https://emg.nysbc.org/issues/67?tab=properties) Actions [Copy link](https://emg.nysbc.org/issues/67) [\#1](https://emg.nysbc.org/issues/67#note-1) #### Updated by [Neil Voss](https://emg.nysbc.org/users/4) [over 15 years](https://emg.nysbc.org/projects/leginon/activity?from=2010-05-27 "05/27/2010 09:44 AM") ago - Target version set to Appion/Leginon 2.1.0 Actions [Copy link](https://emg.nysbc.org/issues/67) [\#2](https://emg.nysbc.org/issues/67#note-2) #### Updated by Amber Herold [over 15 years](https://emg.nysbc.org/projects/leginon/activity?from=2010-07-01 "07/01/2010 10:14 AM") ago - Priority changed from Normal to Low - Target version changed from Appion/Leginon 2.1.0 to Appion/Leginon Future Version - Affected Version set to Appion/Leginon 2.0.1 Actions [Copy link](https://emg.nysbc.org/issues/67) [\#3](https://emg.nysbc.org/issues/67#note-3) #### Updated by [Jim Pulokas](https://emg.nysbc.org/users/3) [about 14 years](https://emg.nysbc.org/projects/leginon/activity?from=2011-08-09 "08/09/2011 04:27 PM") ago - Show in known bugs set to No Anchi was getting an exception which I traced to the leginondata module getting imported twice. This can happen if you do ``` import leginon ``` in one place and ``` import leginon.leginondata ``` in another place. Python is pretty good about not importing the same module more than once if you import them as the same name. In this case, the module is imported as two different modules, therefore a test such as ``` leginon.leginondata.SessionData is leginondata.SessionData ``` return "False". The only solution is to be consistent about how a module is imported. As mentioned previously in this issue, the best practice is to use absolute imports. I solved Anchi's problem by converting all relative imports of the leginondata module to absolute imports. See r15944. Actions [Copy link](https://emg.nysbc.org/issues/67) [\#4](https://emg.nysbc.org/issues/67#note-4) #### Updated by [Anchi Cheng](https://emg.nysbc.org/users/9) [almost 12 years](https://emg.nysbc.org/projects/leginon/activity?from=2013-10-21 "10/21/2013 03:17 PM") ago - Assignee changed from Jim Pulokas to Anchi Cheng Actions [Copy link](https://emg.nysbc.org/issues/67) Also available in: [Atom](https://emg.nysbc.org/issues/67.atom) [PDF](https://emg.nysbc.org/issues/67.pdf) Powered by [Redmine](https://www.redmine.org/) © 2006-2025 Jean-Philippe Lang Loading...
Readable Markdown	null
Shard	104 (laksa)
Root Hash	6313026282925610904
Unparsed URL	org,nysbc!emg,/issues/67 s443