-
Notifications
You must be signed in to change notification settings - Fork 0
TJ's annotated DRAFT PR not meant to be merged. #61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: data_access
Are you sure you want to change the base?
Conversation
| The best way to grab S3 cloud URI data from MAST is using <a href="https://astroquery.readthedocs.io/en/latest/mast/mast_obsquery.html#downloading-data-products">astroquery</a>. | ||
| Let's grab S3 cloud URI data from MAST using <a href="https://astroquery.readthedocs.io/en/latest/mast/mast_obsquery.html#downloading-data-products">astroquery</a>. This module has the *optional* ability to return the location of the data from AWS. | ||
|
|
||
| *Look up which archives' astroquery modules do have this.* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Er, apparently none except us? And all three of ours look different.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that is correct and unfortunate. IRSA has a simple PyVO column 'cloud_access'. I like this one the best. It is direct and easy. MAST uses a astroquery function get_cloud_uris which is fine. IMO HEASARC has the most difficult approach - seemingly hiding it in a datalink table you must read and then find the appropriate line for any URL you may want.
| ```python | ||
| url = filtered_table[8]['access_url'] | ||
| %skip | ||
| # WHY DOESNT THIS WORK? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Xamin issue I've just raised to higher priority.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It works for me? On my local that is, which does not have the same environment set up as Fornax (e.g., I run in Python 3.9.16).
| s3_file = "s3://nasa-irsa-wise/wise/allwise/images/p3am_cdd/08/0830/0830p227_ac51/0830p227_ac51-w2-int-3.fits" | ||
| #s3_uri follows s3://<bucket-name>/<key> structure where the key is often | ||
| # but not always <path>/<filename> | ||
| bucket = json.loads(allwise_table['cloud_access'][0])['aws']['bucket_name'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This bit is the one I really think we ought to sue.
HEASARC demonstration and tutorial notebook review template
Requesting a review
Please request a review (through the GitHub pull request interface) from one of the HEASARC team members associated with the HEASARC-tutorials repository, being mindful of observer's workloads.
Available for all reviews:
Observers/available for some reviews:
Reviewers should attempt to provide initial comments within 1-2 days.
Please feel free to tag any user who you feel would like to discuss the notebook under review.
Critical review criteria
The author of the pull request should make an effort to go through these check points and ensure that their submission satisfies each point - reviewers will also compare to these checklists.
Science review checklist
Tech review checklist
requirements_<notebook_filename>.txtfile listing all its direct dependencies?# !pip install -r <filename>; and has the notebook no other installation related cells?try/exceptstatements that are narrow in scope?importstatements?