Access Laboratory of Systems Pharmacology Datasets on AWS

Access Laboratory of Systems Pharmacology Datasets on AWS 

This page provides instructions to access image data from the Laboratory of Systems Pharmacology, Harvard Medical School. All 2- and 3-dimensional images at full resolution, derived image data (e.g., segmentation masks), and single-cell tables are stored and can be accessed through Amazon Web Services (AWS) S3.

Getting Started 

Locate Bucket Name

• Locate the publication-specific data index. This should be linked in the data availability statement within the publication or landing page.
• Within the publication's data index, locate the AWS bucket name where the associated data is stored:
  • For publications published before 2024: The AWS bucket name will likely structured as "author-year-topic", which can likely be accessed at the server address [bucketname].s3.amazonaws.com. You will use this bucket name in the following instructions to access data on AWS.
  • For publications published after 2024: The AWS bucket name will likely be "lsp-public-data", which can be accessed at the server address lsp-public-data.s3.amazonaws.com.
    • Additionally, locate the folder name within the publication's data index; this is the folder within the lsp-public-data bucket containing the data for the associated publication. The folder will likely be structured as follows "author-year-topic". 

File List and Organization

A detailed description of the files and file organization can also be found on the publication-specific data index, GitHub repository README, or publication. 

AWS Data Access

To browse and download the data either use a graphical file transfer application that supports S3 such as CyberDuck, or the AWS CLI tools. A graphical tool may be more convenient but the CLI tools will likely offer higher download speeds. For users who wish to perform processing within AWS, note that the bucket is located in the us-east-1 region so any other resources must be instantiated in this same region.

CyberDuck or Graphical File Transfer Instructions

Download CyberDuck from https://cyberduck.io/ . We recommend using an adblocker when accessing this website. 

Once installed: 

1. Open CyberDuck.

2. Click "Open Connection". This will open a dialogue box. 

3. From the dropdown menu at the top of the window, select "Amazon S3"

3. Change the Server entry to the bucket name you retreived from the paper index, either lsp-public-data.s3.amazonaws.com or [bucketname].s3.amazonaws.com. Remember to change bucketname to the provided bucket name. 

4. Set the Access Key ID to anonymous. This will then gray out and lock the Access Key ID and Secret Access Key fields. You may need to uncheck "Save Password" to successfully connect.

5. Click "Connect" 

6. For data within the lsp-public-data bucket: From the folder list, navigate to the folder associated with the publication. 

AWS CLI Tip

If you don't have your own AWS account, be sure to pass the --no-sign-request option to all aws commands to connect anonymously. for example: aws s3 ls --no-sign-request s3://bucketname. If you continue to experience issues using the CLI, please follow the instructions for using CyberDuck above or refer to AWS existing documentation.

Contact

Email tissue-atlas(at)hms.harvard.edu with the subject line "bucketname: Data Access" if you experience issues accessing the above S3 buckets. Please include the steps you have already tried to help us troubleshoot.