The Command Line Uploader is a general method used to transfer local files into the Enable Cloud. The main use cases for this uploader includes raw data transfer or miscellaneous metadata uploads. For standard transfers of stitched TIF images for data processing and analysis, please use our Experiment Image Data Uploader. If you have further questions, please contact [email protected].

<aside> 💡 NOTE: Generated Command Line Uploader credentials are rotated every 6 months. If you are no longer able to upload with the Command Line Uploader, try to regenerate your credentials by following the instructions below!

</aside>

Part 1: Setup

1. If you are a group admin for your study, you should have received an email from [email protected] containing your AWS IAM User credentials. Navigate to the email associated with your Enable Medicine account. The credentials emailed to you include:
   1. AWS_ACCESS_KEY
   2. AWS_SECRET_ACCESS_KEY
2. On the Command Line File Uploader page in the Enable Portal, you should also see the AWS Bucket Path displayed. You will need this path later on in these instructions. Substitute this bucket path wherever you see [BUCKET_PATH] in the following steps.
3. Install the AWS CLI
   1. For Mac, follow the AWS documentation here.
   2. For Windows, follow the AWS documentation here.
4. Launch your computer's command line.
   1. Mac
      1. To launch the command line on Mac, type "Terminal" into the Spotlight search bar by clicking the magnifying glass icon in the top right of the screen.
      2. Additional documentation about the terminal can be found here.
   2. Windows
      1. To launch the command line on Windows type "Command Prompt" in the Windows search box.
      2. Additional documentation about the command prompt can be found here.
5. Configure the AWS CLI by running aws configure in the command line.
   1. Enter your AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY, region (us-west-2), and output format (json). The access key and secret access key are included in the email mentioned in step 1.
   2. Additional instructions can be found here.

Part 2: Transfer

1. Determine the file path where your files are located.

   1. Mac:

      1. Open a Finder window and navigate to the directory where the files are located.
      2. At the top of the screen, in the menu bar, choose "View > Show Path Bar".
      3. Right click the Path Bar folder name and select Copy "folder" as Pathname
      4. The pathname should end with the folder's name. For example, if the folder to sync was called experiment1 , the path could look like /Users/machina/Workspace/experiment1.

      

      An example showing how to find the pathname of a specific file on Mac.

   2. Windows:

      1. Navigate to the parent of the directory where the files are located.
      2. Hold down Shift and right click on the folder.
      3. Select Copy as Path.
      4. The pathname should end with the folder's name. For example if the folder to sync was called experiment1 , the path could look like C:\\Users\\path\\to\\images\\experiment1 .

      

      An example showing how to find the pathname of a specific file on Windows.

2. Now that we have our command line ready, we can and run the following command to begin uploading files to AWS.

   1. aws s3 sync "[YOUR_PATHNAME]" "s3://[BUCKET_PATH]/[FOLDER_NAME]"
      1. Make sure to include your pathname before the S3 bucket name.
      2. Make sure both arguments are surrounded by quotes.
      3. For example if [YOUR_PATHNAME] was /Users/path/to/experiment1, the final command will look something like:
         1. aws s3 sync "/Users/path/to/experiment1" "s3://[BUCKET_PATH]/experiment1"
            1. This S3 folder is private to your study, so use whatever folder name is appropriate for its contents
      4. If you are syncing multiple different datasets, you should perform a separate sync command for each dataset.
         1. For example, if you have dataset1and dataset2 you would run two commands:
            1. aws s3 sync "/Users/path/to/dataset1" "s3://[BUCKET_PATH]/dataset1"
            2. Andaws s3 sync "/Users/path/to/dataset2" "s3://[BUCKET_PATH]/dataset2"

3. While your folder is syncing, please ensure your computer does not fall asleep or disconnect from the internet. This should not be a problem as long as you are still using your computer; however, if you are planning on transferring large datasets or stepping away from your computer please see the following links:

   1. Mac
   2. Windows

4. While syncing, a list of files will be displayed as they upload. Please wait for your command prompt to return when the upload is completed.

   

   Upload in progress

   

   Completed upload - command prompt has returned

5. You can cancel an upload at any time by pressing Control + C

6. If an upload is aborted midway, rerun the same command. Only files that have not already been transferred will be uploaded.

Return to Home Directory