Open Data FAQ
Frequently asked questions for the mammography dataset EMBED Open Data.
The Data Use Agreement includes (but is not limited to) these important details you are agreeing to:
- The data must be used directly within an academic or other non-profit research organization under the direction of a lead principal investigator.
- You agree to only use the data for non-commercial, research purposes.
- You agree not to share, distribute, publish, or reproduce any portion of the data. If multiple people in your group would like access, please submit individual requests.
- You agree not to release/commercialize models (including foundation models), model-weights, or embedding spaces produced (in any way) using EMBED.
The full Data Use Agreement is available on our GitHub. Please familiarize yourself with it before using EMBED Open Data.
EMBED Open Data is available for non-commercial research use for members of academic and other non-profit research institutes. To request access:
- Go to the HITI Lab Data Portal and create an account using your email address.
- Sign in and submit an application for EMBED Open Data.
- Wait for our team to review your application. We may approve it, ask you to amend it with clarifications, or deny it.
- If your application is approved, you will receive access for 90 days. We will create AWS credentials for you and provide instructions for configuring the AWS CLI.
Please ensure your proposed project meets the terms of the Data Use Agreement before submitting your application.
No. Create an account on the HITI Lab Data Portal using your email address and submit your application there. You do not need to create a personal AWS account or provide an AWS account ID.
Approved access lasts for 90 days. The credentials provided by HITI Lab are limited to this access period.
Our review team may request clarifications before making a decision. Sign in to the HITI Lab Data Portal, update your application with the requested information, and resubmit it for review.
No, as part of the AWS Open Data Program, egress costs have been waived for EMBED Open Data. Please note you will still incur ingress costs if you download the dataset into an AWS resource that you own (such as a personal/lab S3 bucket) instead of a local directory.
EMBED Open Data contains 480,323 images (76% 2D and 24% synthetic-2D) collected during 76,373 breast imaging exams from 22,382 patients at Emory healthcare centers between 2013 and 2020. The total dataset is approximately 2.5TB and consists of a clinical data table, an image metadata table, and the images.
If space is a concern, we recommend downloading just the files in s3://embed-dataset-open/tables/ first (the tables are <1GB), then using them to identify the subset of images in your cohort. Individual images can then be downloaded with their file paths using a bash script or a python script with the boto3 package.
At the moment, the Open Data version of EMBED contains the 2D and synthetic-2D ('c-view') images for the included patients. We're planning to add the DBT, US, and MRI images for these patients in the future when version 2 of EMBED is completed.
The best resource for meanings/values of clinical variables in EMBED Open Data is the data dictionary in the GitHub repository. The documentation on this website also describes some of the most commonly-used features on the Overview and Label Assignment pages.
You do not need to generate credentials in a personal AWS account. After your application is approved, HITI Lab will create AWS credentials for you and provide instructions for configuring the AWS CLI. Keep these credentials secure and do not share them with anyone.