Skip to main content

File Service FAQ

This article answers common questions about files on the LifeOmic Platform. For more documentation, see Files and File Service.

How do I add files to the LifeOmic Platform?​

Access the file service and load files into the LifeOmic Platform through the following methods:

What is the best method for bulk file uploading?​

Easily upload a large number of files with the following methods:

What is the best method to transfer large individual files (larger than 500 GB)?​

For the majority of use-cases, the LifeOmic CLI transfers files successfully and provides an easy to use terminal experience.

But when you use the LifeOmic CLI to upload very large files, you can encounter issues stemming from a slow internet uplink speed, excessive time required for the upload, or an interrupted connection.

Configuring SFTP to a Project is recommended when transferring files of this size. SFTP can overcome some of the issues you might encounter using the CLI. SFTP can also resume a file transfer for a file that's been partially transferred.

Other good methods are the SDK for Python and Transfer Amazon S3 Files with the LifeOmic Platform Web App. But both of these methods require advanced knowledge and more extensive configuration than SFTP.

How are my files protected?​

LifeOmic File Services encrypts data in transit between the client and server. You do not need to encrypt your files before uploading them. Data is also encrypted at rest in storage. LifeOmic File Services use strong encryption and key management.

What is LifeOmic File Service?​

LifeOmic File Service is a managed service for storing and retrieving file data. For more information and a getting started guide, see File Service.

Once files are uploaded, they are available to the other LifeOmic Platform services for use and analysis.

  • Omics Explorer is an example where VCF files may be uploaded, and once indexed, will allow querying across genetic variants (SNV, CNV, Fusion) in real-time in the LifeOmic Platform Web App.

  • Task Service and Notebook Service are two examples where files may be brought in for analysis/compute with your own code.

Common file data examples uploaded are:

  • Genomic Variants, Gene expression, Proteomics, Pharmacogenetics - file formats such as VCF, BAM, CSV, TSV
  • Documents, Images, and Audio - file formats such as JPEG, PNG, DICOM, PDF, etc.

What parts of the LifeOmic Platform use the LifeOmic File Service?​

All parts of the LifeOmic Platform that store and retrieve file data rely on the LifeOmic File Service.

How are files organized? What is a project (aka data-set)?​

The LifeOmic Platform organizes data, including files, under user-defined projects (aka data-sets). Inside projects, files may be organized further into directory structures so files are not one large file list.

Does deleting a project (aka data-set) delete all associated files?​

Yes, deleting a project deletes the project and all associated data.

After you select Delete this Project, the project stays active for 14 days before the files are permanently deleted. You can cancel the deletion during this time period. You also have the option to delete the files immediately.

What access control is available?​

Projects allow for access control to be put in place by the organization.

Application level access-controls are enforced when viewing, downloading, and uploading files.

Example: A user can be configured to query and search genetic data for a subject, but be restricted in the ability to download the subject's file(s) on a per project basis.

Access control refers to the ability to control who can interact with a resource within the platform. The LifeOmic Platform uses Attribute Based Access Control (ABAC) to assign different attributes and dictate what information users have access to in cases requiring complex Access Control. For more information, see the Access Control Overview.

How durable is LifeOmic File Service?​

LifeOmic File Service is powered by AWS S3. For more information see AWS S3 FAQ.

Are my files backed up?​

Cross-region replication and backups are included as part of the File Service.

Can arbitrary files be stored without a subject/patient?​

The only required information to store a file is a project identifier.

Files are not required to be tied to a specific subject within a project.

If a link to a file not associated with a subject is desired, a FHIR DocumentReference can be used to provide this link using LifeOmic FHIR Service.

What limits are in place for LifeOmic File Service?​

File storage is unlimited. The maximum individual file size is 5 terabytes.

What file name restrictions are in place?​

File names must match, ^([a-zA-Z0-9!\\-+.*'()&$@=;+ ,?:_/]*)$, and be less than 970 characters in length.

What files can be viewed within the LifeOmic Platform Web App?​

You can view common file types, such as text, images, and markdown, within the LifeOmic Platform Web App.

Certain file types open into web-based viewers. Some examples are csv/tsv files, DICOM images, ipynb notebook files, and PDF files.

Files larger than 5 MB open in a new tab.

How can I document my files?​

Placing a README.md in a directory of files will render the markdown below the file listing as an inline description. For more information, see README Files.

How can I reference my file?​

A unique identifier is created for all files uploaded to the LifeOmic Platform. To make a permanent reference to a file, use a LifeOmic Resource Name (LRN). Future file renames will not break the LRN reference.

How can I share my project with collaborators?​

Users may create shareable links (URL) from the LifeOmic Platform Web App for those who have access to the project. For more information, see Share Link.

How can I trigger automation to run against the recent set of file transfers?​

The LifeOmic Platform allows you to configure an omics action that triggers an AWS Step Function upon file ingestion for the genomic processing. For more information, see Omics Actions and Ingestions.