Skip to contents

File formats

Source: vignettes/formats.qmd

1. Pairwise design format

This format is used by nexodiff::PairwiseDesign class.

The Pairwise design records all the samples with the experimental design and data file location. There are 2 formats possible: YML and CSV. It is highly recommended to use the YML format, as it supports more advanced features and provides a more structured way to define the experimental design.

In a normal samplesheet data file are expected to be kallisto output using h5 format.

Batch are defined with groups of sample and only one group per batch as controle such as wihtin a batch this pacxkage allow the generation of pairwise comparison for all groups vs the control group.

Alternatively this format can be used to define a meta-analysis by reusing precomputed

YML format

The samplesheet yml file is structured as follow:

# type: [string]
# required: F
# additional variable in the same order as given below per sample
variables:
  - 'donnorID'
  - 'tissueType'

# type: list<dict<string:(string|dict)>>
# required: T 
design:
  - # type: String
    # required: T
    # Absolute path where the file with count information are stored with the following format: {path}/{sample name}.h5
    # For now only h5 are supported
    path: /path/to/run1

    # type: 
    #   dict<
    #     batchIDs:dict<
    #       groupIDs:dict<   # if no additional vars, this can be a list of strings in place of a dict.
    #         sampleIDs:[strings]
    #       >
    #     >
    #   >
    #   with batchIDs, groupID, sampleIDs and varIDs beings strings matching the following regex pattern (python 3.7 module re) : [a-zA-Z]+[a-zA-Z0-9_.-]*
    # required: T 
    # Note: Only the dictionary dict<varIDs:string|int> might be empty.
    # Note: For now the purpose of variable is to perform an paired analysis
    # hierarchical listing of batch (batchIDs), then groups (groupIDs), samples (sampleIDs) and finally samples variables (varIDs) with variable values.
    import:
      batch1:
        group1:
          sample1: ['1','nasal']
          sample2: ['2','nasal']
          sample3: ['3','nasal']
        [...] # other groups...
      [...] # other batches...

  - path: /path/to/run2
    import:
       batch1:
        group1:
          sample4: ['a','alveol']
          sample5: ['b','alveol']
          sample6: ['c','alveol']
        [...] # other groups...
      [...] # other batches...

# type: dict<string:string>
# required: F
# Batch full string labels for sweet viz
b_labels:
  batch1: Name of batch 1
  [...]

# type: dict<string:string>
# required: F
# Group full string labels for sweet viz
g_labels: 
  group1: Name of group 1
  [...]

# type: dict<string:string>
# required: T
# ctrl group per batch. Only one that has to be listed in design
ctrlGroups:
  batch1: group1
  batch2: group8
  [...]

# type: dict<string:[strings]>
# required: F
# Identification of sample variable to use for paired analysis.
paired_by:
  batch1: ['tissueType']

# type: boolean
# required: F (default is FASLE)
# if False , h5 files are supposed to be found for each sample.
# if T , The differential expression results from another study is imported (rnadiff .csv outputs)
# if T , The Samples and their specific variables are ignored and might be empty
meta: F

The csv format

Example:

batch;group;sample;ctrl
b1;A;A1;F
b1;A;A2;F
b1;A;A3;F
b1;C;C1;T
b1;C;C2;T
b1;C;C3;T

mandatory fields are :

  • batch : code for batch
  • group : code for group
  • sample : sample name
  • ctrl : T if the group is supposed to be control / F else

optional fields are :

  • g_label : group label for figures
  • b_label : batch label for figures
  • run_id : run id if sample comes from different runs

In addition other variable can be added in the csv and will be interpreted as additional variables. However some variable names are forbidden :

  • sample_base
  • replicate

2. Kallisto output format

The nexodiff package can import expression data from Kallisto output files. The expected format is the HDF5 format (.h5), which is a binary format that contains the expression data for each sample.

When initializing an ExprData object with format = "kallisto", the package will look for a file named {sample_name}.h5 in the specified src_dir.

3. Transcriptome annotation format

The nexodiff package can parse GFF files directly to create an annotation object. When a GFF file is processed, it generates an internal annotation format that is stored in a directory for faster loading in subsequent runs. This internal format is a space-delimited file without headers with the following fields:

  • Transcript ID (tx_id)
    • Strings
  • Gene ID (gid)
    • Strings or Integers
  • type
    • Strings
  • Gene Symbol (symbol)
    • Strings
  • Taxonomic ID (taxid)
    • Integers
  • Species Names
    • Strings o Below an exemple of the first 4 lines:
NR_046018.2 100287102 misc_RNA DDX11L1 9606 human
NR_024540.1 653635 misc_RNA WASH7P 9606 human
NR_106918.1 102466751 precursor_RNA MIR6859-1 9606 human
MIR6859-1 102466751 ncRNA MIR6859-1 9606 human

4. id_mapping format

The id mapping format is generated from the web HTTP API of UNIPROT. Belo is an example of request wich is composed of 6 columns:

https://rest.uniprot.org/uniprotkb/stream?compressed=true&fields=accession%2Creviewed%2Cprotein_name%2Cgene_names%2Cxref_geneid%2Cannotation_score&format=tsv&query=(*)+AND+(model_organism%3A9606)

  • Entry Uniprot ID
  • Status Status reviewed or unreviewed
  • Protein names Long string for protein name
  • Gene names space separated gene reported gene names
  • GeneID cross-references separated and terminated by ; with the gene ID (Entrez NCBI db)
  • Annotation Annotation score (float) 5.0 is the best score