pipelines
JSON array
Pipelines are the methods used to analyze data after it has been collected. In other words, the experiment provides the methods to collect the data and the pipelines provide the methods to analyze the data once it has been collected.
Basic pipeline information is stored in the main squirrel.json
file, and complete pipeline information is stored in the pipeline subdirectory in the pipeline.json
file.
JSON Variables
*required
Variable | Type | Description |
| string | Compute cluster engine (sge or slurm). |
| string | Submit username. |
| string | Queue to submit jobs. |
| string | Hostname to submit jobs. |
| JSON array | JSON array of complete files, with relative paths to |
| datetime | Date the pipeline was created. |
| string | How the data is copied to the analysis directory: |
| string |
|
| string |
|
| string |
|
| string | Longer pipeline description. |
| string |
|
| string | Directory where the analyses for this pipeline will be stored. Leave blank to use the default location. |
| string | ID or name of a group on which this pipeline will run |
| string | Either subject or study |
| number | subject-level analysis (1) or group-level analysis (2). REQUIRED |
| number | Maximum allowed clock (wall) time in minutes for the analysis to run. |
| number | Amount of memory in GB requested for a running job. |
| string | Pipeline name. REQUIRED |
| string | Extended notes about the pipeline |
| number | Number of analyses allowed to run at the same time. This number if managed by NiDB and is different than grid engine queue size. |
| number | Number of CPU cores requested for a running job. |
| string | Comma separated list of parent pipelines. |
| string | Executable script to be run at completion of the analysis to find and insert results back into NiDB. |
| number | Delay in hours, after the study datetime, to submit to the cluster. Allows time to upload behavioral data. |
| string | The path to a temporary directory if it is used, on a compute node. |
| bool | true if using the profile option, false otherwise. |
| bool | true if using a temporary directory, false otherwise. |
| number | Version of the pipeline. |
| string | See details of pipeline scripts |
| string | See details of pipeline scripts. |
| string | Path of this pipeline within the squirrel package. |
| number | Number of data steps. |
JSON array |
Directory structure
Files associated with this section are stored in the following directory. PipelineName
is the unique name of the pipeline.
/pipelines/<PipelineName>
Last updated