Default Usernames and Passwords

How to change Linux password

As the root user, run

passwd nidb

-or- as the nidb user, run

passwd

How to change MariaDB passwords

Login to http://localhost/phpMyAdmin using the root MySQL account and password. Go to the User Accounts menu option. Then click Edit privileges for the root (or nidb) account that has a ‘%’ as the hostname. Then click Change password button at the top of the page. Enter a new password and click Go

How to change NiDB admin password

When logged in to NiDB as admin, go to My Account. Enter a new password in the password field(s). Click Save to change the password.

Prerequisites

1. Hardware - There are no minimum specifications. If the hardware can run RHEL 8, then it can run NiDB.

2. RHEL 8 compatible - NiDB runs only on RHEL8 compatible (CentOS 8, Rocky Linux 8, AlmaLinux 8).

3. FSL - Install FSL from https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FslInstallation After installation, note the location of FSL, usually /usr/local/fsl/bin. Or try these commands to install FSL.

   1. wget https://fsl.fmrib.ox.ac.uk/fsldownloads/fslinstaller.py # this may work

   2. yum install python2

   3. python2 fslinstaller.py

4. firejail - firejail is used to run user-defined scripts in a sandboxed environment. Install firejail from https://firejail.wordpress.com/

   • rpm -i firejail-x.y.z.rpm

5. OS packages - yum install epel-release for repo for ImageMagick

Install NiDB

1. Download the latest .rpm package

2. yum --nogpgcheck localinstall nidb-xxxx.xx.xx-1.el8.x86_64.rpm

3. Secure the MariaDB installation by running sudo mysql_secure_installation and using the following responses

    Enter current password for root (enter for none):
    Change the root password? [Y/n] n
    Remove anonymous users? [Y/n] Y
    Disallow root login remotely? [Y/n] Y
    Remove test database and access to it? [Y/n] Y
    Reload privilege tables now? [Y/n] Y

1. Finish Setup - Use Firefox to view http://localhost/setup.php . Follow instructions on the page to configure the server

   • The setup page must be accessed from localhost -or- the config file must be manually edited to include the IP address of the computer you are using the access setup.php.

   • Edit /nidb/nidb.cfg and add your IP address (comma separated list) to the [setupips] config variable. It should look something like [setupips] 127.0.0.1, 192.168.0.1 depending on the IP(s)

See detailed upgrade instructions for a more in-depth explanation of the upgrade.

1. Download latest NiDB release.

2. yum --nogpgcheck localinstall nidb-xxxx.xx.xx-1.el8.x86_64.rpm

3. Make sure your IP address is set in the [setupips] variable in the config file. This can be done manually by editing /nidb/nidb.cfg or by going to Admin → Settings

4. Go to http://localhost/setup.php (Or within NiDB, go to Admin → Setup/upgrade)

5. Follow the instructions on the webpages to complete the upgrade

Sometimes you need to move your installation to a new server. Maybe you were testing in a virtual machine and want to move to a full server, or vice-versa. Maybe your server needs to be upgraded. Follow these steps to migrate an installation from one server to another.

Migration steps

1. On the old server, export the SQL database

   • mysqldump -uroot -ppassword nidb > nidb-backup.sql

2. Copy the exported .sql file to the new server.

3. On the new server, install NiDB as a new installation

4. On the new server, import the new database

   • mysql -uroot -ppassword nidb < nidb-backup.sql

5. Finish upgrade, by going to http://localhost/setup.php . Follow the instructions to continue the upgrade.

NiDB attempts to ensure HIPAA compliance, but is not completely compliant with all aspects of data privacy.

HIPAA Identifiers

There are 18 types of personally identifiable information (from Health and Human Services website). Data that can be stored in NiDB is highlighted.

• Names

• All geographic subdivisions smaller than a state, including street address, city, county, precinct, ZIP code, and their equivalent geocodes, except for the initial three digits of the ZIP code if, according to the current publicly available data from the Bureau of the Census:

  • The geographic unit formed by combining all ZIP codes with the same three initial digits contains more than 20,000 people; and

  • The initial three digits of a ZIP code for all such geographic units containing 20,000 or fewer people is changed to 000

• All elements of dates (except year) for dates that are directly related to an individual, including birth date, admission date, discharge date, death date, and all ages over 89 and all elements of dates (including year) indicative of such age, except that such ages and elements may be aggregated into a single category of age 90 or older

• Telephone numbers

• Vehicle identifiers and serial numbers, including license plate numbers

• Fax numbers

• Device identifiers and serial numbers

• Email addresses

• Web Universal Resource Locators (URLs)

• Social security numbers

• Internet Protocol (IP) addresses

• Medical record numbers

• Biometric identifiers, including finger and voice prints

• Health plan beneficiary numbers

• Full-face photographs and any comparable images

• Account numbers

• Any other unique identifying number, characteristic, or code, except as permitted by paragraph (c) of this section [Paragraph (c) is presented below in the section “Re-identification”]; and

• Certificate/license numbers

PHI on NiDB

The following pieces of information are stored on NiDB. Not all are required.

Ways to reduce PHI exposure

phpMyAdmin

phpMyAdmin is not required to use NiDB, but is extremely useful to maintain the SQL database that NiDB uses.

1. Download latest version of phpMyAdmin from http://phpmyadmin.net/

2. Unzip the contents of .zip file into /var/www/html/phpMyAdmin

3. Visit http://localhost/phpMyAdmin

These changes to MySQL/MariaDB can be helpful to improve performance and capability of NiDB

Variables

max_allowed_packet - Change value to 1​073​741​824

slave_max_allowed_packet - Change to 1​073741​824

DICOM files store lots of protected health information (PHI) and personally identifiable information (PII) by default. This is great for radiologists, but bad for researchers. Any PHI/PII left in your DICOM files when sharing them with collaborators could be a big issue for you. Your IRB might shut down your project, shoot you into space, who knows. Make sure your data is anonymized, and anonymized in the way that your IRB wants.

NiDB offers 3 ways to export, and otherwise handle, DICOM data which are described below

Original - This means there is no anonymization at all. All DICOM tags in the original file will be retained. No tags are added, removed, or changed.

Anonymize - This is the default anonymization method, where most obvious PHI/PII is removed, such as name, DOB, etc. However, dates and locations are retained. The following tags are anonymized

• 0008,0090 ReferringPhysiciansName

• 0008,1050 PerformingPhysiciansName

• 0008,1070 OperatorsName

• 0010,0010 PatientName

• 0010,0030 PatientBirthDate

Anonymize Full - This method removes all PHI/PII, but also removes identifiers that are used by NiDB to accurately archive data by subject/study/series. If most of the tags used to uniquely identify data are removed... it's hard to group the DICOM files into series. So be aware that full anonymization might make it hard to archive the data later on.

• 0008,0090 ReferringPhysiciansName

• 0008,1050 PerformingPhysiciansName

• 0008,1070 OperatorsName

• 0010,0010 PatientName

• 0010,0030 PatientBirthDate

• 0008,0080 InstitutionName

• 0008,0081 InstitutionAddress

• 0008,1010 StationName

• 0008,1030 StudyDescription

• 0008,0020 StudyDate

• 0008,0021 SeriesDate

• 0008,0022 AcquisitionDate

• 0008,0023 ContentDate

• 0008,0030 StudyTime

• 0008,0031 SeriesTime

• 0008,0032 AcquisitionTime

• 0008,0033 ContentTime

• 0010,0020 PatientID

• 0010,1030 PatientWeight

Upgrade NiDB steps

Install NiDB .rpm

Get the most recent .rpm from github. The latest version may be different than the example below. You can also download the latest release .rpm from https://github.com/gbook/nidb/releases/latest

wget https://github.com/gbook/nidb/releases/download/v2021.10.699/nidb-2021.10.699-1.el8.x86_64.rpm
sudo yum localinstall --nogpgcheck nidb-2021.10.699-1.el8.x86_64.rpm

Last metadata expiration check: 0:28:21 ago on Thu 14 Oct 2021 10:01:28 AM EDT.
Dependencies resolved.
============================================================================================================================================
 Package                    Architecture                 Version                                   Repository                          Size
============================================================================================================================================
Upgrading:
 nidb                       x86_64                       2021.10.699-1.el8                         @commandline                        56 M

Transaction Summary
============================================================================================================================================
Upgrade  1 Package

Total size: 56 M
Is this ok [y/N]:

Complete setup on Website

Visit http://localhost/setup.php and follow the pages.

Entry page - Turning off access to the website and disabling all modules can help prevent errors during the upgrade. Always remember to backup the database! Click Next to continue.

Pre-requisites - This page will check for CentOS packages and display an error if a package is missing or the wrong version. If missing any packages, check the output from the NiDB rpm installation or manually install the missing packages. After packages are installed, then refresh this page. Once all pre-requisities are met, click Next to continue.

SQL database connection Enter the root SQL password in this screen. If you want to check what tables will be updated, without updating them, select the Debug checkbox. If you encounter issues upgrading large tables, you can choose to limit the size of the tables that are upgraded and you can then update those manually. This is not recommended however. Click Configure Database to continue.

Schema upgrade The details of the schema upgrade will be displayed. Any errors will be indicated. Click Next to continue.

Configuration Any changes (paths, settings, options, etc) can be changed here. Click Write Config to continue.

All finished! Click Done to complete the upgrade.

Upgrade issues

Schema upgrade errors

The database schema upgrade may not display a SQL error if it occurs, but the errors are stored in the error_log table in the database. Find the latest error(s) and the description will help diagnose the issue.

Row length errors

In phpMyAdmin, select the table in question. Under the operations tab, check the table type. If it is Innodb, change this to Aria, and click Go.

Primary key errors

If the primary key error references a TEXT column, in other words if a TEXT column has a primary key, then delete the key for that column. This can be done in phpMyAdmin.

Overview

A project will often need imaging data of different modalities uploaded to an instance of NiDB. All of the data must be associated with the correct subject, and each modality must have it's own study.

Follow this order of operations when uploading data

1. Create the subject(s) - Subjects must exist in NiDB and be enrolled in a project before uploading any imaging data

2. Upload EEG and ET data before MRI data - MRI data is automatically sorted into subject/session during import which is different than how EEG and ET are imported. Uploading the EEG and ET first makes sure that all of the subjects and associated IDs exist before attempting to upload the MRI data

3. Upload small MRI imaging sessions (less than 1GB in size) using the NiDB website - This is useful to upload data for a single subject.

4. Upload large MRI imaging sessions (greater than 1GB in size, or dozens of subjects), or data that must be anonymized, using the NiDBUploader - This is useful if you need to upload thousands of MRI files. Sometimes a single session might generate 10,000 files, and maybe you have 20 subjects. Might be easier to use the NiDBUploader.

Create a Subject

1. Select the correct instance.

2. On the top menu click Subjects --> Create Subject

3. Fill out the information. First name, Last name, Sex, Date of Birth are required

4. Click Add and confirm on the next screen. The subject is now created

5. On the subject's page, select a project from the Enroll in Project dropdown (you might need to scroll down in the dropdown), and click Enroll.

Updating IDs

1. On the subject's page, click the Edit Subject button on the left.

2. In the IDs section, enter the extra ID(s) in the specific project line. Separate more than one ID with commas, and put a * next to the primary ID. Such as *423, P523, 3543

3. Click Update and confirm at the next screen.

4. If demographic data are stored in a redcap (or other) database and NiDB is storing imaging data, make sure to put each ID in each database. In other words, put the redcap ID into the the NiDB ID field and store the S1234ABC ID in redcap.

Upload EEG and ET data

1. On the subject's page, find the Create New Imaging Studies dropdown. Expand that and find the New empty study dropdown. Select the ET or EEG modality, and click Create. Your study will be created.

2. On the subject's page, click the Study number that was just created, and it will show the study.

3. Fill out the protocol name, the date/time of the series, and any notes, then click Create series.

4. Drag and drop your file(s) onto the Upload button. The status of the upload will be shown below the button.

5. Don't click Refresh or press ctrl+R to reload the page. Instead, click the Study n link at the top of the page.

6. If you need to rename or delete files, click the Manage N file(s) button on the study page.

Upload MRI data through the website (small datasets)

Upload the data

1. On the top menu, click Data. Then click the Import imaging button.

2. Click the New Import button.

3. Choose the files you want to upload. These can indivudual files, or zip files containing the DICOM or par/rec files.

4. Data modality should be Automatically detect.

5. Select the destination project

6. Leave the other matching criteria as the defaults

7. Click Upload.

A new row will be created with your upload. MRI data can contain just about anything, so NiDB needs to read through all the files and see what's there.

Once NiDB has parsed the data you uploaded, you'll need to decide which data to actually import.

1. Click the yellow Choose Data to Import button

2. Details about the import will be displayed. On the bottom will be a list of subjects, studies, and series. You can deselect certain series if you don't want to import them, but likely you'll just want to import all of the series, so click the Archive button.

3. Click on the Back button on the page to go back to the import list.

4. Refresh this page and eventually your import should change to a status of Archived.

Upload MRI data through NiDBUploader (large datasets)

Install it and run the program.

Create a connection

1. Fill in the server: https://yourserver.com, and enter your username/password. Click Add Connection.

2. Click on the connection and click Test Connection. It should say Welcome to NiDB after a little while.

Select the data

1. Select a Data Directory at the top of the program. This should be the parent directory of your data.

2. Change the Modality to MR. Uncheck the Calculate MD5 hash...

3. Click Search. This will slowly populate that list with DICOM/ParRec files that it finds.

4. Once it is done loading files, you can select multiple files and click Remove Selected if you need to.

Set anonymization options

1. Make sure Replace PatientName is checked.

Set the destination

1. Click the ... button for the Instance, which will populate the list of instances. Select your instance. Then select the Project.

2. Click the ... for the Site and Equipment to load the lists. Select the Site and Equipment.

Upload the data

1. Click Upload.

2. It will take a while. Like a long time. Be prepared for that. Depending on the number of files, it could take hours to upload.

3. If any files fail, it will be displayed along with a reason. If you fix the errors, then you can click Resend Failed Objects.

Overview

The Neuroinformatics Database (NiDB) is designed to store, retrieve, analyze, and share neuroimaging data. Modalities include MR, EEG, ET, video, genetics, assessment data, and any binary data. Subject demographics, family relationships, and data imported from RedCap can be stored and queried in the database.

Features

• .rpm based installation for CentOS 8, RHEL 8, Rocky Linux 8 (not for CentOS Stream)

• Store any neuroimaging data, including MR, CT, EEG, ET, Video, Task, GSR, Consent, MEG, TMS, and more

• Store any assessment data (paper-based tasks)

• Store clinical trial information (manage data across multiple days & dose times, etc)

• Built-in DICOM receiver. Send DICOM data from PACS or MRI directly to NiDB

• Bulk import of imaging data

• User and project based permissions, with project admin roles

• Search and manipulate data from subjects across projects

• Automated imaging analysis pipeline system

• "Mini-pipeline" module to process behavioral data files (extract timings)

• All stored data is searchable. Combine results from pipelines, QC output, behavioral data, and more in one searchable

• Export data to NFS, FTP, Web download, NDA (NIMH Data Archive format), or export to a remote NiDB server

• Export to squirrel format

• Project level checklists for imaging data

• Automated motion correction and other QC for MRI data

• Calendar for scheduling equipment and rooms

• Usage reports, audits, tape backup module

• Intuitive, modern UI. Easy to use

Features

.rpm based installation & upgrade

Install or upgrade NiDB in minutes on RHEL compatible Linux OS.

Automated import of DICOM data

DICOM data can be automatically imported using the included dcmrcv DICOM receiver. Setup your MRI or other DICOM compatible device to send images to NiDB, and NiDB will automatically archive them. Image series can arrive on NiDB in any order: partial series, or full series to overlap incomplete series.

Store any type of data

Literally any type of imaging data: binary; assessment; paper based; genetics. See full list of supported modalities. All data is stored in a hierarchy: Subject --> Study --> Series. Data is searchable across project and across subject.

Store clinical trial data

NiDB stores multiple time-points with identifiers for clinical trials; exact day numbers (days 1, 15, 30 ...) or ordinal timepoints (timepoint 1, 2, 3 ...) or both (day1-time1, day1-time2, day2-time1, ... )

Bulk import of imaging data

Got a batch of DICOMs from a collaborator, or from an old DVD? Import them easily

Search and export imaging data

Find imaging data from any project (that you have permissions to...) and export data. Search by dozens of criteria.

Export to multiple formats

Image formats

• Original raw data - DICOM, Par/Rec, Nifti

  • Anonymized DICOM data: partial and full anonymization

• Nifti3d

• Nifti3dgz

• Nifti4d

• Nifti4dgz

• squirrel

Package formats

• squirrel

• BIDS

• NDA/NDAR

Destinations

• NFS share

• Web

• Public download/dataset

• Local FTP

• Remote NiDB instance

Search and export non-imaging data

Data obtained from pipeline analysis, imported and locally generated measures, drugs, vitals, measures, are all searchable.

Full analysis pipeline system

From raw data to analyzed, and storing result values/images. Utilize a compute cluster to process jobs in parallel. Example below, 200,000 hrs of compute time completed in a few weeks. Hundreds of thousands of result values automatically stored in NiDB and are searchable.

Automated MR quality control

Large number of automatically generated metrics. Metrics are exportable as .csv and tables.

Calendar

Fully featured calendar, running securely on your internal network. Repeating appts, blocking appts, and time requests.

Publications

• Book GA, Anderson BM, Stevens MC, Glahn DC, Assaf M, Pearlson GD. Neuroinformatics Database (NiDB)--a modular, portable database for the storage, analysis, and sharing of neuroimaging data. Neuroinformatics. 2013 Oct;11(4):495-505. doi: 10.1007/s12021-013-9194-1. PMID: 23912507; PMCID: PMC3864015. https://pubmed.ncbi.nlm.nih.gov/23912507/

• Book GA, Stevens MC, Assaf M, Glahn DC, Pearlson GD. Neuroimaging data sharing on the neuroinformatics database platform. Neuroimage. 2016 Jan 1;124(Pt B):1089-1092. doi: 10.1016/j.neuroimage.2015.04.022. Epub 2015 Apr 16. PMID: 25888923; PMCID: PMC4608854. https://pubmed.ncbi.nlm.nih.gov/25888923/

Outdated information Watch an overview of the main features of NiDB (recorded 2015, so it's a little outdated): Part 1 | Part 2 | Part 3

Documentation

Getting Started

Using NiDB

Advanced

Prerequisites

Hardware - There are no minimum specifications. If the hardware can run Linux, then it should be able to run NiDB.

RHEL8.5/Rocky8.5/CentOS 8 - NiDB runs only on RHEL 8 compatible OSes. NiDB does not run on Fedora or CentOS Stream.

FSL

Download FSL from and follow the installation instructions. After installation, note the location of FSL, usually /usr/local/fsl.

Alternatively, try these commands to install FSL

firejail

firejail is used to run user-defined scripts in a sandboxed environment. This may be deprecated in future releases of NiDB. Install firejail from https://firejail.wordpress.com/

Install NiDB rpm

Download the latest .rpm package from http://github.com/gbook/nidb

Secure the MariaDB installation by running mysql_secure_installation as root and using the following responses. The MariaDB root password is already set to password.

Finish Setup

1 - Backup SQL database

Copy the mysqldump command and run that on the command line. It should create a .sql file that contains a backup of the database. This is required even for new installations because you should become familiar with, and get int the habit of, backing up the SQL database. After you've backed up the database using mysqldump, refresh the setup page and it should allow you to continue with the setup.

Click Next to continue, and the following page will show the status of Linux packages required by NiDB.

2 - Linux Prerequisites

If there are any missing packages or if a version needs to be updated, it will show here. Install the package and refresh the page. Click Next to continue, and the following page will show the SQL schema upgrade information.

3 - Database connection

Enter the MariaDB root password, which should be password if this is the first installation. The SQL schema will be upgraded using the .sql file listed at the bottom. As your instance of NiDB collects more data, the tables can get very large and tables over 100 million rows are possible. This will cause the setup webpage to time out, so there is an option to skip tables that have more than x rows. This should really only be done if a specific table is preventing the schema upgrade because it so large and you are familiar with how to manually update the schema. The debug option is available to test the upgrade without actually changing the table structure. Click Next to continue, and the following page will perform the actual schema upgrade.

4 - Schema upgrade

If any errors occur during upgrade, they will be displayed at the bottom of the page. You can attempt to fix these, or preferably seek help on the NiDB github support page! Click the red box to dismiss any error messages. Click Next to go to the next page which will show the configuration variables.

5 - Config settings

On this page you can edit variables, paths, name of the instance, email configuration, enable features.

Click Write Config to continue.

The locations of the written config file(s) are noted on this page. nidb-cluster.cfg is meant to be placed on cluster nodes, to allow nidb pipelines running on the cluster to communicate with the main nidb instance and perform check-ins and storing of pipeline results.

Setup should now be complete and you can visit the home page.

Managing Subjects

Editing IDs & Demographics

Find your subject by UID or other methods. On the subject's page, you'll see a demographic summary on the left, and the subject's enrollments and studies on the right. The demographics list may show a red box around the DOB if it appears to be a placeholder date like 1900-01-01 or 1776-07-04. On the left-hand side of the page, click the Edit subject button (#2 on the image below). This will show the form to edit demographics. If the edit button is missing, you can check your permissions for the subject by expanding the permissions listing (#1 on the image below).

When editing subject demographic information, required fields are highlighted. Most fields are optional. You can edit IDs on this page, which are reflected for the subject in all projects. The primary alternate ID should have an asterisk in front of it. The primary alternate ID will now be with the main ID for this subject in the specified project. See the ID note below. Click Update to save the demographic information. Confirm on the following page.

Below are some terms used to describe IDs within NiDB

Project Enrollments

On the subject's main page, you'll see a list of enrollments on the right hand side. To enroll the subject in a new project, select the project and click Enroll. This will create a new enrollment.

You can edit an existing enrollment by clicking the Edit Enrollment button. There isn't a whole of information available on this page, but enrollment checklists can be useful to check if a subject has completed all of the items for the project. Subjects can be marked as excluded, complete. Enrollment group can also be specified, such as CONTROL or PATIENT.

Timelines

For projects with a chronological component, you can view a timeline of series for this enrollment. Click the View Timeline button on the subject's page, for the enrollment. It will display a timeline of series. Bottom axis displays the date/time. You can change the parameters of the timeline by selecting date range or series.

Enroll in Different Project

If your subject is enrolled in a project, but you need to move the enrollment (and all of the imaging studies, enrollment info, assessments, measures, drugs, and vitals) into a different project, you can do that by expanding the Enroll in different project section. Select the new project, and click Move. You must be an admin to do this.

Merging

You must have admin permissions to merge subjects. To merge subjects, first go into one of the subject's main pages. On the lefthand side of the page, expand the Operations section. Click Merge with... and it will bring up the merge page. On the top of the page, you can add other UIDs you want to merge. Once all subjects are added (up to 4), they will be displayed side-by-side.

Select the UID to be the final merged UID. Enter all demographic information that will be in the final UID into that column. Once merged, only the information in that column will be saved for the final subject. All other subjects will be marked inactive. All of the imaging studies will be moved from the other subjects to the final subject. When all information is complete, click Merge. The merge will be queued and will be run in the background. Check the status of the merge under My Account → File IO.

Deleting

Only admins can delete subjects. To delete a subject, go to the subject's page. On the lefthand side, expand the Operations section and click the Delete button. It will confirm that you want to delete this subject. Confirm on the next page. Subjects are not actually removed from the NiDB system, but are instead marked as inactive. Inactive subjects do not appear in search results or summaries, but will show up in certain sections of the project page and if searching by UID directly. A subject can be undeleted if necessary.

Undeleting

Subjects can be undeleted by following the same process as deleting a subject, except the Undelete button will appear under the Operations section of the subject's page.

Managing Studies

MRI/DICOM vs non-MRI/DICOM Studies

DICOM Derived - Studies derived from DICOM data are displayed differently than other modalities because they contain detailed header files which are imported automatically. Because of the complex ways in which subject/study/series heirarchy are stored in DICOM files, archiving is done completely automatically.

MRI - MRI studies allow for storage of behavioral data associated with fMRI tasks. Other data such as eye tracking, simultaneous EEG, or other series specific data can be stored in the behavioral data section of MRI series.

All Other Modalities - Series information is less detailed, series can be created manually, and there is no option to store behavioral data for each series.

Editing Study Information

For any modality, edit a study by viewing the study page and clicking the Edit Study button on the lower left of the page. Depending on the modality, different study information may be available.

For non-MRI modalities, the Study date/time (and all series date/times), visit type, visit number, and visit timepoint can be edited directly on the study page without clicking the Edit Study button.

Merging Studies

Occasionally, weird things can happen when importing data such as each series of an MRI study being inserted into it's own study. If the single study had 15 series, it might create 15 seperate studies, each with one series. This can be fixed by merging all of the series into one study. To merge studies (of the same subject/enrollment/modality) together, go into the study page and click the Operations button. A sub-menu will pop up with a Merge Study with... button. A list of available studies will be displayed.

It will display a list of studies of the same modality that can be merged. Choose the study number you want as the final study, and the merge method. CLick Merge, and your merge will be queued. Check the status of the merge by going to My Account → File IO.

Moving Studies

Studies can be moved into different projects (different enrollment) or to different subjects. To move studies, click the Operations button on the bottom left of the study page which will display options for moving the study.

To move to an existing subject, enter the the UID and click Move. To move the study to an existing enrollment (a project the subject is already enrolled in) select the project and click Move.

Managing Series

Viewing series information

For DICOM derived series, most information will be displayed on the main study page. To view a thumbnail of the series, click the icon below the protocol name. To view DICOM header information, click the protocol name. To view detailed QA information, click the chart icon. To view or edit ratings, click the speech bubble icon. To download this series as a zip file, click the download icon under the Files column. To download the behavior data (if MR series) click the download icon under the Beh column. To view a list of files associated with the series, click View file list button.

Editing series information

Series information can only be edited for non-DICOM derived series. To edit the series information (protocol, datetime, notes) click the series number, edit the information, and click Update. To upload new files to the series, drag and drop them onto the Upload button. To manage the existing files, click the Manage n files button. This will display a list of the files associated with this series. Clicking the file name will download the file. Editing the filename in the Rename column will allow you to rename the file (press enter to finalize the rename). Delete the file by clicking the trash icon. Download the entire series as a .zip file by clicking the Download button.

If the study is an MR modality, you can upload behavioral data by dragging and dropping files onto the Upload button. Behavioral files can be edited by clicking on the number under the Beh column.

Series Operations

For non-DICOM series, you can delete series by selecting the series using the checkbox in the rightmost column and clicking the Delete button.

For DICOM-derived series, more operations are available. Select the series you want to perform an operation on and click With Selected.... A menu will pop up with options

Groups

Groups can be created of existing items, such as subjects, studies, or series. This is useful if you need to group subjects together that are in different projects, or if you want to group a subset of studies from one or more projects. Groups can only contain one type of data, ie they can only contain subjects, studies, or series. It is a similar concept to a SQL database View. Groups can be used in the Search page, and pipelines.

Create new group

Under the Subjects menu item, click the Groups menu item. A list of existing groups will be displayed, and a small form to create a new group. To create a new group, enter a group name, select the group type (subject, study, series) and click Create Group.

Click on a group name to edit the group members, or add or delete group members.

Projects

NiDB is a multi-project database. Data from multiple projects can be managed in one database instance. Each project can have different attributes according to the needs of the project.

Creating a Project

A user with admin rights can create, and manage a project in NiDB. A user with Admin rights will have an extra menu option "Admin". To create a new project in NiDB, click "Admin" from the main menu and then click "Projects" as shown in the figure below.

The following page with the option "Create Project" will appear. This page also contains a list of all the current projects. To create a new project, click on the "Create Project" button on the left corner of the screen as shown in the figure below.

On the next page, fill the following form related to the new project. Name the new project, fill the project number. Select the option "Use Custom IDs" if project need to use its own ID system. Select the Principal Investigator (PI) and project administrator (PA) from the existing NiDB users. The PI and PA can be the same subject. Mention the start and end date if they are known. Also there is an option if you want to copy an existing setting from one of your projects.

After clicking "Add" button, a new project will be added to the project list and it will be shown in the list of existing projects as shown in the figure below.

Project Setup

To setup the project for collecting data click the name of the project on the above page and the following page can be used to add the right set of protocols.

After adding the required set of protocols, a list of protocols will be shown as follow. A protocol can be deleted by clicking the "delete" button in front of an added protocol as shown in the figure below.

To define a protocol, click on the name of a protocol in the above list. For example if we click on EEG-Rest, the following page will appear with the existing list of EEG-series being already used in various projects. You can pick any of those to add to your own protocol group. A group name can be assigned by using the "Protocol group name" box at the end of the page as shown. After clicking the "Add" button the selected series in the group will be added to the group and will be shown on the right.

Working With a Project

After setting up project accordingly, the project can be accessed by users having its rights. A user can access a project via "Projects" menu from the main menu. A list of existing projects will be displayed. TO search a specific project, type the name of a project and the list will reduced to the projects containing the search phrase.

Click the name of the project from the list as shown above. A project specific page will appear as seen below.

A project page consists of some information regarding the current project. Under the project name there is total number of subjects and studies. Undeneath that is a message box consists of number of studies. One can dismiss this message box by clicking "dismiss" button or view all the studies inside the message box.

In the middle of a project page is "Data Views" for subjects, studies, checklist for subjects and an option to QC the MR scans.

To update information regarding the subjects in the current project, click on the "Subjects" button in the data view, a page will appear where the information can be updated for all the subjects and can be saved at once by clicking "Save" button at the end.

By clicking the Studies button from Data Views section, following page will appear. The studies can be selected to perform various operations like adding enrollment tags, moving studies to another project.

Checklist provides a brief summary on the subjects, studies and their status as shown below.

On the right side of the project page is a star that can be selected to make this project "favorite" that will show this project on the main page of NiDB to access it easily from there. Also there are links to the project related tools and their settings. This section is named as "Project tools & settings". This section includes:

• Data Dictionary

• Analysis Builder

• Study Templates

• BIDS Protocol Mapping

• NDA Mapping

• Behavioral Minipipelines

• Recap-> NiDB Transfer

• Reset MRI QA

It also possess the parameters required to connect this project remotely.

The last section of a project page consists of a list of subjects registtered, with its alternate Ids, GUID, DOB, Sex, and status as shown below:

The projects main-menu also has a sub-menu to navigate through various project related tools. The sub-menu includes links to Data Dictionary, Assessments, Subjects, Studies, Checklist, MR Scan QC, Behavioral pipeline and Templates. Also "Project List" can navigate back to the list of all the projects in the current database instance.

The admin modules can be accessed by clicking on the Admin menu item. Your account must have administration permissions to see this menu.

Data (Front-end) Administration

Settings that the end-user will see.

• Users - Create and manage users

• Projects - Create and manage projects

• Reports - Create imaging data reports

System (Back end) Administration

Settings that the end-user will not see

• NiDB Settings

• Informational Links

• Backup

• Modules

• Modalities

• Sites

• Instances

• Mass email

• DICOM receiver

Search

Finding Imaging Data

The search page helps to find the imaging data. The following are the parts of the search page that can be used to define and refine the search.

Subject

There are various subsections on the search screen, those are self-explanatory. The first section is “Subject” as shown in the following figure. A search in this section can be defined based on:

1. Subject Ids (UIDs or Alternate UIDs)

2. Name (First or Last)

3. Range on date of birth

4. Range on age

5. Sex-based

6. Subject group

Enrollment

The next section is enrollment where a search can be restricted based on projects. One can choose a single project or a list of projects from the drop down menu. Also a sub-group if defined can be specified.

Study

In this part search parameters / variables in a project / study can be defined to refine the search. A search can be restricted based on, study Ids, Alternative study IDs, range of study date, modality (MRI,EEG, etc.), Institution (In case of multiple institutions), equipment, Physician name, Operator name, visit type, and study group

Series

A more specific search based on protocol, MR sequence, image type. MR TR value, series number (if a specific series of images is needed) and series group can be defined.

Output

In this section, the structure of the search output can be defined. The output can be grouped based on study or all the series together. The output can be stored in “.csv” file using the summary tab. The Analysis tab is used to structure the pipeline analysis results.

Other Data Queries

Other than imaging data can also be quried using the similar way as mentioned above for the imaging data above. However the required non-imaging data modality can be selected from the modality dropdown menu in the study section as shown below

ID Mapping

The Ids can be mapped using the "Data" menu from the main menu. One can go to the Id-maper page by clicking on the "ID mapper" link as shown below or by selection the ID mapper sub-menu.

The following page will appear that is used to map various Ids.

A list of Ids to be mapped separated by space, tab, period, semicolon, colon, comma and newline can be typed in the box above. the mapping can be restricted to a certain project by selecting the project name from the dropdown menu. The search can only be restricted to the current instance, undeleted subjects and exact matches by selecting the approprriate selection box shown above.

Export

After searching the required data, it can be exported to various destinations.

For this purpose a section named "Transfer & Export Data" will appear at the end of a search as shown in a fiigure below.

Following are some destinations where the searched data can be exported:

Export to NFS

To export the data to a NFS location, you can select the "Linux NFS Mount" option and type the NFS path where you want to download the data.

Export to Remote FTP Site

To export the data to a remote FTP location, you can select the "Remote FTP Site" option and type the FTP information where you want to download the data.

Export to Remote NiDB Site

To export the data to a remote NiDB site, you can select the "Remote NiDB Site" option and select the NiDB site from a drop down menue where you want to download the data.

Export via Web Download

You can select the data to be downloased to the local http location. you can select "Web http download" option for this purpose as shown below.

Export to NDAR/ RDoCdb/NDA

NiDB has a unique ability to download the data that is required to submit to NDAR/RDoC/NDA. It automatically prepares the data according to the NDAR submission requirnments. Also one can download the data inforamation in terms of .csv that is required to submit NDAR data. The following the the two options to download the data accordigly.

Export status

After starting the transfer by clicking the transfer button at the end of the search, a transfer request will be send to NiDB. The status of a request can be seen via Search-->Export Status page as shown below. The status of 30 most recent serches will be shown by default. All the previoius searches can be seen by clicking on the "Show all" button on the left corner of the screen as shown below.

Public Downloads

This is another option in the "Transfer and Export" section to transfer "searched data" and to make it as public downloadable. There are options to describe briefly about the dataset, setting up a passowrd for secure tranmitability and making the public download updateable to the users having rights on the data. One can select "Required Registration" option to restrict the dowload to NiDB users only. An expiration date for the download can be set to 7, 30 and 90 days. One should select "No Expiration" if the public data should be available for longer than 90 days or for indefinite period.

The "public Download" will be created after pressing the "Transfer" button at the end of search page. The public downloads can be accessed via Data --> Public Download menue. The following is a a page with Public download information:

Request a Dataset

Sometimes you as a user have no idea how the data is stored for a particular project, or you don't have permissions to the project. If you are lucky enough to have a data manager, you can send a request for data to a data manager who can then follow your instructions to find the data and send it to them.

To request a dataset from NiDB-based database, select Data --> Request a Dataset. The following page will appear.

Click Submit New Dataset Request button, and fill the following form to request a dataset from the NiDB-databse.

Analysis Builder

Analysis builder is a report generating tool that can be used to generate various types of reports using the stored data in NiDB. This report generating tool builds on the base of various types of data variables that is being stored in the NiDB. This is different than the search tool where you can search different types of data and download it. In this tool you can search the variables those are generated and stored / imported in the NiDB (Example: You can query the variables generated from a task using MRI / EEG data, but not the actual EEG and MRI data). Analysis builder can be reached via Search --> Analysis Builder or you can go to a specific project front page and select the option (Analysis Builder) on the right from "Project tool and settings" and you will land on the following screen.

Usage

Analysis builder is designed to create reports based on various types of parameters from different types of measures. It has been categorized in the various types of measures like MR, EEG, ET, etc. as shown below.

After selecting the project from the drop down menu "Select Project", click "Use Project" button. Now the project just selected will be the current project for data retrieval.

Overview

All modules in NiDB system are run from the nidb command line program. Modules are automated by being started from cron.

nidb can be run manually to test modules and get debugging information. It can also be used when running on a cluster to insert results back into the database. Running nidb without command line parameters will display the usage.

Running Modules

Avaiable modules are: import, export, fileio, mriqa, qc, modulemanager, importuploaded, upload, pipeline, minipipeline, and backup

For example, to run the import module, run as the nidb user

This will output

As with all modules, detailed log files are written to /nidb/logs and are kept for 4 days.

Running from cluster

To run nidb from the cluster, for the purpose of inserting results into the database or for checkins while running pipelines, this would be run on the cluster node itself. Access to an nidb.cfg file is necessary to run nidb somewhere other than on the main database server. A second config file /nidb/nidb-cluster.cfg can be copied to the cluster location along with the nidb executable.

pipelinecheckin

To check-in when running a pipeline, use the following

The analysisid is the rowid of the analysis which is bring reported on. Status can include one of the following: started, startedrerun, startedsupplement, processing, completererun, completesupplement, complete. Message can be an string, enclosed in double quotes.

updateanalysis

This option counts the byte size of the analysis directory and number of files and updates the analysis details in the main database.

checkcompleteanalysis

This option checks if the 'complete files' list exists. These files are specified as part of the pipeline definition. If the files exist, the analysis is marked as successfuly complete.

resultinsert

Text, number, and images can be inserted using this command. Examples

Settings

Config variables

The NiDB Settings page contains all configuration variables for the system. These variables can be edited on the Settings page, or by editing the nidb.cfg file. The default path for this file should be /nidb/nidb.cfg. The exact location of the config file is specified on the NiDB Settings page.

PHP Variables

PHP has default resource limits, which may cause issues with NiDB. Limits are increased during the installation/upgrade of NiDB. The current limits are listed on the bottom of the Settings page as a reference if your NiDB installation is not working as expected.

cron

NiDB replaces the crontab for the nidb account with a list of modules required to run NiDB. This crontab is cleared and re-setup with the default nidb crontab each time NiDB is setup/upgraded. Any items you add to the crontab will be erased during an upgrade and need to be setup again.

System messages

At the top of the Settings page, you can specify messages which are displayed system-wide when a user logs in. These can be messages related to planned system down time or other notifications.

Informational Links

NiDB is often run on a network with many other websites such as compute node status, internal Wikis, and project documentation. Links to websites can be specified on the Admin page directly.

Backup

Depending on the size or importance of your data, you may want to backup your data in an off-line format rather than simply mirroring the hard drives onto another server. A backup system is available to permanently archive imaging data onto magnetic tape. LTO tapes are written in triplicate to prevent loss of data. Each tape can be stored in a separate location and data integrity ensured with a majority rules approach to data validation.

Backup process

Backup directory paths are specified in the config file. See the Config variables section.

Data is automatically copied to the backupdir when it is written to the archivedir. Data older than 24 hours is moved from backupdir to backupstagingdir. When backupstagingdir is at least the size of backupsize, then a tape is ready to be written.

Tape 0 lists the current size of the backupstagingdir.

Modules

NiDB has several modules that control backend operations. These can be enabled, disabled, put into debug mode, and the logs viewed.

Enabled modules are listed in green. Running modules will list the process id of the instance of the module. Some modules can have multiple instances running, ie multithreaded, while some modules can only run 1 instance. Each running instance is color-coded with green having checked in recently and red having checked in 2 hours.

Each module has lock file(s) stored in /nidb/lock and log files in /nidb/logs

Module manager

The module manager monitors modules to see if they have crashed, and restarts them if they have. If a module does not checkin within 2 hours (except for the backup module) it is assumed that it has crashed, and the module manager will reset the module by deleting the lock file and removing the database entry.

Modalities

Each modality requires it's own SQL table. Details of the SQL tables, including number of rows and table size, can be viewed on the modalities page.

Sites

Sites are used in various places within NiDB. This section is used when data is collected at multiple sites and stores details about each site.

Instances

NiDB has the ability to separate projects into different instances, basically creating project groups, to which access permissions can be applied. For example, a user can be part of certain instances, giving them the opportunity to view projects within that instance if they have permissions. This can be a good way to group projects from a multi-site project.

Mass email

This will attempt to send an email to every registered email address within the system. It's spam, so use it sparingly.

DICOM receiver

Calendar View

If the calendar is enabled in your NiDB installation, a link on the top menu will be available. The default view is the current week. Click Day, Week, or Month to change view. Click arrows to go forward or backward in the calendar. Click the dropdown list of calendars to change the current calendar. Information about what you are viewing will be displayed at the top of each page.

Managing the Calendars

If you are an admin, you can click the Manage link on the second row of the menu. This will show a list of calendars. Click the Add Calendar button to create a new calendar. Click the calendar name to edit the calendar.

Creating Appointments

On the Day, Week, or Month views, click the + button to add an appointment. Fill out the information, and click Add. Appointments can regular, or can be an all day event or time request, the latter special types do not block overlapping appointments from being created. For example, if a regular appointment is scheduled from 2:00pm-3:00pm, another appointment could not be scheduled from 2:30-3:30pm. But both appointments can be created if either one is a time request or all day event.

Issues

If you try to create an appointment and it says it conflicts with an existing appointment, but you can't see the offending appointment on the calendar, check if there is an appointment that spans more than one day. For example, if an appointment runs from 3pm Monday to 3pm Wednesday, you will not be able to create any appointments on Tuesday. This can be fixed by setting the multi-day appointment to be all day.

Modifying Appointments

Click on the appointment to show appointment form. Make any changes and click Update. You can also modify a repeating appointment, or delete appointment.

Data Hierarchy

Data within NiDB is stored in a hierarchy:

• The top level data item is a subject.

• Subjects are enrolled in projects

• Each subject has imaging studies, associated with an enrollment

• Each study has series

• Each series has files

See diagrams and examples of the hierarchy .

Create Subject

On the main menu, find the Subjects tab. A page will be displayed in which you can search for existing subjects, and a button to create a new subject

1. Subjects page menu item

2. Create Subject button

3. Obliterate subjects button: an intimidating sounding button that only appears for NiDB admins

Fill out as much information as you need. Name, Sex, DOB are required to ensure a unique subject. Most other information is optional. While fields for contact information are available, be mindful and consider whether you really need to fill those out. Chances are that contact information for research participants is already stored in a more temporary location and does not need to exist for as long as the imaging data does.

The subject will now be assigned a UID, but will not be enrolled in any projects. Enroll the subject in the next section.

Importing Subjects from Recap

For a project, subjects can be imported from redcap using an option on the project page as shown below:

Fill the following form requiring information for API connection to redcap and required redcap field names. After providing the required fields click "Subjects Information" button.

If all the above information is correct, then the list of the subjects from redcap will be shown as follows:

There can be four types of subjects in the list. Those are:

1. Ready to Import: are the one those are in redcap and can be imported.

2. Found in an other project: these are present in another project under inthe NiDB database. They can also be imported, but need to be selected to get import.

3. Processing: these are already in the process of being imported and cannot be selected to import.

4. Already exist in the project: these already exist in the current project and cannot be duplicated.

After selecting the required subjects click "Import Selected Subjects" to start the import process.

Enroll Subject in a Project

In the enrollments section, select the project you want to enroll in, and click Enroll. The subject will now be enrolled in the project. Permissions within NiDB are determined by the project, which is in theory associated with an IRB approved protocol. If a subject is not enrolled in a project, the default is to have no permissions to view or edit the subject. Now that the subject is part of a project, you will have permissions to edit the subject's details. Once enrolled, you can edit the enrollment details and create studies.

Create Imaging Study

There are three options for creating studies

1. Create a single empty study for a specific modality

2. Create a single study prefilled with empty series, from a template

3. Create a group of studies with empty series, from a template

Click Create new imaging studies to see these options. To create study templates or project templates, see Study Templates.

Once the study is created, it will appear in the list of imaging studies. Studies are given a unique number starting at 1 in order in which they are created. The studies are sorted by date in this list. While studies will often appear sequential by date and study number, this is because study numbers are incremented by each new study date added, and each new study often occurs at a later date. However, studies may be numbered in any order, regardless of date. If you create several studies for previous dates, if importing older data, if deleting or merging studies, this will cause study numbers to appear random. This is the normal behavior.

Create Single Series/Upload data

MRI and non-MRI data are handled differently, because of the substantial amount of information contained in MRI headers. MRI series are created automatically during import, while all other imaging data can be imported automatically or manually.

MRI

MRI series cannot be created manually, they must be imported as part of a dataset. See Bulk Import of Large Datasets or Automatic Import via DICOM receiver. MRI series can be managed individually after automatic importing has occured.

Non-MRI

Non-MRI data be imported automatically or manually. To manually import non-MRI data, first go into the imaging study. Then fill out the series number, protocol, date, notes. Series number and date are automatically filled, so change these if you need to. When done filling out the fields, click Create Series.

The series will be created, with an option to create another series below it. Upload files by clicking the Upload button, or by dragging and dropping onto the Upload button. If you need to delete or rename files, click the Manage files button. This will display a list of files in that series, and you can rename the file by typing in the filename box.

Bulk Import of Large Datasets

The imaging import page can be accessed by the Data → Import Imaging menu. Because datasets can be large and take hours to days to completely import and archive, they are queued in import jobs. To import a dataset, click the New Import button.

This will bring up the new import page.

Data Location

Data Modality

Destination Project - Data must be imported into an existing project.

Matching Criteria - DICOM data only

After all of the import information is filled out, click Upload. You can view the import by clicking on it. The import has 5 stages, described below.

Once the parsing stage is complete, you will need to select which series you want to import. This step gives you the opportunity to see exactly what datasets were identified in the import. If you were expecting a dataset to be in the import, but it wasn't found, this is a chance to find out why. Parsing issues such as missing data or duplicate datasets are often related to the matching criteria options. Sometimes the uniquely identifying information is not contained in the DICOM field it is supposed to be. That can lead to all series being put into one subject, or a new subject/study created for each series. There are so many ways in which data is organized and uniquely identified, so careful inspection of your data headers is important to select the right options.

After you've selected the series you want to archive, click the Archive button. This will move the import to the next stage and queue the data to be archived.

At the end of archiving, the import should have a complete status. If there are any errors, the import will be marked error and you can view the error messages.

Automatic Import via DICOM receiver

NiDB was originally designed to automatically import MRI data as it is collected on the scanner, so this method of import is the most robust. After each series is reconstructed on the MRI, it is automatically sent to a DICOM node (DICOM receiver running on NiDB). From there, NiDB parses incoming data and will automatically create the subject/enrollment/study/series for each DICOM file it receives.

Bulk Upload of non-MRI data

For non-MRI data, you can upload data in bulk to existing series. For example, if you have a directory full of task files, but each file belongs to a different subject. Rather than go into each subject/study and upload the file individually, you can upload the files as a batch. This method is best when used in conjunction with study templates.

This upload method assumes that you have already created all of the subjects, studies, and series. The series can be empty, or not. To create empty studies by template, see the Create Imaging Study section on use of templates.

Start by searching on the Search page for the series you are interested in uploading data into. For example, search for all 'GoNoGo' TASKs in a particular project. This will show a list of just the series from that project, from the TASK modality, and for existing GoNoGo series. Select the series you want, and go toward the bottom of the page, in the Operations section, click the Batch Upload button.

This will display a list of just those series, with an area to drag&drop files onto. Existing files for each series are displayed on the right side of the page.

Drag and drop files onto those series, and click Refresh Page to view the newly uploaded files.

Users

Accessing the users page

Access the user administration page from the Admin page. Admin page is only accessible if you are logged in as an administrator

Creating Users

NIS Users

Regular Users

To create a regular user, go to Admin → Users. Click the Add User button. Enter their information, including password and email address. The username can be any field, such as an alphanumeric string, or an email address. If the user is given NiDB admin permissions, then they will be able to add/edit users.

Account Self-registration

On public servers, or systems where users are allowed to register themselves, they can create an account and verify their email address to fully register the account. The account will then exist, but they will have no permissions to any projects within NiDB. After a user registers, they will appear on the Admin → Users → All Other Users tab. Click the username to edit their project permissions. Note: be careful allowing users to self-register, for obvious reasons.

Managing Users

There are 3 options of where to find users A) users in the current instance (switch instance by clicking the instance list in the upper left menu) B) users not in the current instance C) deleted users

To manage project permissions for users, go to Admin → Users and click on the username you want to manage. The page can change the name, password, email, admin status, if the account is enabled/disabled, and the projects to which the user has permissions. After changing any information on the page, click the Save button at the bottom of the page. See list of user options and settings below.

Projects

Data collected in the system must be associated with a subject, and that subject must be enrolled in a project. There is a default project in NiDB called Generic Project, but its preferable to create projects parallel to IRB approved studies.

Projects are listed after clicking on the Admin → Projects menu. Clicking the project allows editing of the project options. Clicking the Create Project button will show the new project form. Fill out the form, or edit the form, using the following descriptions of the options

Reports

Reports of imaging studies (often used for billing/accounting purposes on MRI equipment for example) are organized by modality or equipment. Clicking any of the 'year' links will display a calendar for that year with the number of studies per day matching the specified criteria. Clicking the month name will show a report for that month and modality/equipment. Clicking the day will show a report of studies collected on that day.

Data is stored in NiDB using a hierarchy. The root object is the subject. Each subject can be enrolled in multiple projects. Within each of those projects, the subject can have multiple imaging studies with different modalities. And within each imaging study, there are multiple series. Additionally measures and drugs are attached at the enrollment level. And analyses are attached to imaging studies.

Below is an example subject, enrolled in multiple projects, with imaging studies, and analyses.

MR scan parameters defining the quality of a MR scan are displayed on this page. The information is available for each NiDB project. This MR scan quality information page can be accessed from project's main page by clicking on the "MR Scan QC" sub-menu as shown below.

The MR scan quality control page shows the following information for each MR scan stored in the corresponding project. These parameters includes:

• Series Desc. : Description of the MR scan

• Params good? : Green if parameters are within range. Yellow if criteria is not defined and Red if parameters are out of specified range

• Files on disk? : Green if files are found on the specifies location on the disc. Red if the files are found on the specified location.

• Num Files : Actual number of files

• Avg Rating : Average ratings

• Basic QC : Red if parameters are out of specification, yellow if limit is not defined and green for within range

• Disp (X,Y,Z) : X, Y and Z displacement value

• Mot(X,Y,Z) : Motion in X, Y and Z direction.

• Rot(P,R,Y) : Rotation pitch, roll and yaw values.

• SNR : Signal to noise ratio

• FD : Frame-wise Displacement

• DVARS : Root mean square of temporal change of the voxel-wise signal at each time point. (D is for temporal derivative of timecourses, VARS refers to RMS variance over voxels.)

The QC scan quality control table can be download as ".csv" file by clicking "Save As CSV" button at the end of the MR QC table.

Step 1 - Choose an Import Method

There are two main methods to import DICOM data into NiDB

(a) Global Import - used by the DICOM receiver. All files go into the same directory to be archived completely unattended. Filenames must be unique. Preferable if you have a large batch of disorganized DICOM data

(b) Individual Import - import a single zip file or directory. The import can contain subdirectories. This will parse and display the contents of the import and wait until you select which series to archive before any data will be imported. Preferable if you have smaller batches of data, or data that must be parsed differently than the default global method

Step 2 - (a) Global Import

Overview

DICOM files are parsed into subject/study/series groups using 3 DICOM header tags (or set of tags). These are parsed in order.

1. Subject - PatientID (0010,0020) - this uniquely identifies the subject. PatientID will match to existing subjects in the database (regardless of project enrollment) by comparing the UID, and alternate UID fields in NiDB

2. Study - Modality (0008,0020) & StudyDate (0008,0020) & StudyTime (0008,0030) - this set of tags uniquely identifies the study. This will match to existing studies within NiDB. Those existing studies must also be associated with the Subject from the previous step.

3. Series - SeriesNumber (0020,0011) - this uniquely identifies the series. This will match to existing series in NiDB based on this series number, as well as the study and subject from the previous step.

Copy/move DICOM files into the import directory

Check your configuration (Admin-->Settings-->NiDB Config) for the incomingdir variable. It will most likely be /nidb/data/dicomincoming. This will be the directory NiDB will search every minute for new data, which will then be automatically parsed and archived.

From a Linux terminal on the NiDB server, run the following commands as the nidb user to find and copy all dicom files.

You can also move files, instead of copying, by replacing cp with mv. If your files have a different extension, such as .IMG, or no extension, you can change that in the command as well.

Check status of archiving

Go to Admin-->Modules-->import-->View Logs to view the log files generated by the import process. The global import process expects to process a stream of data, where there is no beginning and no end, so the log file will not delineate this particular import from any other data that were found and archived. This import method is also designed to take a stream of potentially random data, and only utilize readable DICOM files.

Here's a sample section of import log file. Log files can be very detailed, but any errors will show up here. Problems

You may ask... where's my data? You can search, on the Search page, by ID, dates, protocol, and other information.

Potential problems

The global import method will group files by the method specified above. If one of those fields are blank for some or all of your data, that could cause the archiving process to create a subject/study/series hierarchy that does not match what you are expecting. Sometimes you will find that each series is placed in it's own study. Or each study is placed in a unique subject.

To troubleshoot these issues, try using the individual import method described below. This allows you to select different matching criteria and preview the data found before archiving it.

Step 2 - (b) Individual Import

This tutorial is based on the Importing data section of the User's guide, but the content on the current page is more detailed. See link to the user's guide:

Go to Data-->Import Imaging. Click the New Import button.

Fill out the required information. Choose if you are uploading a file, or if the data is located in an NFS path. Select the modality and project. Then select the matching criteria, which will determine how the data will structured into a subject/study/series hierarchy. When everything is all set, click Upload.

Check the status of the import by going to Data-->Import Imaging and finding the import that was just created. The current import step will be displayed, and you can click View Import to view more details. Details of the import will be displayed.

If the import has finished parsing, it will ask for your attention. You'll need to review the subjects, studies, and series that were found and then select which series you want to archive.

Why is my data missing??

Sometimes you go to download data from a subject, and it's not there. I don't mean the series are missing from the NiDB website, but the data is actually missing from the disk.

This can happen for a lot of reasons, usually because studies are moved from one subject to another before they are completely archived. Also for the following reasons

• Subjects are merged, but data is not completely copied over on disk

• Subject ID is incorrectly entered on the MR scanner console. This causes a new ID to be generated. If the study is later moved to the correct ID, some data might not be moved over on disk

• A subject is deleted. But since data is never really deleted from NiDB, it's possible that a study was moved to that subject and not all data on disk is copied over

Example

Suppose we have subject S1234ABC. This subject has one study, and ten series in that study. We'd expect to see the following on the website for subject S1234BC study 2.

But, we go to export the data through the search page or through a pipeline, and not all of the series have data! If we look on the disk, we see there are series missing.

That's not good. This could also appear as though all series directories do exist, but if we dig deeper, we find that the dicom directory for each series is missing or empty. So, where's the data? We have to do some detective work.

Let's look around the subject's directory on the disk.

That's interesting, there appears to be another directory. Our study is 2, but there's also a study 1, and it doesn't show up on the NiDB website. Maybe our data is in there? Let's look.

That looks like our data! We can verify by doing a diff between directories that exist in both studies.

If this is the data we are looking for, we can copy all of the data from study 1 to study 2.

After the copying is done, you should be able to go back to the study page, and click the View file list button at the bottom and see all of expected series.

Why do subjects have more than one ID?

A few possible reasons

• Subject can be enrolled in more than one project, and assigned a different ID for each enrollment

• Subjects are assigned more than one ID within a project

• Data are imported from other databases. The subjects retain the original ID and assigned a new ID

• Imaging studies are assigned unique IDs, regardless of subject

Subject IDs

In this example, a subject is enrolled in 3 projects, where each project has a different ID scheme.

• Project 1 has an ID range of 400 to 499

• Project 2 a range of A100 to A200 and B100 to 200

• Project 3 a range of 10000 to 10100

These IDs can be managed within the subject demographics page. On the left hand side of the Subject's page, edit the subject by clicking the Edit Subject button.

Then scroll down part way on the page and you'll see the ID section, where you can enter all IDs, for all projects/enrollments for this subject. This is a list of Alternate Subject IDs. The asterisk * indicates this is the Primary Alternate Subject ID.

In this example, the Testing project has more than one ID. This can happen if a subject is assigned more than one ID, for example the subject was collected under 2 different IDs and merged, or collected at a different site with different ID scheme, or there is more than one ID format for the project.

Study IDs

Some imaging centers give a unique ID every time the participant comes in (yes, this can be a nightmare to organize later on). Imagine subject comes in on 3 different occasions and receives a different subject ID each time. If you are able to associate these IDs back with the same subject, you can treat these as the Study IDs. The default study is the study number appended to the UID, for example S1234ABC1. In NiDB, all other study IDs are considered Alternate Study IDs.

Mapping subject IDs

The simplest way to find a subject by any ID is to use the ID mapper. Go to Data --> ID Mapper. Enter your ID(s) in textbox and click Map IDs. There are some options available to filter by project, instance, only matches, and only active subjects.

The next page will show any matching subjects.

The first column Foreign ID is the ID you searched for. If that ID is found anywhere in the system, there will be details about it in the Local columns to the right.

Deleted? - indicates if this subject has been deleted or not. Deleted subjects are not actually deleted from the system, they are just marked inactive

Alt Subject ID - If the foreign ID was found under this field, it will show up in this column.

Alt Study ID - If the foreign ID was found under this field, it will be show in this column.

UID - If a subject was found, the UID will be displayed in this column

Enrollment - There may be more than one row found for each foreign ID, and more than one ID for the enrollment in each row. The enrollment will be displayed in this column.

Click on the UID to see your subject.

Step 1

Gather the following information from Redcap administrator for API connection .

• Redcap Server

• Redcap API Token

Step 2

Use 'Projects' menu in NiDB to get to the desired project's main page. From Data Transfer section of the links on the right, click "Global Redcap Settings" link.

Step 3

• Enter the Redcap server address and API token information and press "Update Connection Settings" button on the right as shown below.

Step 4

Next steps to import the data correctly from redcap into NiDB is testing the connection, mapping each variable / field from redcap to NiDB and transferring the data. To test and established the connection with Redcap follow the following steps:

• Go to the project page and click the "Import from Redcap" link as shown below.

• Click on the "Connect To Redcap" button on the right. If the connection is successful, a table with the Redcap Project information as shown below will appear.

Step 5

The Mapping / Transfer page will appear as shown below. This page is used to map variables or transfer data according to established mapping.

To start new or edit existing mapping Click on the "Edit Mapping" button on the left as shown in the above figure. A new page will appear as shown below.

Each Redcap form is required to map separately. Pick NiDB data type and "Redcap Form" from the drop-down list shown above.

Select a type of data that redcap form contains. NiDB handles this in three types of data, which are following:

• Measures: Redcap forms storing cognitive measures and similar other measures are stored as this data form in NiDB

• Vitals: Redcap forms that contains information of vitals like hearth rate, blood pressure, blood test results are stored as this form of data. Also any tests that need to be done multiple times in a day can be recorded as this form.

• Drug / dose: If your project have information related to administrating drugs, this type of Redcap form is stored as Drugs / Dose in NiDB.

After choosing the Redcap "Form", a new section to map the variables from Redcap to NiDB will appear as shown in the figure below.

A variable mapping table has two sides: NiDB and Recap.

NiDB Variable Side

The NiDB variable side contains two columns. These columns will automatically filled with the same variable and instrument names based on the Redcap choices of the form and variables. However, one can change these names. These are the names that will be stored in NiDB for corresponding Redcap Form and variable names.

Redcap Variable Side

This side has seven columns. Following is the explanation of each column on Redcap side.

• Event: A Redcap project can have multiple events. All the events will be listed in this column. Any number of events can be chosen from the list that is needed to map. In our example we chose only one event because the Redcap form selected tp map contain only data for that event.

• Value: Pick the Redcap variable to map from a dropdown menu list.

• Date: Pick the Redcap variable storing "date" information of the redcap form from a dropdown menu list.

• Rater: Pick the Redcap variable storing "rater" information from a dropdown menu list.

• Notes: Pick the Redcap variable storing "notes" information from a dropdown menu list.

• Start / End Time: Pick the Redcap variable storing "start and end time" information form from a dropdown menu list.

Defining the correct type of field is very crucial for the mapping in NiDB. Especially time and date are very important to create reports based on the information stored in NiDB.

Add the Mapping

After defining one variable in a form, hit "Add" button on the right to add this mapping definition.

In case of any mistake, a mapping item can be deleted and later can be added again according to the above stated process.

After completing the mapping for a redcap form. Complete mapping the other redcap forms similarly.

Step 6

Before the last step it is critical to recheck all the mapping information. It is important, because the integrity, and accuracy of data transfer is based on accurate mapping. So check, recheck and make sure!

After you have done with your recheck, you are ready to transfer the data from Redcap to NiDB.

• Click on the "Transfer Data" button, the following screen will appear.

• First select the NiDB instrument (mapped in the mapping step) to transfer the data for.

• Select the redcap variable storing the NiDB unique id of the subjects. This is the variable that connects the records in the Redcap and NiDB.

• Click the "Transfer" button to transfer data. This may take some time and the data transfer information will be displayed after data is transferred.

Tips / Information

You can complete all the mapping for the Redcap forms to be exported at once and then transfer the data one by one OR you can transfer the data of one Redcap form mapped and then go to the next forms to map and transfer.

To transfer / synchronized the data, Just press the "Transfer" button on the right The data will be transferred / synchronized for the selected NiDB instrument.

You need to transfer the data for each mapped instrument separately by selecting them one by one.

Reports on data can be generated by using the "Analysis Builder" tool, selection from a project's main page from "Tools" section on the right.

Reports in Analysis Builder

Analysis builder is a report generating tool in NiDB. In Analysis builder, a report can be build using variables extracted from various types of imaging data, pipelines and biological or cognitive measures. This tool works different than the search tool where you can search stored data (Imaging or Other) and download it. In this tool you can search the variables those are generated and stored or imported in the NiDB (For example: You can query the variables generated from a task using MRI / EEG data OR variables imported from Redcap). Analysis builder can be invoked from a project's main page by selecting the option Analysis Builder on the right from Tools section. Following is the main interface of the Analysis Builder

The interface for Analysis Builder is self explanatory. The main sections consists of selecting a project from the dropdown list, selecting the desired variables, drugs / dose information, choosing the various report parameters like grouping, value replacing a blank or missing entry, and finally the output format of the report.

In the next section the steps to create two reports are listed showing how various options can be employed to create a desired report.

Building Reports

Analysis builder is designed to create reports based on variables that can be selected from different types of measures, and modalities shown in the Analysis Builder interface screen above. This includes modalities like MR, EEG, cognitive and biological measurements.

Simple Report

Following are the steps to create a simple report where data is not grouped and there is no drug / dose variable is used. The following figure shows the selection of variables and settings to generate this report.

Steps - Simple Report

1. To generate a report, select a project from the dropdown menu at the top of the screen.

2. Select the variables for any one or combination of modalities and measures. We chose four cognitive variables those are imported from Redcap.

3. Choose if you want to group data on the base of date, or measure. We are not selecting this option for the simple report.

4. The output of a report can be control by various option like:

   • Selection of all subjects data regardless of their data presence

   • What should be displayed for a blank and missing values

   • Inclusion of, event duration, time, date, subject's height, weight and date of birth

5. Select the output format of the report, showing it on the screen or saving it as csv file.

6. Hit the Update Summary button to generate the final report as shown on the right section of the screen below.

Repeated Measures Report

This is a report that involve the variables which are repetitive in nature. Also we want the variables to display with respect to the drug administered time. Analysis Builder will automatically create variables that holds the time since dose information.

For this type of report, usually time repeated measure are reported, but other measures may be added if desired.

Steps - Repeated Measure Report

1. Select a project from the dropdown list of projects on the top of Analysis Builder interface.

2. Choose the variables to display in this report. As mentioned above this is a repetitive measures report, so the variables that are repetitive in nature; collected multiple times in a day and on multiple days are selected. These include measure from MRI and biological data. Also these variables are collected before or after administration of drug.

3. To include the drug / dose information on this report, select the drug related variable as shown in the above figure in green rectangle. Different drug dose was administrated on three different days, and all three days were selected.

4. The time since dose variables shall be calculated and displayed if the option Include Time Since Dose is selected as shown above. All three dose day variables are also selected. The time will be displayed in minutes as per above selection.

5. To group the data based on drug days, check the Group by Event Date checkbox from Grouping Option.

6. After choosing the output parameters, hit the Update Summary button that generates a report as shown in the figure below.

There are many reports that can be generated using Analysis Builder based on the data stored in a project. More exploration on the Analysis Builder with different options is recommended.

Sometimes you need to rename a series. Maybe it was collected in error or was a bad series; you want to keep it, but rename it so the series isn't automatically picked up in pipeline processes or searches.

Renaming MR series

On the study page, find the series, or series(s), you want to rename. On the rightmost column, check the checkbox(es) for the series. Scroll down to the With Selected... menu.

From the menu, select Rename

Rename the series(s) and click Rename.

The series(s) will now be renamed.

Renaming non-MR series

Click the protocol name for the series, and an edit box will appear. Edit the protocol (series name) and press enter.

Followings are three sections about Importing data and subjects from Redcap and creating reports using the data that is imported from Redcap.

• Importing Data from a Redcap Project

• Importing Subjects from Redcap

• Reports based on data imported from Redcap

NiDB users can have many different permissions, from complete system administration to read-only access. Most users will fall into the project-based permissions. Below are the description of each permission. Protected health information (PHI) and personally identifiable information (PII) are both referred to as PHI below.

Why would anyone want to do this?

There exists the possibility that you may need to completely erase all data from an NiDB installation. Maybe you were importing a bunch of test data and now you want to wipe it clean without reinstalling NiDB. Whatever your reason, you want to make an existing installation clean.

This procedure is not part of NiDB and there are no scripts or automated ways to do this because of the possibility of accidents. You may want to completely empty your refrigerator and toss all food in the trash, but you don't want a button available on the side of the fridge to do it.

How to Clean the System

Database

Truncate all tables except the following

• instance

• modalities

• modules

• users

Filesystem

Clear the contents of the following directories. Only delete the files in the directories, do not delete the directories.

/nidb/data/archive
/nidb/data/backup
/nidb/data/backupupstaging
/nidb/data/deleted
/nidb/data/dicomincoming
/nidb/data/download
/nidb/data/ftp
/nidb/data/problem
/nidb/data/tmp
/nidb/data/upload
/nidb/data/uploaded
/nidb/data/uploadstaging

There is no need to clear the log files or lock files or any other directories.

NiDB should now be ready to import new data.

The squirrel data format allows sharing of all information necessary to recreate an experiment and its results, from raw to analyzed data, and experiment parameters to analysis pipelines.

The squirrel format specification is implemented in NiDB. A DICOM-to-squirrel converter, and squirrel validator are available.

The squirrel command line program allows converstion of DICOM to squirrel, BIDS to squirrel, modification of existing squirrel packages, and listing of information from packages.

Installing squirrel utilities

Download squirrel from https://github.com/gbook/squirrel/releases

sudo yum localinstall --nogpgcheck squirrel-xxx.xx.xxx-1.elx.x86_64.rpm

sudo apt install p7zip # p7zip required by squirrel
sudo dpkg -i squirrel_xxxx.xx.xxx.deb

# increase open file limit (temporarily for the current session)
ulimit -n 2048

# increase open file limit (permanently)
# append these lines to /etc/security/limits.conf
*               soft    nofile            2048
*               hard    nofile            2048

Basic Command Line Usage

Convert DICOM to squirrel

# Default DICOM to squirrel conversion
squirrel dicom2squirrel /path/to/dicoms outPackgeName.sqrl

# Specify the output format
squirrel dicom2squirrel /path/to/dicoms outPackge.sqrl --dataformat niti4gz

# Specify the package directory format
squirrel dicom2squirrel /path/to/dicoms outPackage.sqrl --dirformat seq

Convert BIDS to squirrel

squirrel bids2squirrel /path/to/bids outPackage.sqrl

Modify existing squirrel package

# add a subject to a package
squirrel modify /path/to/package.sqrl --add subject --datapath /path/to/new/data --objectdata 'SubjectID=S1234ABC&DateOfBorth=199-12-31&Sex=M&Gender=M'

# remove a study (remove study 1 from subject S1234ABC)
squirrel modify /path/to/package.sqrl --remove study --subjectid S1234ABC --objectid 1

List information about a squirrel package

#list package information
[user@hostname]$ squirrel info ~/testing.sqrl
Squirrel Package: /home/nidb/testing.sqrl
  DataFormat: orig
  Date: Thu May 23 16:16:16 2024
  Description: Dataset description
  DirectoryFormat (subject, study, series): orig, orig, orig
  FileMode: ExistingPackage
  Files:
    314 files
    19181701506 bytes (unzipped)
  PackageName: Squirrel package
  SquirrelBuild: 2024.5.218
  SquirrelVersion: 1.0
  Objects:
    ├── 8 subjects
    │  ├── 8 measures
    │  ├── 0 drugs
    │  ├── 11 studies
    │  ├──── 314 series
    │  └──── 0 analyses
    ├── 0 experiments
    ├── 0 pipelines
    ├── 0 group analyses
    └── 0 data dictionary
    
# list subjects
[user@hostname]$ squirrel info ~/testing.sqrl --object subject
Subjects: sub-ASDS3050KAE sub-ASDS6316BWH sub-ASDS6634GJK sub-ASDS7478SKA sub-ASDS8498GQDCBT sub-HCS8276XPS sub-S4328FSC sub-S7508DDH

# list studies for a specific subject
[user@hostname]$ squirrel info ~/testing.sqrl --object study --subjectid sub-ASDS3050KAE
Studies: 1 2

#list all subjects as CSV format
[user@hostname]$ squirrel info ~/testing.sqrl --object subject --csv
ID, AlternateIDs, DateOfBirth, Ethnicity1, Ethnicity2, GUID, Gender, Sex
"sub-ASDS3050KAE","","","","","","U","U"
"sub-ASDS6316BWH","","","","","","U","U"
"sub-ASDS6634GJK","","","","","","U","U"
"sub-ASDS7478SKA","","","","","","U","U"
"sub-ASDS8498GQDCBT","","","","","","U","U"
"sub-HCS8276XPS","","","","","","U","U"
"sub-S4328FSC","","","","","","",""
"sub-S7508DDH","","","","","","",""

Permissions within NiDB are based on project. This probably corresponds an IRB approved project, such that certain personnel should have access to the project.

To add (or remove) a user's permissions to a project

1. Go to Admin-->User

2. Click on the username of the user you want to change

3. If you want to give the user global admin permissions within NiDB, check the NiDB Admin box at the top of the page. This allows the user to assign permissions to other users, and allows users to delete subjects, studies, and series, for the projects they have permissions for

4. To give permissions to specific projects, check the boxes next to the project name.

   1. Project admin allows them to enroll subjects into that project

   2. View/Edit data/PHI are not clearly separated in their functionality. Just select all of these to allow the user access to the project.

5. To remove permissions to specific projects, uncheck the appropriate boxes

6. Click Update at the bottom of the page

System-wide settings are stored in the config file. The default location is /nidb/nidb.cfg.

The NiDB Settings page allows you to edit the configuration file directly. When the page is saved, the config file is updated. But the config file can be edited manually, which is useful when the website is unavailable or you need to edit settings through the command line. To edit the file by hand, start vim from a terminal. (if vim is not be installed on your system, run sudo yum install vim)

vim /nidb/nidb.cfg

This will start vim in the terminal. Within vim:

1. Use the arrow keys to navigate to the variable you want to edit

2. Press the [insert] key

3. Edit as normal

4. When done editing, press [esc] key

5. Type :wq which will save the file and quit vim

Special Config Variables

Some variables can only be changed by editing the config file directly and cannot be changed from the NiDB settings page.

offline - Set to 1 if the website should be unavailable to users, 0 for normal access. Default is 0

debug - Set to 1 if the website should print out every SQL statement, and other debug information. Default is 0

hideerrors - Set to 1 if the website should hide SQL errors from the user. 0 otherwise. Default is 0

BIDS and squirrel are both file formats designed to store neuroimaging data. They are similar, but different in implementation. If you are familiar with BIDS, squirrel will be easy to understand.

squirrel vs BIDS objects

The squirrel library is built using the Qt framework and gdcm. Both are available as open-source, and make development of the squirrel library much more efficient.

Including squirrel

The squirrel library can be included at the top of your program. Make sure the path to the squirrel library is in the INCLUDE path for your compiler.

#include "squirrel.h"

Reading

Create an object and read an existing squirrel package

squirrel *sqrl = new squirrel();
sqrl->SetPackagePath("/path/to/data.sqrl");
if (sqrl->Read()) {
    cout << "Successfuly read squirrel package" << endl;
else
    cout << "Error reading squirrel package. Log [" << sqrl->GetLog() << "]" << endl;

/* print the entire package */
sqrl->Print();

/* access individual package meta-data */
cout << sqrl->name;

/* delete squirrel object */
delete sqrl;

Iterating subject/study/series data

Functions are provided to retrieve lists of objects.

/* iterate through the subjects */
QList<squirrelSubject> subjects = sqrl->GetSubjectList();
foreach (squirrelSubject subject, subjects) {
    cout <<"Found subject [" + subject.ID + "]");

    /* get studies */
    QList<squirrelStudy> studies = sqrl->GetStudyList(subject.GetObjectID());
    foreach (squirrelStudy study, studies) {
        n->Log(QString("Found study [%1]").arg(study.StudyNumber));

        /* get series */
        QList<squirrelSeries> serieses = sqrl->GetSeriesList(study.GetObjectID());
        foreach (squirrelSeries series, serieses) {
            n->Log(QString("Found series [%1]").arg(series.SeriesNumber));
            int numfiles = series.files.size();
        }
    }
}