Developing a tool to easily store data and study details

sim2bids

GitHub repository Documentation page PyPI page

The project consists of an easy-to-use GUI that is built using the Holoviz Panel package. The approach I took to building the app was to minimize manual work as much as possible. Fortunately, my mentors gave me the green light to come up with ideas and implement what I had in my mind. The app includes several great features to ease researchers’ lives:

• First, it can run both locally and on a server, it's deployed on PyPI and can be accessed from anywhere.
• Second, users do not need to go through a painful manual process of renaming files. They can pass the files to the app, go to the Preprocess Data tab, and select which files to rename into which standard naming schemes. This process is applied for every file having a unique extension. For example, if they have 100 subjects and there's a file called SC (structural connectivity), they can safely select the "weights" file from a drop-down menu. All files having "SC" will then be renamed.
• Third, users can change the converted JSON files directly in the app. For that, they need to navigate to the View Results tab and select the file of their choice. There is an additional feature that changes the changed field in other files as well. Of course, the app will not change the critical parts that are unique for each file.
• Last, the app comes with an in-app user guide which can be found in the User Guide tab. It explains how to use the app, explains any required files for renaming purposes, and more.

All the aforementioned functionalities are shown in the video below. The video shows the app running on an inline Jupyter notebook on the EBRAINS server.

Accepted files

The app supports a wide range of files, specifically: text (.txt), tab-separated values (.tsv), generic format (.dat), NumPy arrays (.npy), MATLAB files (.mat), HDF5 (.h5), and zip folders to extract network and coordinates files. It also supports three programming languages: Python, MATLAB, and R to extract model-specific parameters.

How to use the app

First, we need to download the package which can be easily done using the following command:

!pip install sim2bids

Then, run the following commands in the Jupyter notebook/Terminal/Python file:

import sim2bids
import panel as pn
pn.extension('tabulator', 'ace', 'jsoneditor', 'ipywidgets', sizing_mode='stretch_width', notifications=True)
pn.serve(sim2bids.sim2bids.MainArea().view())

Output

Below you can see the output folder for simulation data conversion where each file is shown in its respective folders. For example, all coordinate files are stored in the global `coord` location. It is also important to note that these simulations were run on TVB v2.6. The app captures the source code to replicate the study.

comp_validator

GitHub repository PyPI page

One of the requirements for the project was to also create a BIDS validator for computational data since it's not covered in the current form of the BIDS validator. The current BIDS validator app mainly runs on the website where users can upload their folders directly. For our use case, it was important to use Python-based implementation which can be run directly in the app or in the Jupyter notebook.
The custom BIDS validator for BEP034, comp_validator, has the same structure and rules as specified in the previous BIDS validator. However, it also introduces naming conventions, data types, and accepted files as specified in the BEP034 proposal. It outputs two different Markdown files covering errors and warnings. BEP034 proposal. It output two different Markdown files covering errors and warnings.

How to use the comp_validator app

First, we need to download the package which can be easily done using the following command:

!pip install comp_validator

Then, run the following commands in the Jupyter notebook/Terminal/Python file:

import comp_validator.comp_validator as validate
# specify path to the converted folder
validate.validate(PATH)

Output

Possible outputs of these commands are shown below:

Errors Markdown file

Warnings Markdown file