NeuroJ - Browse, Query, and Convert Neuroimaging Datasets

NeuroJ

Browse, query, and convert neuroimaging datasets from NeuroJSON.io

Version 0.8 Shell/Perl RESTful API BSD-3-Clause NIH Funded

🌐 What is NeuroJ?

NeuroJ is a lightweight shell-based toolkit that provides simple utilities to browse, query, and download thousands of neuroimaging datasets from NeuroJSON.io, and convert arbitrary datasets (including BIDS) to JSON format ready for upload to NoSQL databases.

It bridges traditional neuroimaging data formats with modern document-store databases (MongoDB, CouchDB, Redis) enabling scalable, searchable, and reusable data sharing.

🏆 NeuroJSON Project

NeuroJ is the command-line client for NeuroJSON.io, funded by US NIH grant U24-NS124027. It provides essential tools for dataset conversion, querying, and interaction with the NeuroJSON data repository.

1000s
Datasets Available
2
Core Tools
REST
API Access
NoSQL
Database Ready
View on GitHub NeuroJSON.io See Examples

🔄 Data Conversion Strategy

NeuroJ separates datasets into searchable and non-searchable components for optimal NoSQL database performance:

Data Structure CouchDB/NoSQL Object Examples
Dataset Collection CouchDB Database openneuro, dandi, openfnirs
Single Dataset CouchDB Document ds000001, ds000002
Subject Files/Folders JSON Keys in Document sub-01, sub-01/anat/scan.tsv
Small Binary (readable) Document Attachment .png, .jpg, .pdf files
Large Binary (raw data) _DataLink_ External Link .nii.gz, .snirf data files
View Examples

⚡ Key Capabilities

  • Browse & Query: List and search thousands of datasets on NeuroJSON.io
  • RESTful API: Full access to CouchDB REST API for advanced queries
  • BIDS Support: Convert BIDS datasets to JSON automatically
  • Parallel Conversion: Multi-threaded processing for large collections
  • Format Support: .json, .tsv, .csv, .nii.gz, .snirf, and more
  • Data Linking: External binary data linked via _DataLink_ annotations
  • Upload/Download: Push datasets to and pull from NeuroJSON.io
  • NoSQL Ready: JSON output ready for MongoDB, CouchDB, Redis
Get Started Browse Datasets

Why Use NeuroJ?

Modern data management for neuroimaging

🔍

Search at Scale

Query millions of datasets in milliseconds using NoSQL database indexing

🌐

Web-Ready Data

JSON format enables direct integration with web applications and APIs

🔄

BIDS Conversion

Automatically convert BIDS datasets to searchable JSON documents

⚡

Parallel Processing

Multi-threaded conversion for fast processing of large collections

📊

RESTful Access

Full CouchDB REST API support for advanced queries and automation

🔗

Smart Linking

External data linking keeps documents small and queries fast

Usage Examples

Command-line interface for dataset management

# Convert entire database to JSON (dry-run)
neuroj -i /path/to/database/openneuro \
       -o /path/to/output/json -db openneuro

# Convert database with 12 parallel threads
neuroj -i /path/to/database/openneuro \
       -o /path/to/output/json -db openneuro \
       --convert -t 12

# Convert single dataset ds000001 to JSON
neuroj -i /path/to/database/openneuro \
       -o /path/to/output/json \
       -db openneuro -ds ds000001 --convert

# Auto-detect database name from path
neuroj -i /path/to/database/databasename \
       -o /path/to/output/json
# List all databases on NeuroJSON.io
neuroj --list

# List with formatted output using jq
neuroj --list | jq '.'

# List all datasets in openneuro database
neuroj --list -db openneuro

# Print dataset IDs using jq filter
neuroj --list -db openneuro | jq '.rows[] | .id'

# Query database information (count, size, etc)
neuroj --info -db openneuro

# Search using CouchDB _find API
neuroj -db openneuro --find \
  '{"selector":{},"fields":["_id"],"limit":10,"skip":2}'

# Complex query with dataset_description
neuroj -db openneuro --find \
  '{"selector":{},"fields":["_id","bids_dataset_info.dataset_description\\\\.json"],"limit":2}' \
  | jq '.'
# Retrieve and display dataset JSON
neuroj --get -db openneuro -ds ds000001

# Download entire database (slow, large)
neuroj --get -db openneuro

# Retrieve specific dataset revision
neuroj --get -db openneuro -ds ds000001 -v revision_hash

# Save output to file
neuroj --get -db openneuro -ds ds000001 > ds000001.json

# Use custom NeuroJSON.io server URL
neuroj --get -db mydb -ds mydataset \
       --url https://myserver.com:7777
# Admin tasks require authentication
# Method 1: Use .netrc file (recommended)
neuroj -n --put dataset.json -db openneuro

# Method 2: Set NEUROJSON_IO environment variable
export NEUROJSON_IO="https://user:pass@neurojson.io:7777"
neuroj --put dataset.json -db openneuro

# Method 3: Use -U/-P flags (less secure)
neuroj -U admin -P password --put dataset.json -db openneuro

# Upload dataset (auto-detect name from filename)
neuroj --put ds000001.json -db openneuro

# Upload with explicit dataset name
neuroj --put mydata.json -db openneuro -ds ds000999

# Create new database (admin only)
neuroj --create -db newdatabase

# Delete dataset (admin only)
neuroj --delete -db openneuro -ds ds000001
# njprep: Convert datasets/files to JSON

# Convert all datasets in a database
njprep /database/root/ /output/json/root/ database_name

# Convert specific dataset in database
njprep /database/root/ /output/json/root/ \
       database_name dataset_name

# Convert single file or subject folder
njprep /database/root/ /output/json/root/ \
       database_name dataset_name /path/to/file

# Supported conversions:
# - .json files: copied as-is
# - .tsv/.csv: converted to JSON arrays
# - .nii.gz: header to JSON, data to binary
# - .snirf: HDF5 to JSON structure
# - symlinks: converted to _DataLink_ JSON
# - .png/.jpg/.pdf: stored as attachments

Key Features

Complete neuroimaging data management toolkit

🗄️

Database Access

Browse and query thousands of public datasets on NeuroJSON.io

🔄

BIDS to JSON

Automatic conversion of BIDS datasets to searchable JSON format

⚡

Parallel Processing

Multi-threaded conversion for handling large dataset collections

🔍

Advanced Search

CouchDB _find API integration for complex dataset queries

📤

Upload/Download

Push converted datasets to NeuroJSON.io or pull existing ones

🔗

Smart Data Linking

Separate searchable metadata from large binary files efficiently

Tools & Utilities

Complete command-line toolkit

Primary Tools

  • neuroj - Main client for NeuroJSON.io
  • njprep - Dataset/file converter

Conversion Utilities

  • bids2json - Merge dataset JSONs
  • mergejson - Merge subject JSONs
  • tsv2json - TSV/CSV to JSON
  • link2json - Symlink converter

Data Management

  • listdatalink - Extract _DataLink_ URLs
  • --list - List databases/datasets
  • --info - Query database info
  • --find - Search with selectors

neuroj Flags

  • -i, --input - Input folder path
  • -o, --output - Output folder path
  • -db, --database - Database name
  • -ds, --dataset - Dataset name
  • -r, --convert - Run conversion
  • -t, --threads - Thread count

Query Flags

  • -l, --list - List resources
  • -q, --info - Query information
  • -f, --find - Search datasets
  • -g, --get - Retrieve data

Admin Flags

  • -p, --put - Upload dataset
  • -c, --create - Create database
  • -d, --delete - Delete dataset
  • -n - Use .netrc auth
  • -U, --user - Username
  • -P, --pass - Password

Installation

Setup in minutes

🐧 Linux Installation

Clone and install dependencies:

git clone --recursive \
  https://github.com/NeuroJSON/neuroj.git

# Install dependencies
sudo apt-get install jq curl octave \
  libparallel-forkmanager-perl \
  libwww-perl libjson-xs-perl

🍎 macOS Installation

Using Homebrew:

git clone --recursive \
  https://github.com/NeuroJSON/neuroj.git

# Install dependencies via Homebrew
brew install jq curl octave

# Install Perl modules
cpan Parallel::ForkManager LWP::UserAgent JSON::XS

🪟 Windows (Cygwin/MSYS2)

Install Cygwin or MSYS2 first:

# In Cygwin/MSYS2 terminal:
git clone --recursive \
  https://github.com/NeuroJSON/neuroj.git

# Install packages via package manager
# (specific commands depend on Cygwin/MSYS2)

🐳 Docker

Pre-configured environment:

# Pull Docker image
docker pull openjdata/neuroj

# Run neuroj in container
docker run -it openjdata/neuroj neuroj --list

📦 Add to PATH

Make commands globally available:

# Add to ~/.bashrc or ~/.zshrc
export PATH="/path/to/neuroj/bin:$PATH"

# Reload shell
source ~/.bashrc

📚 Dependencies

Required components:

  • jq - JSON processor
  • curl - HTTP client
  • GNU Octave - For .nii/.snirf conversion
  • jbids toolbox (included as submodule)
  • Perl modules (parallel, LWP, JSON::XS)

Ready to Access Neuroimaging Data?

Connect to thousands of open datasets

1000s
Available Datasets
0.8
Current Version
REST
API Ready
Download Latest Release Visit NeuroJSON.io Docker Image

Supported by NIH Grant U24-NS124027

BSD-3-Clause License

Powered by Habitat