Summary:

This paper describes a multi-threaded implementation of a previously published techniuqe for stochastic fiber tracking.  The paper is well written and addresses an important issue of providing advanced fiber tracking algorithms to the imaging community.  The authors provide their implementation as an ITK filter, a stand alone command line program, and a module for Slicer.

 Hypothesis:

A multi-threaded approach for stochastic fiber tracking will improve performance.

 Evidence:

The authors demonstrate with a table a substantial improvement in computation time when using a multi-threaded approach.

Open Science:

The authors provide their code as an ITK filter, a stand alone command line program and a module in Slicer.

 Reproducibility:

I did not attempt to reproduce the author's work or download/compile the code.

Requests for additional information:

In figure 1, what was the ROI used to generate this connectivity map?  The authors should also define what the colors represent.

How is the matrix set for the SetMeasurementFrame function call?  i.e. how are the imaging and diffusion gradients used to generate this matrix?

 When creating the connectivity map for a collection of seedpoints, does changing the seedpoint and rerunning the filter update the connectivity map and sampled tracts to contain the combined statistics for all seedpoints, or do the individual maps need to be combined outside the filter?

 Is there a reference as to how to put DTI data into the NRRD format?

 Additional Comments:

The authors state in the abstract that this technique can be used to study abnormailites in fiber tracks.  How will this be done?  Does this technique offer unique methods for studying fiber track abnormalities as compared with standard streamlining techniques?

 How does the multi-threaded computation time compare with streamlining techniques? 

What is the memory requirements for a typical dataset?

Overall, this work will provide a valuable resource to the imaging/scientific community.

Summary:

The authors present a multi-threaded implementation of a published stochastic tractography algorithm (Friman, TMI 2006). The paper contains a brief review of the algorithm, a discussion of the implementation strategy, and an overview of the API. The algorithm is available via an ITK class and a Slicer3-compatible command-line program. It appears that this implementation will be very useful to the community studying white matter connectivity.

Evidence:

A screen shot of the filter's output in Slicer3 indicates that the code works as intended. No other validation of the code/method is presented.

Reproducibility:

Based on the authors' description, it appears that it will be very easy to apply their algorithm to new data from the command line or using Slicer3.

Use of Open Source Software:

Yes. ITK, Slicer.

Code Quality:

Not assessed.

Suggestions for future work:
Are tests provided with the implementation? It would be helpful to mention them in the paper.

It would be helpful to write a step-by-step tutorial for the Slicer3 module.


Additional Comments:

Minor comments and personal opinions:

There are several magic numbers and unexplained abbreviations within the paper.

For figure 4 (Computation time vs. Total Sampled Tracts), it might be easier to digest/understand the effect of parallelization if, rather than the current plot, Speedup vs. Number of Threads is plotted for several problems sizes.

I feel that the discussion of ITK's multithreading framework is too detailed (Section3, paragraph 2).

I feel that the discussion of your mutex lock strategy is important but too detailed.

It would be helpful to include some equations that relate the inputs of the filter (including, for example, MeasurementFrame) to the outputs in a precise and concise way.

Summary:
A very well written paper on open-source code for a previously published Bayesian approach to DTI fiber tractography.

The authors provide a multi-threaded, ITK-filter-based implementation of the DTI algorithm as well as a command-line executable that follows the Slicer execution model (so that the executable can be called from the command-line or optionally generates a GUI within Slicer and can thereby be called from within Slicer).

The use of the filters is discussed, but demonstrations/evaluations of the filter are not specifically given - although data is distributed with the source.

Hypothesis:
DTI fiber tracking is one of the hot-topics in medical image analysis and this submission provides an outstanding basis for others to apply DTI fiber tracking to data and for others to conduct further algorithm research.

Evidence:
The source code is provided.  Sample data is provided.

Regretfully, the authors refer to different data in the paper than what is distributed with the code. 

Open Science:
The authors very effectively address open-science.  Their code compiled without problems.  The code is offered as an ITK filter, a command-line executable, and as a Slicer module so that an extremely wide range of users can benefit from it.

Reproducibility:
Compiled without problems.

It appears as though the authors attempted to comply with the IJ automatic compilation strategy, but the code was not recognized by the IJ and no dashboard review was generated.  The authors should contact the IJ to determine why the process failed.

Use of Open Source Software:
The license and copyright of the code is not given.  Hopefully it would follow a BSD-style license.

The code makes use of Slicer and ITK.  Slicer is not required to build the software (the necessary Slicer components are redistributed with the code). 

Code Quality:
The variable names, etc follow most of the standards of ITK.  The line length limit of 80 characters is often exceeded, making review more difficult.  Comments are a bit sparse, but effective when present.   It will be easy for those familiar with ITK to understand and use this code.

Suggestions for future work:
Two questions:

1) The program makes heavy use of mutex (at the pixel level).  We have found that even when a mutex collision doesn't occur, a mutex check can be time consuming.   Perhaps instead of threading by fiber track, it would be possible to thread by image region (an intelligent allocation of image regions to threads could be done based on prior knowledge).  In that manner, each thread will write to a unique section of the probability images - collisions will not occur and mutex locks will not be needed.  When tracts pass from one thread region to another it could be added to boundary queue whose read/write access could be managed by design or by mutex (infrequently called).

2) You state that using more than one thread on a single-processor (assume single core) CPU could offer some advantages.  This is very processor architecture specific - we have found that certain architectures (e.g., the sun architecture) will slowdown significantly as more threads are allocated to a single processor.

Requests for additional information from authors:
What is the copyright and distribution license of the code? 

How do I run the test data?