Summary:
The author describe some methods to improve the performance of the connected component filter, including a new algorithm, and show an important increase of performance.

Evidence:
The author compare the output of the current itk's filter and his modified ones. However he doesn't show the result images, thing that shouold be done to validate his program of comparison of labeled images.

Open Science:
The autho provide a good description of his work on current filter, and of the new algorithm, and even give some comment on optimization methods. This is open science.

Reproducibility:
Several test are failing, both on my 32 and 64 bits linux box. It should be fixed before including the result in ITK

Open Source Contributions:
The code is usable without problem

Code Quality:
Code have very good quality

Applicability to other problems:
Again a filter which can be used as the base of lots of image analysis methods

Suggestions for future work:
I think the algorithm can be threaded, and thus, give another performance improvement.
Also, I think you should use an internal type which support a higher range of values (for example NumericTraits< PixelType >::AccumulateType), so the labeling can succeed even if the number of values is near the maximum value of the type. This addition would require to change the labels (with a new equivalency table ?) and would also avoid the problem of relabeling after the labeling process only to have labels set like 0, 1 and 18. The cost of shouldn't be too high.

Additional Comments:
the "only" 4 starts are only for the broken tests; it's a very good contribution.
I will send you the error messages.

Summary:
The paper describes two connected component algorithms and analyzes their improved performance compared to the connected component filter in the InsightToolkit.
In a three-step process, every connected component in a binary image receives a different label. The pixel intensity inside every component is replaced with the value of its label.
This operation is particularly useful as a pre-processing step to extract the largest component in an image or to measure the size of anatomical structures.

Evidence:
The author describes his experiments in details and gives useful tips for researchers interested in optimizing their algorithms.
The code and data for the performance analysis are provided. However a pair of example input/output images would help to check if the executable gives the expected result.

Open Science:
Fully Open Science. Almost Open Data: the input images are provided, not the output images.
I understand that this was not intentional but the author did not see the need for providing the output images. Those images could be easily added as a revision.
I really appreciate that the author comments his experiment process and mentions other directions.

Reproducibility:
I compiled the code with MS VS8 and run it. The process is seamless.
I could not compare the output image I got with what the code is expected to return (see above comment).

Open Source:
100% ITK. Excellent code quality.

Requests for additional information from authors:
I would like to see a pair of input/output images in the paper and the archive, along with the list of arguments used.

Additional Comments:
What would be impact on the performance to collapse the labels in a way that all labels from 0 to number_of_components are used?
I tried the code on an image with 2 components on a black background, and the result image had labels 0, 1 and 18.
I would like to avoid having a second filter to re-label 0, 1 and 18 to 0, 1 and 2.