Reviewer Checklist For Technical Release

Technical  Release criteria

  • Is the language of sufficient quality?

Although the editorial team may also assess the quality of the written English, please do comment if you consider the standard is below that expected for a scientific publication.

  • Is there a clear statement of need explaining what problems the software is designed to solve and who the target audience is?

The authors should clearly state what problems the software is designed to solve and who the target audience is.

  • Is the source code available, and has an appropriate Open Source Initiative license been assigned to the code?

We require all source code to be openly available for reproducibility and reuse. Did the authors indicate where the software tools and relevant source code are available, under an appropriate  Open Source Initiative compliant license? Test data and software also needs to be credited according to the Data Citation principles and the recommendations of the FORCE11 Software Citation Implementation Working Group

  • As Open Source Software are there guidelines on how to contribute, report issues or seek support on the code? 

There must be clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

  • Is the code executable?

Can this tool be easily run on a computer? If not, please comment on what problems were encountered.  

  • Is the documentation provided clear and user friendly?

There must be enough clear information in the documentation to run this tool and information on where to seek help if required.

  • Is installation/deployment sufficiently outlined in the paper and documentation, and does it proceed as outlined?

Please comment on any problems with installation/ deployment. Should this be handled with an automated package management solution?

  • Is there a clearly-stated list of dependencies, and is the core functionality of the software documented to a satisfactory level?

A clear list of dependencies should be included. Are they using readthedocs or do they have a well documented code repository? Please comment if more is required.

  • Have any claims of performance been sufficiently tested and compared to other commonly-used packages? 

If there are performance claims and are insufficiently tested, please comment. If there are no performance claims, please comment “not applicable”.

  • Are there (ideally real world) examples demonstrating use of the software?

  • Is automated testing used or are there manual steps described so that the functionality of the software can be verified?

Automated testing is listed in the bioinformatics best practice checklist. Have the authors used a test framework to check the functionality of their software? If there is no automated testing, do the authors provide example data for testing their software and define what the expected results are?