PONE-D-21-04191

Meta-Analysis of Three Studies Measuring Fluorescence from Engineered Bacterial Genetic Constructs

PLOS ONE

Dear Dr. Beal,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by May 03 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Pasquale Avino, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  Thank you for stating the following in the Financial Disclosure section: "Partial support for this work was provided by NSF Expeditions in Computing Program 469 Award #1522074 as part of the Living Computing Project. Funder URL: https://www.nsf.gov/

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

And competing Interest: "The authors of this manuscript have read the journal’s policy and have the following competing interests: The authors received no specific commercial funding for this work. The following authors are employed by for-profit companies: Jacob Beal is employed by Raytheon BBN Technologies; Markus Gershater and Vishal Sanchania are employed by Synthace, and their work on this paper was thus indirectly supported by their salaries. This does not alter the authors’ adherence to PLOS ONE policies on sharing data and materials."

We note that one or more of the authors are employed by a commercial company: Raytheon BBN Technologies and Synthace.

a) Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

b)  Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc. 

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

3.  Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

4.  Please consider amending the title to more accurately reflect the nature of the work. We note that 'meta-analysis' typically refers to an analysis following a systematic review and utilising a specific framework.

5.   Please note that authors are responsible for ensuring that anyone named in the Acknowledgments agrees to be named (https://journals.plos.org/plosone/s/submission-guidelines#loc-acknowledgments).

Reviewers' comments:

Reviewer #1: In this manuscript, Beal et al conduct a metanalysis of fluorescence from bacterial strains. The data were generated by participants in the 2016, 2017, and 2018 IGEM competitions. The analysis conducted by the authors reveals high reproducibility in fluorescence across laboratories and years but also uncovers a large anomaly in fluorescence measurements from 2016. The authors conduct further analysis and suggest that the anomaly stems from errors in preparation of the fluorescent standard. The authors conduct an experiment that supports this

hypothesis. The authors conclude the article by providing recommendations for investigators working in synthetic biology. Overall, this is a well-written article, and the conclusions are generally supported by the data. By dealing with the often-overlooked issue of experimental reproducibility, the study represents an important contribution to the field of synthetic biology and should be accepted with only minor revisions.

We request that the authors consider addressing the following points:

Line 12: metrological traceability – unclear term – what does this mean to the reader not familiar with the field of instrument calibration? It would be useful to define this term here.

Line 98 – do you have a citation or reference for the statement that DH5-alpha and TOP10 are fairly similar? How do you define “fairly similar”?

Line 105 – do the authors have a reference for the statement that FITC and fluorescein are near-identical compounds with near-identical spectra.

Line 185 – “analysis of the 2018 study calibrant data finds this value to be 1.33e8” … how did the authors come up with this number?

Line 246-248 – can the authors speculate if there are particular factors that may contribute to the remaining variation not explained by issues in fluorescence calibration?

Reviewer #2: The manuscript by Beal and colleagues describes a meta-analysis of calibrated flow cytometry and plate reader data from the iGEM inter-lab study across 2016, 2017 and 2018. The main results are that calibration of both data types provides reproducibility across laboratories and machines. This message is very important for the synthetic biology field if it is really going to mature into a true engineering discipline. Another point from the paper is that errors in the calibration protocol can be disastrous for metrology and it is interesting to see the possible impact.

Overall, I think the authors have done a thorough job in collating the data across years and the message of the paper should (hopefully) have a big impact in the field. I have one minor comment:

When talking about the different protocols across years, the authors say the results across years are expected to be equivalent, and thus directly comparable. While this may be true for the expected values, it is not obvious to me that the noise distributions should be the same and changing concentrations etc will affect the variability. Can the authors add a comment on this?

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Comparative Analysis of Three Studies Measuring Fluorescence from Engineered Bacterial Genetic Constructs

PONE-D-21-04191R1

Dear Dr. Beal,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Pasquale Avino, Ph.D.

Academic Editor

PLOS ONE