Initial commit

skills/peer-review/SKILL.md

@@ -0,0 +1,565 @@
+---
+name: peer-review
+description: "Systematic peer review toolkit. Evaluate methodology, statistics, design, reproducibility, ethics, figure integrity, reporting standards, for manuscript and grant review across disciplines."
+allowed-tools: [Read, Write, Edit, Bash]
+---
+
+# Scientific Critical Evaluation and Peer Review
+
+## Overview
+
+Peer review is a systematic process for evaluating scientific manuscripts. Assess methodology, statistics, design, reproducibility, ethics, and reporting standards. Apply this skill for manuscript and grant review across disciplines with constructive, rigorous evaluation.
+
+## When to Use This Skill
+
+This skill should be used when:
+- Conducting peer review of scientific manuscripts for journals
+- Evaluating grant proposals and research applications
+- Assessing methodology and experimental design rigor
+- Reviewing statistical analyses and reporting standards
+- Evaluating reproducibility and data availability
+- Checking compliance with reporting guidelines (CONSORT, STROBE, PRISMA)
+- Providing constructive feedback on scientific writing
+
+## Visual Enhancement with Scientific Schematics
+
+**When creating documents with this skill, always consider adding scientific diagrams and schematics to enhance visual communication.**
+
+If your document does not already contain schematics or diagrams:
+- Use the **scientific-schematics** skill to generate AI-powered publication-quality diagrams
+- Simply describe your desired diagram in natural language
+- Nano Banana Pro will automatically generate, review, and refine the schematic
+
+**For new documents:** Scientific schematics should be generated by default to visually represent key concepts, workflows, architectures, or relationships described in the text.
+
+**How to generate schematics:**
+```bash
+python scripts/generate_schematic.py "your diagram description" -o figures/output.png
+```
+
+The AI will automatically:
+- Create publication-quality images with proper formatting
+- Review and refine through multiple iterations
+- Ensure accessibility (colorblind-friendly, high contrast)
+- Save outputs in the figures/ directory
+
+**When to add schematics:**
+- Peer review workflow diagrams
+- Evaluation criteria decision trees
+- Review process flowcharts
+- Methodology assessment frameworks
+- Quality assessment visualizations
+- Reporting guidelines compliance diagrams
+- Any complex concept that benefits from visualization
+
+For detailed guidance on creating schematics, refer to the scientific-schematics skill documentation.
+
+---
+
+## Peer Review Workflow
+
+Conduct peer review systematically through the following stages, adapting depth and focus based on the manuscript type and discipline.
+
+### Stage 1: Initial Assessment
+
+Begin with a high-level evaluation to determine the manuscript's scope, novelty, and overall quality.
+
+**Key Questions:**
+- What is the central research question or hypothesis?
+- What are the main findings and conclusions?
+- Is the work scientifically sound and significant?
+- Is the work appropriate for the intended venue?
+- Are there any immediate major flaws that would preclude publication?
+
+**Output:** Brief summary (2-3 sentences) capturing the manuscript's essence and initial impression.
+
+### Stage 2: Detailed Section-by-Section Review
+
+Conduct a thorough evaluation of each manuscript section, documenting specific concerns and strengths.
+
+#### Abstract and Title
+- **Accuracy:** Does the abstract accurately reflect the study's content and conclusions?
+- **Clarity:** Is the title specific, accurate, and informative?
+- **Completeness:** Are key findings and methods summarized appropriately?
+- **Accessibility:** Is the abstract comprehensible to a broad scientific audience?
+
+#### Introduction
+- **Context:** Is the background information adequate and current?
+- **Rationale:** Is the research question clearly motivated and justified?
+- **Novelty:** Is the work's originality and significance clearly articulated?
+- **Literature:** Are relevant prior studies appropriately cited?
+- **Objectives:** Are research aims/hypotheses clearly stated?
+
+#### Methods
+- **Reproducibility:** Can another researcher replicate the study from the description provided?
+- **Rigor:** Are the methods appropriate for addressing the research questions?
+- **Detail:** Are protocols, reagents, equipment, and parameters sufficiently described?
+- **Ethics:** Are ethical approvals, consent, and data handling properly documented?
+- **Statistics:** Are statistical methods appropriate, clearly described, and justified?
+- **Validation:** Are controls, replicates, and validation approaches adequate?
+
+**Critical elements to verify:**
+- Sample sizes and power calculations
+- Randomization and blinding procedures
+- Inclusion/exclusion criteria
+- Data collection protocols
+- Computational methods and software versions
+- Statistical tests and correction for multiple comparisons
+
+#### Results
+- **Presentation:** Are results presented logically and clearly?
+- **Figures/Tables:** Are visualizations appropriate, clear, and properly labeled?
+- **Statistics:** Are statistical results properly reported (effect sizes, confidence intervals, p-values)?
+- **Objectivity:** Are results presented without over-interpretation?
+- **Completeness:** Are all relevant results included, including negative results?
+- **Reproducibility:** Are raw data or summary statistics provided?
+
+**Common issues to identify:**
+- Selective reporting of results
+- Inappropriate statistical tests
+- Missing error bars or measures of variability
+- Over-fitting or circular analysis
+- Batch effects or confounding variables
+- Missing controls or validation experiments
+
+#### Discussion
+- **Interpretation:** Are conclusions supported by the data?
+- **Limitations:** Are study limitations acknowledged and discussed?
+- **Context:** Are findings placed appropriately within existing literature?
+- **Speculation:** Is speculation clearly distinguished from data-supported conclusions?
+- **Significance:** Are implications and importance clearly articulated?
+- **Future directions:** Are next steps or unanswered questions discussed?
+
+**Red flags:**
+- Overstated conclusions
+- Ignoring contradictory evidence
+- Causal claims from correlational data
+- Inadequate discussion of limitations
+- Mechanistic claims without mechanistic evidence
+
+#### References
+- **Completeness:** Are key relevant papers cited?
+- **Currency:** Are recent important studies included?
+- **Balance:** Are contrary viewpoints appropriately cited?
+- **Accuracy:** Are citations accurate and appropriate?
+- **Self-citation:** Is there excessive or inappropriate self-citation?
+
+### Stage 3: Methodological and Statistical Rigor
+
+Evaluate the technical quality and rigor of the research with particular attention to common pitfalls.
+
+**Statistical Assessment:**
+- Are statistical assumptions met (normality, independence, homoscedasticity)?
+- Are effect sizes reported alongside p-values?
+- Is multiple testing correction applied appropriately?
+- Are confidence intervals provided?
+- Is sample size justified with power analysis?
+- Are parametric vs. non-parametric tests chosen appropriately?
+- Are missing data handled properly?
+- Are exploratory vs. confirmatory analyses distinguished?
+
+**Experimental Design:**
+- Are controls appropriate and adequate?
+- Is replication sufficient (biological and technical)?
+- Are potential confounders identified and controlled?
+- Is randomization properly implemented?
+- Are blinding procedures adequate?
+- Is the experimental design optimal for the research question?
+
+**Computational/Bioinformatics:**
+- Are computational methods clearly described and justified?
+- Are software versions and parameters documented?
+- Is code made available for reproducibility?
+- Are algorithms and models validated appropriately?
+- Are assumptions of computational methods met?
+- Is batch correction applied appropriately?
+
+### Stage 4: Reproducibility and Transparency
+
+Assess whether the research meets modern standards for reproducibility and open science.
+
+**Data Availability:**
+- Are raw data deposited in appropriate repositories?
+- Are accession numbers provided for public databases?
+- Are data sharing restrictions justified (e.g., patient privacy)?
+- Are data formats standard and accessible?
+
+**Code and Materials:**
+- Is analysis code made available (GitHub, Zenodo, etc.)?
+- Are unique materials available or described sufficiently for recreation?
+- Are protocols detailed in sufficient depth?
+
+**Reporting Standards:**
+- Does the manuscript follow discipline-specific reporting guidelines (CONSORT, PRISMA, ARRIVE, MIAME, MINSEQE, etc.)?
+- See `references/reporting_standards.md` for common guidelines
+- Are all elements of the appropriate checklist addressed?
+
+### Stage 5: Figure and Data Presentation
+
+Evaluate the quality, clarity, and integrity of data visualization.
+
+**Quality Checks:**
+- Are figures high resolution and clearly labeled?
+- Are axes properly labeled with units?
+- Are error bars defined (SD, SEM, CI)?
+- Are statistical significance indicators explained?
+- Are color schemes appropriate and accessible (colorblind-friendly)?
+- Are scale bars included for images?
+- Is data visualization appropriate for the data type?
+
+**Integrity Checks:**
+- Are there signs of image manipulation (duplications, splicing)?
+- Are Western blots and gels appropriately presented?
+- Are representative images truly representative?
+- Are all conditions shown (no selective presentation)?
+
+**Clarity:**
+- Can figures stand alone with their legends?
+- Is the message of each figure immediately clear?
+- Are there redundant figures or panels?
+- Would data be better presented as tables or figures?
+
+### Stage 6: Ethical Considerations
+
+Verify that the research meets ethical standards and guidelines.
+
+**Human Subjects:**
+- Is IRB/ethics approval documented?
+- Is informed consent described?
+- Are vulnerable populations appropriately protected?
+- Is patient privacy adequately protected?
+- Are potential conflicts of interest disclosed?
+
+**Animal Research:**
+- Is IACUC or equivalent approval documented?
+- Are procedures humane and justified?
+- Are the 3Rs (replacement, reduction, refinement) considered?
+- Are euthanasia methods appropriate?
+
+**Research Integrity:**
+- Are there concerns about data fabrication or falsification?
+- Is authorship appropriate and justified?
+- Are competing interests disclosed?
+- Is funding source disclosed?
+- Are there concerns about plagiarism or duplicate publication?
+
+### Stage 7: Writing Quality and Clarity
+
+Assess the manuscript's clarity, organization, and accessibility.
+
+**Structure and Organization:**
+- Is the manuscript logically organized?
+- Do sections flow coherently?
+- Are transitions between ideas clear?
+- Is the narrative compelling and clear?
+
+**Writing Quality:**
+- Is the language clear, precise, and concise?
+- Are jargon and acronyms minimized and defined?
+- Is grammar and spelling correct?
+- Are sentences unnecessarily complex?
+- Is the passive voice overused?
+
+**Accessibility:**
+- Can a non-specialist understand the main findings?
+- Are technical terms explained?
+- Is the significance clear to a broad audience?
+
+## Structuring Peer Review Reports
+
+Organize feedback in a hierarchical structure that prioritizes issues and provides actionable guidance.
+
+### Summary Statement
+
+Provide a concise overall assessment (1-2 paragraphs):
+- Brief synopsis of the research
+- Overall recommendation (accept, minor revisions, major revisions, reject)
+- Key strengths (2-3 bullet points)
+- Key weaknesses (2-3 bullet points)
+- Bottom-line assessment of significance and soundness
+
+### Major Comments
+
+List critical issues that significantly impact the manuscript's validity, interpretability, or significance. Number these sequentially for easy reference.
+
+**Major comments typically include:**
+- Fundamental methodological flaws
+- Inappropriate statistical analyses
+- Unsupported or overstated conclusions
+- Missing critical controls or experiments
+- Serious reproducibility concerns
+- Major gaps in literature coverage
+- Ethical concerns
+
+**For each major comment:**
+1. Clearly state the issue
+2. Explain why it's problematic
+3. Suggest specific solutions or additional experiments
+4. Indicate if addressing it is essential for publication
+
+### Minor Comments
+
+List less critical issues that would improve clarity, completeness, or presentation. Number these sequentially.
+
+**Minor comments typically include:**
+- Unclear figure labels or legends
+- Missing methodological details
+- Typographical or grammatical errors
+- Suggestions for improved data presentation
+- Minor statistical reporting issues
+- Supplementary analyses that would strengthen conclusions
+- Requests for clarification
+
+**For each minor comment:**
+1. Identify the specific location (section, paragraph, figure)
+2. State the issue clearly
+3. Suggest how to address it
+
+### Specific Line-by-Line Comments (Optional)
+
+For manuscripts requiring detailed feedback, provide section-specific or line-by-line comments:
+- Reference specific page/line numbers or sections
+- Note factual errors, unclear statements, or missing citations
+- Suggest specific edits for clarity
+
+### Questions for Authors
+
+List specific questions that need clarification:
+- Methodological details that are unclear
+- Seemingly contradictory results
+- Missing information needed to evaluate the work
+- Requests for additional data or analyses
+
+## Tone and Approach
+
+Maintain a constructive, professional, and collegial tone throughout the review.
+
+**Best Practices:**
+- **Be constructive:** Frame criticism as opportunities for improvement
+- **Be specific:** Provide concrete examples and actionable suggestions
+- **Be balanced:** Acknowledge strengths as well as weaknesses
+- **Be respectful:** Remember that authors have invested significant effort
+- **Be objective:** Focus on the science, not the scientists
+- **Be thorough:** Don't overlook issues, but prioritize appropriately
+- **Be clear:** Avoid ambiguous or vague criticism
+
+**Avoid:**
+- Personal attacks or dismissive language
+- Sarcasm or condescension
+- Vague criticism without specific examples
+- Requesting unnecessary experiments beyond the scope
+- Demanding adherence to personal preferences vs. best practices
+- Revealing your identity if reviewing is double-blind
+
+## Special Considerations by Manuscript Type
+
+### Original Research Articles
+- Emphasize rigor, reproducibility, and novelty
+- Assess significance and impact
+- Verify that conclusions are data-driven
+- Check for complete methods and appropriate controls
+
+### Reviews and Meta-Analyses
+- Evaluate comprehensiveness of literature coverage
+- Assess search strategy and inclusion/exclusion criteria
+- Verify systematic approach and lack of bias
+- Check for critical analysis vs. mere summarization
+- For meta-analyses, evaluate statistical approach and heterogeneity
+
+### Methods Papers
+- Emphasize validation and comparison to existing methods
+- Assess reproducibility and availability of protocols/code
+- Evaluate improvements over existing approaches
+- Check for sufficient detail for implementation
+
+### Short Reports/Letters
+- Adapt expectations for brevity
+- Ensure core findings are still rigorous and significant
+- Verify that format is appropriate for findings
+
+### Preprints
+- Recognize that these have not undergone formal peer review
+- May be less polished than journal submissions
+- Still apply rigorous standards for scientific validity
+- Consider providing constructive feedback to help authors improve before journal submission
+
+### Presentations and Slide Decks
+
+**⚠️ CRITICAL: For presentations, NEVER read the PDF directly. ALWAYS convert to images first.**
+
+When reviewing scientific presentations (PowerPoint, Beamer, slide decks):
+
+#### Mandatory Image-Based Review Workflow
+
+**NEVER attempt to read presentation PDFs directly** - this causes buffer overflow errors and doesn't show visual formatting issues.
+
+**Required Process:**
+1. Convert PDF to images using pdftoppm:
+   ```bash
+   pdftoppm -jpeg -r 150 presentation.pdf review/slide
+   # Creates: review/slide-1.jpg, review/slide-2.jpg, etc.
+   ```
+2. Read and inspect EACH slide image file sequentially
+3. Document issues with specific slide numbers
+4. Provide feedback on visual formatting and content
+
+**Print when starting review:**
+```
+[HH:MM:SS] PEER REVIEW: Presentation detected - converting to images for review
+[HH:MM:SS] PDF REVIEW: NEVER reading PDF directly - using image-based inspection
+```
+
+#### Presentation-Specific Evaluation Criteria
+
+**Visual Design and Readability:**
+- [ ] Text is large enough (minimum 18pt, ideally 24pt+ for body text)
+- [ ] High contrast between text and background (4.5:1 minimum, 7:1 preferred)
+- [ ] Color scheme is professional and colorblind-accessible
+- [ ] Consistent visual design across all slides
+- [ ] White space is adequate (not cramped)
+- [ ] Fonts are clear and professional
+
+**Layout and Formatting (Check EVERY Slide Image):**
+- [ ] No text overflow or truncation at slide edges
+- [ ] No element overlaps (text over images, overlapping shapes)
+- [ ] Titles are consistently positioned
+- [ ] Content is properly aligned
+- [ ] Bullets and text are not cut off
+- [ ] Figures fit within slide boundaries
+- [ ] Captions and labels are visible and readable
+
+**Content Quality:**
+- [ ] One main idea per slide (not overloaded)
+- [ ] Minimal text (3-6 bullets per slide maximum)
+- [ ] Bullet points are concise (5-7 words each)
+- [ ] Figures are simplified and clear (not copy-pasted from papers)
+- [ ] Data visualizations have large, readable labels
+- [ ] Citations are present and properly formatted
+- [ ] Results/data slides dominate the presentation (40-50% of content)
+
+**Structure and Flow:**
+- [ ] Clear narrative arc (introduction → methods → results → discussion)
+- [ ] Logical progression between slides
+- [ ] Slide count appropriate for talk duration (~1 slide per minute)
+- [ ] Title slide includes authors, affiliation, date
+- [ ] Introduction cites relevant background literature (3-5 papers)
+- [ ] Discussion cites comparison papers (3-5 papers)
+- [ ] Conclusions slide summarizes key findings
+- [ ] Acknowledgments/funding slide at end
+
+**Scientific Content:**
+- [ ] Research question clearly stated
+- [ ] Methods adequately summarized (not excessive detail)
+- [ ] Results presented logically with clear visualizations
+- [ ] Statistical significance indicated appropriately
+- [ ] Conclusions supported by data shown
+- [ ] Limitations acknowledged where appropriate
+- [ ] Future directions or broader impact discussed
+
+**Common Presentation Issues to Flag:**
+
+**Critical Issues (Must Fix):**
+- Text overflow making content unreadable
+- Font sizes too small (<18pt)
+- Element overlaps obscuring data
+- Insufficient contrast (text hard to read)
+- Figures too complex or illegible
+- No citations (completely unsupported claims)
+- Slide count drastically mismatched to duration
+
+**Major Issues (Should Fix):**
+- Inconsistent design across slides
+- Too much text (walls of text, not bullets)
+- Poorly simplified figures (axis labels too small)
+- Cramped layout with insufficient white space
+- Missing key structural elements (no conclusion slide)
+- Poor color choices (not colorblind-safe)
+- Minimal results content (<30% of slides)
+
+**Minor Issues (Suggestions for Improvement):**
+- Could use more visuals/diagrams
+- Some slides slightly text-heavy
+- Minor alignment inconsistencies
+- Could benefit from more white space
+- Additional citations would strengthen claims
+- Color scheme could be more modern
+
+#### Review Report Format for Presentations
+
+**Summary Statement:**
+- Overall impression of presentation quality
+- Appropriateness for target audience and duration
+- Key strengths (visual design, content, clarity)
+- Key weaknesses (formatting issues, content gaps)
+- Recommendation (ready to present, minor revisions, major revisions)
+
+**Layout and Formatting Issues (By Slide Number):**
+```
+Slide 3: Text overflow - bullet point 4 extends beyond right margin
+Slide 7: Element overlap - figure overlaps with caption text
+Slide 12: Font size - axis labels too small to read from distance
+Slide 18: Alignment - title not centered
+```
+
+**Content and Structure Feedback:**
+- Adequacy of background context and citations
+- Clarity of research question and objectives
+- Quality of methods summary
+- Effectiveness of results presentation
+- Strength of conclusions and implications
+
+**Design and Accessibility:**
+- Overall visual appeal and professionalism
+- Color contrast and readability
+- Colorblind accessibility
+- Consistency across slides
+
+**Timing and Scope:**
+- Whether slide count matches intended duration
+- Appropriate level of detail for talk type
+- Balance between sections
+
+#### Example Image-Based Review Process
+
+```
+[14:30:00] PEER REVIEW: Starting review of presentation
+[14:30:05] PEER REVIEW: Presentation detected - converting to images
+[14:30:10] PDF REVIEW: Running pdftoppm on presentation.pdf
+[14:30:15] PDF REVIEW: Converted 25 slides to images in review/ directory
+[14:30:20] PDF REVIEW: Inspecting slide 1/25 - title slide
+[14:30:25] PDF REVIEW: Inspecting slide 2/25 - introduction
+...
+[14:35:40] PDF REVIEW: Inspecting slide 25/25 - acknowledgments
+[14:35:45] PDF REVIEW: Completed image-based review
+[14:35:50] PEER REVIEW: Found 8 layout issues, 3 content issues
+[14:35:55] PEER REVIEW: Generating structured feedback by slide number
+```
+
+**Remember:** For presentations, the visual inspection via images is MANDATORY. Never attempt to read presentation PDFs as text - it will fail and miss all visual formatting issues.
+
+## Resources
+
+This skill includes reference materials to support comprehensive peer review:
+
+### references/reporting_standards.md
+Guidelines for major reporting standards across disciplines (CONSORT, PRISMA, ARRIVE, MIAME, STROBE, etc.) to evaluate completeness of methods and results reporting.
+
+### references/common_issues.md
+Catalog of frequent methodological and statistical issues encountered in peer review, with guidance on identifying and addressing them.
+
+## Final Checklist
+
+Before finalizing the review, verify:
+
+- [ ] Summary statement clearly conveys overall assessment
+- [ ] Major concerns are clearly identified and justified
+- [ ] Suggested revisions are specific and actionable
+- [ ] Minor issues are noted but properly categorized
+- [ ] Statistical methods have been evaluated
+- [ ] Reproducibility and data availability assessed
+- [ ] Ethical considerations verified
+- [ ] Figures and tables evaluated for quality and integrity
+- [ ] Writing quality assessed
+- [ ] Tone is constructive and professional throughout
+- [ ] Review is thorough but proportionate to manuscript scope
+- [ ] Recommendation is consistent with identified issues