zhongwei/gh-cskiro-claudex-testing-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Zhongwei Li
2025-11-29 18:17:04 +08:00
commit e758c0ab84
56 changed files with 9997 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

414
skills/playwright-e2e-automation/data/accessibility-checks.md Normal file
View File

@@ -0,0 +1,414 @@
# Accessibility Checks for Visual Analysis
WCAG 2.1 AA compliance criteria for LLM-powered screenshot analysis.
## Overview
When analyzing screenshots, check for these accessibility violations. This guide follows WCAG 2.1 Level AA standards.
## 1. Color Contrast
### Minimum Contrast Ratios
**Text:**
- Normal text (< 18pt or < 14pt bold): **4.5:1**
- Large text (≥ 18pt or ≥ 14pt bold): **3:1**
**UI Components:**
- Form inputs, buttons, icons: **3:1** against background
### Common Violations in Screenshots
```
❌ Light gray text on white background (2:1 ratio)
✅ Dark gray #595959 on white #FFFFFF (7:1 ratio)
❌ Blue link #4A90E2 on light blue #E8F4FF (1.8:1 ratio)
✅ Blue link #0066CC on white #FFFFFF (8.2:1 ratio)
❌ Gray placeholder text #CCCCCC on white (1.6:1 ratio)
✅ Gray placeholder text #757575 on white (4.6:1 ratio)
```
### Visual Indicators
When analyzing screenshots, look for:
- Pale or faded text that's hard to read
- Low-contrast buttons that don't stand out
- Links that blend into surrounding text
- Disabled states that are barely distinguishable
## 2. Text Size and Readability
### Minimum Font Sizes
- Body text: **16px** minimum (1rem)
- Small text acceptable: **14px** for secondary content
- Avoid: Text smaller than **12px** (fails WCAG)
### Common Violations
```
❌ Body text at 12px - too small for many users
✅ Body text at 16px or larger
❌ Mobile text at 10px - illegible on small screens
✅ Mobile text at 14px minimum
❌ Long paragraphs with no line height spacing
✅ Line height 1.5x for body text (e.g., 16px text with 24px line height)
```
### Visual Indicators
- Text that appears squished or compressed
- Long lines of text with minimal spacing
- Tiny labels on buttons or form fields
## 3. Focus Indicators
### Requirements
All interactive elements must have **visible focus indicators**:
- Minimum **2px** outline or border
- Contrast ratio of **3:1** against background
- Clearly visible when tabbing through interface
### Common Violations
```
❌ No visible outline when button is focused
✅ Blue 2px outline appears on focus
❌ Focus outline same color as background (invisible)
✅ High-contrast focus outline (e.g., black on white)
❌ Focus state only indicated by subtle background color change
✅ Focus state with outline + background color change
```
### Visual Indicators in Screenshots
Look for:
- Focused element (if screenshot captures tab state)
- Absence of visible outline or border
- Focus indicator that's too subtle or low-contrast
## 4. Form Labels and Instructions
### Requirements
- Every form input must have a visible **<label>** or aria-label
- Labels must be **adjacent** to their inputs
- Required fields must be clearly indicated
- Error messages must be **visible and associated** with inputs
### Common Violations
```
❌ Input with only placeholder text (disappears when typing)
✅ Input with persistent label above or beside it
❌ Label far away from input (hard to associate)
✅ Label immediately adjacent to input
❌ Required field marked only with color (red border)
✅ Required field marked with * and "Required" text
❌ Error message in different part of page
✅ Error message directly below input field
```
### Visual Indicators
- Inputs without visible labels
- Placeholder text used as labels (disappears on focus)
- Required fields indicated only by color
- Error states without clear error text
## 5. Heading Hierarchy
### Requirements
- Headings must follow logical order: **H1 → H2 → H3** (no skipping)
- Page should have exactly **one H1** (page title)
- Headings should be **visually distinct** from body text
### Common Violations
```
❌ Page with H1 → H4 (skips H2, H3)
✅ Page with H1 → H2 → H3
❌ Multiple H1 headings on same page
✅ Single H1 for page title, H2s for sections
❌ Heading text same size as body text
✅ Headings progressively larger: H3 < H2 < H1
```
### Visual Indicators in Screenshots
- Headings that don't look like headings (same size as body)
- Missing visual hierarchy (all headings same size)
- Text that looks like headings but isn't (bold body text)
## 6. Alternative Text for Images
### Requirements
- Decorative images: Empty alt="" or aria-hidden="true"
- Informative images: Descriptive alt text
- Complex images (charts, graphs): Detailed description
### Common Violations
**Note:** Can't always detect from screenshots alone, but can identify likely issues:
```
❌ Icon buttons with no visible text label (likely missing aria-label)
✅ Icon buttons with visible text label or tooltip
❌ Charts/graphs with no accompanying data table or description
✅ Charts with descriptive caption or linked data table
❌ Images that convey important info but might lack alt text
✅ Important info also available in visible text
```
### Visual Indicators
- Icon-only buttons without text labels
- Charts/infographics without textual explanations
- Images that appear to contain important information
## 7. Keyboard Navigation
### Requirements
- All interactive elements accessible via keyboard
- Logical tab order (top to bottom, left to right)
- No keyboard traps
- Skip links for navigation
### Visual Analysis Cues
**Can identify potential issues from screenshots:**
```
❌ Custom dropdown without visible keyboard focus states
✅ Standard HTML select or custom with clear focus indicators
❌ Modal dialog with no visible close button (might trap keyboard)
✅ Modal with visible, accessible close button
❌ Navigation menu requiring hover (might be keyboard inaccessible)
✅ Navigation menu that works on click/enter
```
## 8. Touch Target Size
### Minimum Sizes (Mobile)
- Interactive elements: **44x44 CSS pixels** minimum
- Adequate spacing between targets: **8px** minimum
### Common Violations
```
❌ Mobile buttons at 32x32px (too small)
✅ Mobile buttons at 48x48px
❌ Links in mobile menu spaced 4px apart (accidental taps)
✅ Links spaced 12px apart
❌ Checkbox at 16x16px on mobile (hard to tap)
✅ Checkbox with expanded tap area 44x44px
```
### Visual Indicators in Mobile Screenshots
- Tiny buttons that would be hard to tap accurately
- Densely packed clickable elements
- Links or buttons too close together
## 9. Responsive Design
### Requirements
- Content readable without horizontal scrolling
- No text truncation
- Proper scaling on different viewports
- No overlapping content
### Common Violations
```
❌ Desktop layout on mobile with horizontal scroll
✅ Mobile-optimized layout with no horizontal scroll
❌ Text cut off at viewport edge
✅ Text wraps properly within viewport
❌ Fixed-width elements overflow on small screens
✅ Flexible/responsive elements scale to screen size
❌ Overlapping elements on mobile (buttons on top of text)
✅ Elements stack vertically with proper spacing
```
### Visual Indicators Across Viewports
When comparing desktop/tablet/mobile screenshots:
- Text that gets cut off on smaller screens
- Overlapping or compressed elements
- Horizontal scrollbars on mobile
- Unreadable small text on mobile
## 10. Color Not Sole Indicator
### Requirements
- Information must not rely on **color alone**
- Use patterns, icons, or text in addition to color
### Common Violations
```
❌ Required fields indicated only by red border
✅ Required fields with red border + "*" icon + "Required" text
❌ Success/error only shown by green/red color
✅ Success/error shown by color + icon + text message
❌ Chart legend with only colored boxes
✅ Chart legend with colored boxes + patterns + labels
❌ Form validation using only red/green highlighting
✅ Form validation with color + icons + error text
```
### Visual Indicators
- Status indicators using only color
- Charts relying solely on color to differentiate data
- Form states indicated only by color changes
- Links distinguished only by color (not underline)
## Visual Analysis Workflow
When analyzing a screenshot for accessibility:
### Step 1: Text and Contrast
1. Check all text for sufficient contrast (4.5:1 for body, 3:1 for large)
2. Verify text is large enough (16px minimum)
3. Check line height and spacing for readability
### Step 2: Interactive Elements
1. Identify all buttons, links, form inputs
2. Verify they have sufficient size (44x44px on mobile)
3. Check for visible focus indicators (if focus state captured)
4. Ensure adequate spacing between targets
### Step 3: Form Elements
1. Check each input has visible label
2. Verify required fields clearly marked (not just color)
3. Look for error messages (should be near inputs)
### Step 4: Structure
1. Check heading hierarchy (visual size progression)
2. Verify logical content flow
3. Look for proper spacing and organization
### Step 5: Responsive Issues
1. Check for text truncation or cutoff
2. Look for overlapping elements
3. Verify no horizontal scroll
4. Ensure touch targets appropriate for viewport
### Step 6: Color Usage
1. Identify any color-only indicators
2. Verify status messages use icons or text too
3. Check charts/graphs have non-color differentiation
## Severity Levels
When reporting accessibility issues from screenshots:
### Critical (P0)
- Contrast ratio < 3:1 for any text
- Missing form labels
- Keyboard trap (if detectable)
- Content not accessible without horizontal scroll
### High (P1)
- Contrast ratio 3:1-4.4:1 for normal text
- Touch targets < 44x44px on mobile
- Heading hierarchy violations
- Color as sole indicator for critical info
### Medium (P2)
- Text size < 14px for body content
- Insufficient spacing between touch targets (< 8px)
- Inconsistent focus indicators
- Minor responsive issues
### Low (P3)
- Line height < 1.4 for long text blocks
- Decorative images possibly missing alt (can't confirm from screenshot)
- Minor visual hierarchy inconsistencies
## Example Analysis Output
```markdown
## Accessibility Issues Found
### Critical (1)
1. **Insufficient color contrast on form labels**
- Location: Contact form, all input labels
- Issue: Light gray #AAAAAA on white #FFFFFF (2.6:1 ratio)
- Requirement: 4.5:1 for normal text
- Fix: Use darker gray #595959 (7:1 ratio)
### High (2)
1. **Missing visible labels on inputs**
- Location: Email and password fields
- Issue: Only placeholder text, no persistent label
- Fix: Add visible <label> elements above inputs
2. **Touch targets too small on mobile**
- Location: Social media icons in footer
- Issue: Icons are 24x24px (below 44x44px minimum)
- Fix: Increase tap area to 44x44px with padding
### Medium (1)
1. **Body text too small**
- Location: Article content
- Issue: 14px font size (recommended 16px minimum)
- Fix: Increase base font size to 16px
```
## Tools for Automated Checking
While visual analysis is manual, recommend these tools for comprehensive checks:
```typescript
// Integrate axe-core in Playwright tests
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test('should not have accessibility violations', async ({ page }) => {
await page.goto('/');
const accessibilityScanResults = await new AxeBuilder({ page }).analyze();
expect(accessibilityScanResults.violations).toEqual([]);
});
```
---
**References:**
- [WCAG 2.1 Guidelines](https://www.w3.org/WAI/WCAG21/quickref/)
- [WebAIM Contrast Checker](https://webaim.org/resources/contrastchecker/)
- [Axe DevTools](https://www.deque.com/axe/devtools/)

557
skills/playwright-e2e-automation/data/common-ui-bugs.md Normal file
View File

@@ -0,0 +1,557 @@
# Common UI Bugs - Visual Analysis Guide
Patterns and indicators for identifying UI bugs from screenshots during LLM-powered visual analysis.
## Layout Issues
### 1. Overlapping Elements
**Visual Indicators:**
- Text overlapping other text
- Buttons overlapping images or other buttons
- Content extending beyond container boundaries
- Z-index issues causing incorrect stacking
**Examples:**
```
❌ Modal dialog overlapped by dropdown menu
❌ Footer content overlapping main content
❌ Notification banner covering navigation
❌ Search results hidden behind fixed header
```
**Screenshot Analysis:**
- Look for any elements that appear on top of others unexpectedly
- Check if all content is fully visible and not obscured
- Verify layering makes sense (modals on top, backgrounds behind)
### 2. Text Truncation / Overflow
**Visual Indicators:**
- Text cut off mid-word or mid-letter
- Ellipsis (...) in unexpected places
- Content extending outside visible area
- Horizontal scrollbars on text containers
**Examples:**
```
❌ Button text: "Continue to Chec..." (truncated on mobile)
❌ Table header: "Customer N..." instead of "Customer Name"
❌ Card title cut off at viewport edge
❌ Long email addresses broken into random positions
```
**Screenshot Analysis:**
- Check if all text is fully visible
- Look for truncation indicators (...)
- Verify important text isn't cut off
- Check if text wraps properly on smaller viewports
### 3. Broken Grid/Flexbox Layouts
**Visual Indicators:**
- Cards or items with inconsistent sizes
- Uneven spacing between elements
- Elements not aligned in columns/rows
- One element significantly larger/smaller than siblings
**Examples:**
```
❌ Product grid: 3 cards same height, 1 card twice as tall
❌ Navigation items: uneven spacing (10px, 20px, 15px)
❌ Form inputs: labels misaligned with inputs
❌ Cards: some with images, some without, causing height mismatch
```
**Screenshot Analysis:**
- Check if grid items are evenly sized
- Verify consistent spacing between elements
- Look for alignment issues in rows/columns
- Identify items breaking out of grid structure
### 4. Responsive Breakpoint Issues
**Visual Indicators (comparing viewport sizes):**
- Desktop layout on mobile (very small text, horizontal scroll)
- Mobile layout on desktop (everything too large, wasted space)
- Sudden jumps in layout between similar viewport sizes
- Media queries not triggering at expected breakpoints
**Examples:**
```
❌ Desktop: 3-column layout → Mobile: Still 3 columns (too cramped)
✅ Desktop: 3-column layout → Tablet: 2 columns → Mobile: 1 column
❌ Desktop: 16px text → Mobile: 16px text (too small on small screen)
✅ Desktop: 16px text → Mobile: 14px text with increased line height
❌ Fixed sidebar pushes main content off screen on tablet
✅ Sidebar collapses to hamburger menu on tablet
```
**Screenshot Analysis:**
- Compare same page across desktop (1280px+), tablet (768px), mobile (375px)
- Check if layout adapts appropriately at each size
- Verify no horizontal scrolling on mobile
- Ensure touch targets are 44x44px minimum on mobile
## Component-Specific Issues
### 5. Form Validation Problems
**Visual Indicators:**
- Error messages in wrong location (far from input)
- No visible error state (input looks normal despite error)
- Success state not clearly indicated
- Disabled buttons without indication why
**Examples:**
```
❌ Error message at top of page, input in middle (hard to associate)
✅ Error message directly below relevant input
❌ Invalid input: red border only (color-blind users miss it)
✅ Invalid input: red border + error icon + error text
❌ Submit button disabled, no explanation why
✅ Submit button disabled with tooltip "Complete required fields"
❌ Multiple validation errors shown as one generic message
✅ Each field shows its specific error message
```
**Screenshot Analysis:**
- Check if error states are clearly visible
- Verify error messages are near their inputs
- Look for validation indicators beyond just color
- Confirm required fields are clearly marked
### 6. Button States
**Visual Indicators:**
- Active/hover/focus states indistinguishable
- Disabled buttons look clickable
- Primary vs secondary buttons unclear
- Loading/submitting state not indicated
**Examples:**
```
❌ Disabled button: gray text on light gray (looks clickable)
✅ Disabled button: clear visual indication (opacity, cursor, label)
❌ Primary and secondary buttons identical appearance
✅ Primary: bold color, secondary: outline only
❌ Button clicked but no loading indicator (looks broken)
✅ Button shows spinner or "Loading..." text when clicked
❌ Hover state same as default state (no feedback)
✅ Hover state: darker background or subtle animation
```
**Screenshot Analysis:**
- Verify different button states are visually distinct
- Check if disabled buttons clearly look non-interactive
- Look for visual feedback on interactive states
- Ensure primary actions are visually prominent
### 7. Image Loading Issues
**Visual Indicators:**
- Broken image icons (usually a small icon or alt text)
- Missing images (blank space where image should be)
- Images with wrong aspect ratio (stretched/squashed)
- Low-resolution images appearing pixelated
**Examples:**
```
❌ Product image slot shows broken image icon
❌ Profile picture area: empty circle (image failed to load)
❌ Banner image: 16:9 image stretched to 1:1 (distorted)
❌ Thumbnail: tiny image scaled up 3x (pixelated)
```
**Screenshot Analysis:**
- Look for broken image indicators
- Check if all images loaded successfully
- Verify images maintain proper aspect ratios
- Identify pixelated or low-quality images
### 8. Table/List Issues
**Visual Indicators:**
- Headers not aligned with columns
- Inconsistent row heights
- Text overflow in cells
- Missing borders or inconsistent borders
- Poor mobile table handling (horizontal scroll)
**Examples:**
```
❌ Table headers offset from column data
❌ One row 2x height of others (content wrapping differently)
❌ Cell content: "john.doe@verylongemailaddr..." (truncated)
❌ Mobile table: requires horizontal scroll to see all columns
```
**Screenshot Analysis:**
- Verify column headers align with data
- Check for consistent row heights
- Look for text overflow in cells
- Ensure tables are readable on mobile (responsive design)
## Content Issues
### 9. Missing Content
**Visual Indicators:**
- Empty sections (just headers, no content)
- Placeholder text in production ("Lorem ipsum", "TBD", "Coming soon")
- Missing images or icons where expected
- Incomplete sentences or paragraphs
**Examples:**
```
❌ Section header "Latest Articles" with no articles below
❌ Product description: "Lorem ipsum dolor sit amet..."
❌ Icon placeholder: gray square instead of actual icon
❌ Bio section: ends mid-sentence
```
**Screenshot Analysis:**
- Identify any placeholder content
- Look for empty sections or containers
- Check if all expected content is present
- Verify no incomplete text
### 10. Inconsistent Spacing/Padding
**Visual Indicators:**
- Uneven margins between sections
- Inconsistent padding inside containers
- Elements touching edges (no breathing room)
- Random spacing that doesn't follow a system
**Examples:**
```
❌ Section 1: 40px margin → Section 2: 25px margin → Section 3: 35px
✅ All sections: consistent 40px margin
❌ Card padding: 16px top, 20px right, 14px bottom, 18px left
✅ Card padding: 16px all sides
❌ Button text touching button edge (no padding)
✅ Button: 12px vertical, 20px horizontal padding
```
**Screenshot Analysis:**
- Check for consistent spacing throughout page
- Verify elements have appropriate padding
- Look for crowded areas with insufficient spacing
- Identify spacing that breaks visual rhythm
## Typography Issues
### 11. Font Rendering Problems
**Visual Indicators:**
- Jagged or pixelated text
- Text weight too thin (hard to read)
- Inconsistent font families
- Line height too tight or too loose
**Examples:**
```
❌ Body text: font-weight 100 (barely visible)
✅ Body text: font-weight 400 (readable)
❌ Headings: Arial → Body: Times New Roman (inconsistent)
✅ All text: consistent font family
❌ Long paragraph: line-height 1.0 (text touching)
✅ Long paragraph: line-height 1.5 (readable spacing)
```
**Screenshot Analysis:**
- Check if text is crisp and readable
- Verify consistent font families
- Look for appropriate line height (1.4-1.6 for body text)
- Ensure font weights are accessible
### 12. Text Alignment Issues
**Visual Indicators:**
- Center-aligned paragraphs (hard to read)
- Inconsistent alignment within a section
- Right-aligned text in LTR layout without reason
- Justified text with large gaps
**Examples:**
```
❌ Long paragraph: center-aligned (hard to follow)
✅ Long paragraph: left-aligned
❌ Form: labels left-aligned, some center-aligned randomly
✅ Form: all labels consistently left-aligned
❌ Justified text: large gaps between words ("rivers")
✅ Left-aligned text with ragged right edge
```
**Screenshot Analysis:**
- Check if alignment aids readability
- Verify consistent alignment within sections
- Look for awkward gaps in justified text
- Ensure alignment makes sense for content type
## Interactive Element Issues
### 13. Hover/Focus States Missing
**Note:** Only detectable if screenshot captures focused/hovered state
**Visual Indicators:**
- Link looks identical to surrounding text (no underline, same color)
- Focused input indistinguishable from unfocused
- Hovered button shows no change
- Dropdown menu items don't highlight on hover
**Examples:**
```
❌ Link: blue text, no underline, same as normal text
✅ Link: blue text with underline, or different color
❌ Input focused: looks identical to unfocused state
✅ Input focused: blue border or outline appears
❌ Menu item hovered: no visual change
✅ Menu item hovered: background color change
```
**Screenshot Analysis (if interactive state captured):**
- Verify interactive elements show visual feedback
- Check if focused element has clear indicator
- Look for hover states that provide feedback
- Ensure keyboard focus is visible
### 14. Icon Issues
**Visual Indicators:**
- Icons misaligned with text
- Icons wrong size (too large or too small)
- Icons wrong color (low contrast, invisible)
- Icon-only buttons without labels or tooltips
**Examples:**
```
❌ Icon button: 16x16px icon in 48x48px button (looks lost)
✅ Icon button: 24x24px icon in 48x48px button (balanced)
❌ Icon: white on light gray background (barely visible)
✅ Icon: dark gray on light gray (clear contrast)
❌ Icon: baseline-aligned with text (appears raised)
✅ Icon: center-aligned with text
❌ Icon-only button with no label (unclear purpose)
✅ Icon button with aria-label or visible text label
```
**Screenshot Analysis:**
- Check if icons are appropriately sized
- Verify icons have sufficient contrast
- Look for proper alignment with adjacent text
- Ensure icon buttons have clear purpose
## Color and Theme Issues
### 15. Dark Mode Issues
**Visual Indicators (when comparing light/dark screenshots):**
- White text on light background (inverted incorrectly)
- Hard-coded colors not switching with theme
- Images/logos with wrong theme variant
- Insufficient contrast in dark mode
**Examples:**
```
❌ Dark mode: #333 text on #000 background (low contrast)
✅ Dark mode: #E0E0E0 text on #1A1A1A background
❌ Light mode logo on dark background (invisible)
✅ Dark mode variant logo displayed
❌ Input background: white in both modes (wrong in dark)
✅ Input background: white in light, #2A2A2A in dark
```
**Screenshot Analysis (compare light/dark if available):**
- Verify all colors invert appropriately
- Check contrast ratios in both modes
- Look for hard-coded colors that don't adapt
- Ensure images/logos have correct variants
### 16. Brand Color Misuse
**Visual Indicators:**
- Too many competing colors
- Brand colors used incorrectly (primary for everything)
- Status colors confusing (green for error, red for success)
- Inaccessible color combinations
**Examples:**
```
❌ All buttons primary color (no hierarchy)
✅ Primary button: brand color, secondary: gray/outline
❌ Success message in red, error in green (confusing)
✅ Success in green, error in red, warning in amber
❌ 8 different colors used on one page (chaotic)
✅ Consistent color palette: 2-3 main colors + neutrals
```
**Screenshot Analysis:**
- Check if color usage is consistent and meaningful
- Verify status colors match conventions (green=success, red=error)
- Look for excessive color variety
- Ensure brand colors used appropriately
## Animation and Transition Issues
**Note:** Difficult to detect from static screenshots, but can infer
### 17. Loading States
**Visual Indicators:**
- Content area completely empty (no skeleton/spinner)
- "Loading..." text with no visual indicator
- Sudden content appearance (jarring)
- Infinite loading (screenshot shows spinner forever)
**Examples:**
```
❌ Empty white space while loading (looks broken)
✅ Skeleton UI placeholders during load
❌ Just text "Loading..." (static, looks stuck)
✅ Animated spinner + "Loading..." text
❌ Screenshot from 30 seconds ago: still loading (timeout issue)
✅ Content loads within reasonable time (< 3 seconds)
```
**Screenshot Analysis:**
- Look for loading indicators
- Check if empty states have placeholders
- Identify potential timeout issues (loading too long)
## Mobile-Specific Issues
### 18. Fixed Positioning Problems
**Visual Indicators:**
- Fixed header covering content (not enough top padding)
- Fixed footer hiding interactive elements
- Input fields hidden behind keyboard (inferred)
- Fixed elements overlapping each other
**Examples:**
```
❌ Fixed header: covers first line of content
✅ Content has top padding equal to header height
❌ Fixed "Chat with us" button: covers form submit button
✅ Fixed button repositions when other content appears
❌ Input field: likely behind keyboard when focused
✅ Page scrolls input into view above keyboard
```
**Screenshot Analysis (mobile viewports):**
- Check if fixed headers leave room for content
- Verify fixed elements don't overlap important content
- Look for sufficient padding to account for fixed elements
### 19. Orientation Issues
**Visual Indicators (portrait vs landscape):**
- Content cut off in landscape mode
- Poor use of available space in landscape
- Fixed height elements that don't adapt
- Horizontal layout forced into vertical space
**Examples:**
```
❌ Portrait: shows full content → Landscape: content cut off
✅ Both orientations show full content
❌ Landscape: wide empty margins, cramped center content
✅ Landscape: content uses available width appropriately
```
**Screenshot Analysis (if both orientations available):**
- Compare same page in portrait and landscape
- Verify content adapts to available space
- Check if all content remains accessible
## Analysis Priority
When analyzing screenshots, prioritize issues by impact:
### Critical (Stop immediately)
1. Content completely missing or invisible
2. Major layout breaks (overlapping, off-screen)
3. Severe contrast violations (< 3:1)
4. Broken images or core UI elements
### High (Fix soon)
1. Text truncation losing important info
2. Form validation not visible
3. Responsive breakpoint failures
4. Touch targets too small (< 44px)
### Medium (Fix in next iteration)
1. Inconsistent spacing
2. Minor alignment issues
3. Missing hover/focus states
4. Moderate contrast issues (3:1-4.4:1)
### Low (Polish)
1. Minor typography inconsistencies
2. Slight spacing irregularities
3. Non-critical icon sizing
4. Subtle animation issues
## Generating Bug Reports
For each issue found, provide:
```markdown
### [Issue Title]
**Severity**: Critical | High | Medium | Low
**Location**: [Specific page/component where visible]
**Screenshot**: `path/to/screenshot.png` (timestamp: YYYY-MM-DD HH:MM:SS)
**Viewport**: Desktop 1280x720 | Tablet 768x1024 | Mobile 375x667
**Description**: [Clear description of what's wrong]
**Expected Behavior**: [What should appear instead]
**Likely Cause**: [Technical reason, e.g., "Missing max-width constraint", "Improper flexbox configuration"]
**Recommended Fix**:
- **File**: `src/components/Button.tsx`
- **Line**: 45
- **Current**: \`className="px-4 text-xl"\`
- **Fixed**: \`className="px-4 text-sm sm:text-base md:text-xl max-w-full"\`
- **Reasoning**: Text size needs responsive scaling and max-width to prevent overflow on mobile
```
---
**Remember**: Focus on bugs that impact **usability** and **accessibility**. Not every minor imperfection is critical. Prioritize issues that prevent users from completing tasks or accessing content.

415
skills/playwright-e2e-automation/data/error-patterns.yaml Normal file
View File

@@ -0,0 +1,415 @@
# Error Pattern Recovery Database
# Maps common error patterns to diagnosis and recovery steps
# This file is used for:
# 1. Pre-flight health checks - detect errors before running full test suite
# 2. Test failure analysis - provide actionable fixes when tests fail
# 3. User guidance - self-service troubleshooting
# ========== CSS & STYLING ERRORS ==========
css_errors:
tailwind_v4_syntax_mismatch:
pattern: "Cannot apply unknown utility class"
alternative_patterns:
- "Utilities must be known at build time"
- "Unknown utility class"
diagnosis: "Tailwind CSS v4 detected but v3 syntax used in CSS file"
severity: "critical"
category: "configuration"
root_cause: |
Tailwind CSS v4 changed from @tailwind directives to @import syntax.
Your CSS file likely still uses the old @tailwind directives.
detection_method: "console_error"
recovery_steps:
- step: "Identify your main CSS file"
details: "Usually src/index.css, src/App.css, or src/globals.css"
- step: "Update CSS directives"
from: |
@tailwind base;
@tailwind components;
@tailwind utilities;
to: |
@import "tailwindcss";
files_to_check:
- "src/index.css"
- "src/App.css"
- "src/globals.css"
- "src/styles/globals.css"
- step: "Update PostCSS configuration"
from: |
plugins: {
tailwindcss: {},
autoprefixer: {},
}
to: |
plugins: {
'@tailwindcss/postcss': {},
autoprefixer: {},
}
files_to_check:
- "postcss.config.js"
- "postcss.config.cjs"
- "postcss.config.mjs"
- step: "Restart dev server"
command: "npm run dev"
reason: "CSS changes require server restart"
- step: "Clear browser cache and reload"
details: "Hard refresh: Ctrl+Shift+R (Windows) or Cmd+Shift+R (Mac)"
prevention: |
The skill now detects Tailwind version and uses appropriate templates.
This error should not occur in new setups.
documentation: "https://tailwindcss.com/docs/upgrade-guide"
related_errors:
- "postcss_plugin_not_found"
postcss_plugin_not_found:
pattern: "Plugin tailwindcss not found"
alternative_patterns:
- "Cannot find module 'tailwindcss'"
- "postcss plugin tailwindcss not found"
diagnosis: "PostCSS configuration uses old Tailwind v3 plugin name with Tailwind v4"
severity: "critical"
category: "configuration"
root_cause: |
Tailwind CSS v4 renamed its PostCSS plugin from 'tailwindcss' to '@tailwindcss/postcss'.
Your postcss.config.js still references the old plugin name.
recovery_steps:
- step: "Update postcss.config.js"
from: |
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};
to: |
export default {
plugins: {
'@tailwindcss/postcss': {},
autoprefixer: {},
},
};
file: "postcss.config.js"
- step: "Verify @tailwindcss/postcss is installed"
command: "npm list @tailwindcss/postcss"
if_not_installed: "npm install -D @tailwindcss/postcss"
- step: "Restart dev server"
command: "npm run dev"
documentation: "https://tailwindcss.com/docs/upgrade-guide#migrating-from-v3"
# ========== ACCESSIBILITY ERRORS ==========
accessibility_errors:
heading_hierarchy_violation:
pattern: "heading-order - Heading levels should only increase by one"
alternative_patterns:
- "Heading levels should increase by one"
- "heading-order violation"
diagnosis: "WCAG heading hierarchy violation - skipped heading levels"
severity: "moderate"
category: "accessibility"
root_cause: |
HTML heading elements (h1-h6) must follow logical order without skipping levels.
For example: h1 → h2 → h3 is correct, but h1 → h3 (skipping h2) is incorrect.
recovery_steps:
- step: "Locate the problematic heading in test output"
details: "Playwright accessibility tests will show file and line number"
- step: "Check heading hierarchy in that component"
example: |
❌ Bad:
<h1>Page Title</h1>
<h3>Section</h3> <!-- Skips h2 -->
✅ Good:
<h1>Page Title</h1>
<h2>Section</h2>
- step: "Fix heading levels to follow order"
details: "Ensure each heading is only one level deeper than its parent"
- step: "Re-run accessibility tests"
command: "npm run test:e2e -- accessibility.spec.ts"
prevention: |
Always outline content structure before implementing:
- Page title: h1 (only one per page)
- Main sections: h2
- Subsections: h3
- Sub-subsections: h4
documentation: "https://www.w3.org/WAI/WCAG21/Understanding/info-and-relationships.html"
missing_form_labels:
pattern: "Form elements must have labels"
alternative_patterns:
- "label - Form elements must have labels"
- "Inputs must have associated labels"
diagnosis: "Form input missing associated label element"
severity: "high"
category: "accessibility"
root_cause: |
Every form input must have an associated <label> element for screen reader users.
Placeholder text alone is NOT sufficient.
recovery_steps:
- step: "Add label element to input"
from: |
<input type="email" placeholder="Email" />
to: |
<label htmlFor="email">Email Address</label>
<input id="email" type="email" placeholder="Email" />
- step: "Alternative: Use aria-label if visual label not desired"
example: |
<input
type="search"
aria-label="Search"
placeholder="Search..."
/>
- step: "Ensure label and input are properly associated"
details: "Use 'htmlFor' attribute matching input 'id'"
- step: "Re-run accessibility tests"
command: "npm run test:e2e -- accessibility.spec.ts"
documentation: "https://www.w3.org/WAI/tutorials/forms/labels/"
insufficient_color_contrast:
pattern: "color-contrast - Elements must meet minimum color contrast ratio"
alternative_patterns:
- "color contrast ratio"
- "contrast ratio"
diagnosis: "Text or UI element has insufficient color contrast (WCAG 2.1 AA violation)"
severity: "high"
category: "accessibility"
root_cause: |
WCAG 2.1 AA requires:
- Normal text: 4.5:1 contrast ratio
- Large text (18pt+ or 14pt+ bold): 3:1 contrast ratio
- UI components: 3:1 contrast ratio
recovery_steps:
- step: "Identify low-contrast element from test output"
details: "Test results show which element and current ratio"
- step: "Use WebAIM contrast checker"
url: "https://webaim.org/resources/contrastchecker/"
details: "Test current colors and find compliant alternatives"
- step: "Update color values"
example: |
❌ Bad: #AAAAAA on #FFFFFF (2.6:1 ratio)
✅ Good: #595959 on #FFFFFF (7:1 ratio)
For Tailwind:
❌ text-gray-400 → ✅ text-gray-700
- step: "Re-run accessibility tests"
command: "npm run test:e2e -- accessibility.spec.ts"
prevention: |
Use established color palettes with pre-tested contrast ratios.
Tailwind's default palette (500+ for text) generally meets WCAG AA.
documentation: "https://www.w3.org/WAI/WCAG21/Understanding/contrast-minimum.html"
# ========== BUILD & CONFIGURATION ERRORS ==========
build_errors:
vite_port_in_use:
pattern: "Port 5173 is in use"
alternative_patterns:
- "EADDRINUSE"
- "address already in use"
diagnosis: "Vite dev server port already in use by another process"
severity: "moderate"
category: "environment"
recovery_steps:
- step: "Find process using port 5173"
command: "lsof -i :5173"
platforms: ["mac", "linux"]
- step: "Kill the process"
command: "kill -9 <PID>"
details: "Replace <PID> with process ID from previous command"
- step: "Alternative: Use different port"
details: "Add to vite.config.ts:"
config: |
server: {
port: 3000
}
- step: "Restart dev server"
command: "npm run dev"
playwright_browsers_not_installed:
pattern: "Executable doesn't exist"
alternative_patterns:
- "browserType.launch: Executable doesn't exist"
- "Browser not found"
diagnosis: "Playwright browser binaries not installed"
severity: "critical"
category: "installation"
recovery_steps:
- step: "Install Playwright browsers"
command: "npx playwright install"
details: "Downloads Chromium, Firefox, and WebKit"
- step: "Install system dependencies (Linux only)"
command: "npx playwright install-deps"
platforms: ["linux"]
- step: "Verify installation"
command: "npx playwright test --list"
expected: "Should list available tests without errors"
typescript_strict_errors:
pattern: "TypeScript strict mode errors"
alternative_patterns:
- "TS2345"
- "TS2322"
- "Type 'any' is not assignable"
diagnosis: "TypeScript strict mode violations in generated code"
severity: "moderate"
category: "type_safety"
recovery_steps:
- step: "Review TypeScript errors in output"
details: "Each error shows file, line, and type issue"
- step: "Add explicit type annotations"
example: |
❌ const data = await fetch(url);
✅ const data: Response = await fetch(url);
- step: "Use type assertions cautiously"
example: |
const element = page.locator('#id') as Locator;
- step: "Fix null/undefined handling"
example: |
❌ const text = element.textContent();
✅ const text = await element.textContent() ?? '';
# ========== RESPONSIVE & LAYOUT ERRORS ==========
layout_errors:
horizontal_scroll_mobile:
pattern: "Horizontal scroll detected on mobile viewport"
diagnosis: "Content wider than viewport on mobile, causing horizontal scroll"
severity: "moderate"
category: "responsive"
root_cause: |
Fixed-width elements or overflow content exceeds mobile viewport width.
Common causes: large images, wide tables, fixed px widths.
recovery_steps:
- step: "Identify overflowing element"
details: "Use browser DevTools → Elements → Computed → scroll width"
- step: "Make element responsive"
example: |
❌ width: 800px;
✅ max-width: 100%;
❌ className="w-[800px]"
✅ className="w-full max-w-screen-lg"
- step: "Add overflow handling"
example: |
className="overflow-x-auto" // For tables
className="overflow-hidden" // For containers
- step: "Test on mobile viewport"
command: "npm run test:e2e -- --project=mobile-chrome"
# ========== RECOVERY STRATEGIES ==========
recovery_strategies:
incremental_fix:
description: "Fix errors one at a time, test after each fix"
when_to_use: "Multiple related errors"
steps:
- "Fix highest severity error first"
- "Run tests to verify fix"
- "Move to next error"
- "Repeat until all errors resolved"
clean_slate:
description: "Remove generated files and regenerate"
when_to_use: "Configuration completely broken"
steps:
- "Delete generated playwright.config.ts"
- "Delete tests/ directory"
- "Clear node_modules/.cache/"
- "Re-run skill"
version_rollback:
description: "Downgrade to last working version"
when_to_use: "Breaking change in newly installed package"
steps:
- "Check package.json for recently updated packages"
- "Install previous version: npm install package@previous-version"
- "Lock version in package.json"
- "Test to confirm working"
# ========== ERROR SEVERITY LEVELS ==========
severity_definitions:
critical:
impact: "App won't build or run"
response_time: "Fix immediately"
examples: ["Syntax errors", "Missing dependencies", "Config errors"]
high:
impact: "Major functionality broken"
response_time: "Fix within hours"
examples: ["Accessibility violations", "Failed test cases", "Type errors"]
moderate:
impact: "Reduced usability or performance"
response_time: "Fix within days"
examples: ["Layout issues", "Minor accessibility", "Warnings"]
low:
impact: "Cosmetic or minor issues"
response_time: "Fix when convenient"
examples: ["Code style", "Documentation", "Minor visual bugs"]

430
skills/playwright-e2e-automation/data/framework-detection-patterns.yaml Normal file
View File

@@ -0,0 +1,430 @@
# Framework Detection Patterns
# Used to automatically identify application type and determine optimal Playwright configuration
# Detection happens in phases:
# 1. Read package.json dependencies
# 2. Check for config files
# 3. Identify dev server command
# 4. Determine base URL and port
frameworks:
# ========== REACT + VITE ==========
react_vite:
name: "React + Vite"
priority: 1 # Check first (most common modern setup)
detection:
package_json_dependencies:
required:
- "react"
- "vite"
optional:
- "react-dom"
- "@vitejs/plugin-react"
config_files:
- "vite.config.ts"
- "vite.config.js"
- "vite.config.mjs"
indicators:
- "src/main.tsx exists"
- "src/main.jsx exists"
- "index.html with Vite script tag"
configuration:
base_url: "http://localhost:5173"
dev_server_command: "npm run dev"
build_command: "npm run build"
preview_command: "npm run preview"
playwright_config:
timeout: 30000
retries: 2
use:
viewport: { width: 1280, height: 720 }
screenshot: "only-on-failure"
video: "retain-on-failure"
projects:
- name: "chromium"
use:
browserName: "chromium"
- name: "mobile"
use:
...devices["iPhone 13"]
webServer:
command: "npm run dev"
url: "http://localhost:5173"
reuseExistingServer: true
timeout: 120000
test_generation:
entry_point: "src/App.tsx"
routing: "react-router-dom" # If detected in dependencies
state_management: ["redux", "zustand", "jotai"] # Check for these
common_pages:
- "/"
- "/about"
- "/contact"
- "/login"
- "/dashboard"
# ========== NEXT.JS ==========
nextjs:
name: "Next.js"
priority: 2
detection:
package_json_dependencies:
required:
- "next"
- "react"
optional:
- "react-dom"
config_files:
- "next.config.js"
- "next.config.mjs"
- "next.config.ts"
indicators:
- "app/ directory exists" # App Router
- "pages/ directory exists" # Pages Router
- ".next/ directory exists"
configuration:
base_url: "http://localhost:3000"
dev_server_command: "npm run dev"
build_command: "npm run build"
playwright_config:
timeout: 45000 # Next.js can be slower on first load
retries: 2
webServer:
command: "npm run dev"
url: "http://localhost:3000"
reuseExistingServer: true
timeout: 120000
test_generation:
router_type: "app" # or "pages" - detect from directory structure
entry_point: "app/page.tsx" # App Router
api_routes: "app/api/" # Check if API routes exist
common_pages:
- "/"
- "/about"
- "/blog"
- "/contact"
# ========== CREATE REACT APP ==========
create_react_app:
name: "Create React App"
priority: 3
detection:
package_json_dependencies:
required:
- "react"
- "react-scripts"
optional:
- "react-dom"
config_files:
- "public/index.html"
indicators:
- "src/index.js exists"
- "src/index.tsx exists"
- "package.json scripts.start includes react-scripts"
configuration:
base_url: "http://localhost:3000"
dev_server_command: "npm start"
build_command: "npm run build"
playwright_config:
timeout: 30000
retries: 2
webServer:
command: "npm start"
url: "http://localhost:3000"
reuseExistingServer: true
timeout: 120000
test_generation:
entry_point: "src/App.js"
routing: "react-router-dom"
# ========== NODE.JS + EXPRESS ==========
express:
name: "Node.js + Express"
priority: 4
detection:
package_json_dependencies:
required:
- "express"
optional:
- "ejs"
- "pug"
- "handlebars"
indicators:
- "app.js exists"
- "server.js exists"
- "index.js exists"
- "views/ directory exists"
configuration:
base_url: "http://localhost:3000" # Default, check process.env.PORT
dev_server_command: "npm run dev"
alternative_commands:
- "npm start"
- "node server.js"
- "nodemon server.js"
playwright_config:
timeout: 30000
retries: 2
webServer:
command: "npm run dev"
url: "http://localhost:3000"
reuseExistingServer: true
test_generation:
entry_point: "server.js"
template_engine: "ejs" # Detect from dependencies
api_endpoints: true # Generate API tests
common_routes:
- "/"
- "/api/health"
- "/api/users"
# ========== STATIC HTML/CSS/JS ==========
static:
name: "Static HTML/CSS/JS"
priority: 10 # Check last (fallback)
detection:
indicators:
- "index.html exists in root"
- "No package.json"
- "No build tools detected"
configuration:
base_url: "http://localhost:8080"
dev_server_command: "npx serve ."
install_dev_server: "npm install -g serve" # Install if needed
playwright_config:
timeout: 15000 # Faster, no build step
retries: 1
webServer:
command: "npx serve . -l 8080"
url: "http://localhost:8080"
reuseExistingServer: true
test_generation:
entry_point: "index.html"
detect_pages_from:
- "HTML files in root"
- "Links in index.html"
# ========== ASTRO ==========
astro:
name: "Astro"
priority: 5
detection:
package_json_dependencies:
required:
- "astro"
config_files:
- "astro.config.mjs"
indicators:
- "src/pages/ directory exists"
configuration:
base_url: "http://localhost:4321"
dev_server_command: "npm run dev"
playwright_config:
webServer:
command: "npm run dev"
url: "http://localhost:4321"
reuseExistingServer: true
# ========== SVELTE + VITE ==========
svelte_vite:
name: "Svelte + Vite"
priority: 6
detection:
package_json_dependencies:
required:
- "svelte"
- "vite"
optional:
- "@sveltejs/vite-plugin-svelte"
config_files:
- "svelte.config.js"
- "vite.config.js"
configuration:
base_url: "http://localhost:5173"
dev_server_command: "npm run dev"
# ========== VUE + VITE ==========
vue_vite:
name: "Vue + Vite"
priority: 7
detection:
package_json_dependencies:
required:
- "vue"
- "vite"
optional:
- "@vitejs/plugin-vue"
config_files:
- "vite.config.ts"
- "vite.config.js"
configuration:
base_url: "http://localhost:5173"
dev_server_command: "npm run dev"
# ========== NUXT ==========
nuxt:
name: "Nuxt"
priority: 8
detection:
package_json_dependencies:
required:
- "nuxt"
config_files:
- "nuxt.config.ts"
- "nuxt.config.js"
configuration:
base_url: "http://localhost:3000"
dev_server_command: "npm run dev"
# ========== DETECTION WORKFLOW ==========
detection_workflow:
steps:
- name: "Check for package.json"
action: "Read package.json if exists"
output: "dependencies and devDependencies lists"
- name: "Match framework patterns"
action: "Compare dependencies against framework patterns"
priority: "Use priority field to determine order"
output: "List of matching frameworks (highest priority first)"
- name: "Verify with config files"
action: "Check if expected config files exist"
output: "Confirm framework match"
- name: "Check additional indicators"
action: "Verify directory structure and entry points"
output: "Final framework identification"
- name: "Determine dev server"
action: "Read package.json scripts section"
fallback: "Use framework's default dev_server_command"
output: "Command to start dev server"
- name: "Detect port"
action: "Check if server is already running"
fallback: "Use framework's default base_url port"
output: "Base URL for tests"
# ========== PORT DETECTION ==========
port_detection:
methods:
- name: "Check running processes"
command: "lsof -i :3000 -i :5173 -i :8080 -i :4321"
description: "See if common ports are in use"
- name: "Check package.json scripts"
pattern: "--port (\\d+)"
description: "Extract port from dev script"
- name: "Check config files"
files:
- "vite.config.ts": "server.port"
- "next.config.js": "devServer.port"
- ".env": "PORT"
- name: "Attempt connection"
ports: [3000, 5173, 8080, 4321, 8000, 4200]
description: "Try common ports in order"
# ========== MULTI-APP DETECTION ==========
fullstack_detection:
patterns:
monorepo:
indicators:
- "package.json with workspaces field"
- "packages/ or apps/ directory"
- "Lerna or Nx configuration"
strategy:
- "Detect each package separately"
- "Generate tests for each app"
- "Configure multiple web servers if needed"
separate_frontend_backend:
indicators:
- "client/ and server/ directories"
- "frontend/ and backend/ directories"
- "Multiple package.json files"
strategy:
- "Detect frontend framework"
- "Detect backend framework"
- "Start both servers for e2e tests"
- "Configure API proxy if needed"
# ========== FALLBACK STRATEGY ==========
fallback:
when_no_match:
- "Prompt user to specify framework manually"
- "List detected files and ask for clarification"
- "Suggest generic static server approach"
generic_config:
base_url: "http://localhost:8080"
dev_server: "npx serve . -l 8080"
timeout: 30000
# ========== VALIDATION ==========
validation:
checks:
- name: "Server starts successfully"
test: "Run dev server command, wait for port to open"
timeout: 120000
failure: "Ask user to start server manually"
- name: "Base URL accessible"
test: "HTTP GET to base_url returns 200"
timeout: 30000
failure: "Check if port is different, ask user"
- name: "Page renders content"
test: "Page has visible content (not blank)"
timeout: 10000
failure: "Possible SPA routing issue, check entry point"

303
skills/playwright-e2e-automation/data/framework-versions.yaml Normal file
View File

@@ -0,0 +1,303 @@
# Framework Version Compatibility Database
# Used for version-aware template selection and configuration generation
# This file maps framework versions to their syntax, configuration requirements,
# and breaking changes. When the skill detects installed package versions, it
# consults this database to select appropriate templates and warn about issues.
# ========== TAILWIND CSS ==========
tailwindcss:
v3:
version_range: ">=3.0.0 <4.0.0"
detection_priority: 2
syntax:
css_directives:
- "@tailwind base;"
- "@tailwind components;"
- "@tailwind utilities;"
postcss_plugin: "tailwindcss"
config_file: "tailwind.config.js"
templates:
css: "templates/css/tailwind-v3.css"
postcss_config: "templates/configs/postcss-tailwind-v3.js"
tailwind_config: "templates/configs/tailwind-v3.config.js"
notes: "Stable version with @tailwind directive syntax"
v4:
version_range: ">=4.0.0"
detection_priority: 1 # Check first (latest)
syntax:
css_directives:
- "@import \"tailwindcss\";"
postcss_plugin: "@tailwindcss/postcss"
config_file: "tailwind.config.ts" # Now TypeScript by default
templates:
css: "templates/css/tailwind-v4.css"
postcss_config: "templates/configs/postcss-tailwind-v4.js"
tailwind_config: "templates/configs/tailwind-v4.config.ts"
breaking_changes:
- "CSS syntax changed from @tailwind directives to @import"
- "PostCSS plugin renamed from 'tailwindcss' to '@tailwindcss/postcss'"
- "Configuration file now TypeScript by default"
- "Some utility classes restructured (check migration guide)"
migration_guide: "https://tailwindcss.com/docs/upgrade-guide"
notes: "Major rewrite with new @import syntax and improved PostCSS integration"
# ========== REACT ==========
react:
v17:
version_range: ">=17.0.0 <18.0.0"
detection_priority: 2
features:
jsx_transform: "classic" # Requires React import
concurrent_features: false
automatic_batching: false
templates:
component: "templates/react/component-v17.tsx"
notes: "Classic JSX transform, requires 'import React from react'"
v18:
version_range: ">=18.0.0 <19.0.0"
detection_priority: 1
features:
jsx_transform: "automatic" # No React import needed
concurrent_features: true
automatic_batching: true
use_client_directive: false # Not yet (that's React 19)
templates:
component: "templates/react/component-v18.tsx"
breaking_changes:
- "Automatic batching may affect state update timing"
- "Concurrent features require opt-in (via createRoot)"
- "IE11 no longer supported"
migration_guide: "https://react.dev/blog/2022/03/08/react-18-upgrade-guide"
notes: "New JSX transform, concurrent features, automatic batching"
v19:
version_range: ">=19.0.0"
detection_priority: 1
features:
jsx_transform: "automatic"
concurrent_features: true
automatic_batching: true
use_client_directive: true # Server Components
actions: true # Server Actions
templates:
component: "templates/react/component-v19.tsx"
breaking_changes:
- "'use client' directive required for client components in RSC apps"
- "ref is now a regular prop (no forwardRef needed)"
- "Context.Provider shorthand removed (use <Context> directly)"
migration_guide: "https://react.dev/blog/2024/04/25/react-19-upgrade-guide"
notes: "Server Components, Actions, ref as prop, improved performance"
# ========== NEXT.JS ==========
nextjs:
v13:
version_range: ">=13.0.0 <14.0.0"
detection_priority: 2
features:
app_router: true # Optional
pages_router: true
turbopack: "beta"
server_actions: "alpha"
router_detection:
app_router: "app/ directory exists"
pages_router: "pages/ directory exists"
templates:
config: "templates/nextjs/next-v13.config.js"
page_app: "templates/nextjs/page-v13-app.tsx"
page_pages: "templates/nextjs/page-v13-pages.tsx"
notes: "App Router introduced alongside Pages Router"
v14:
version_range: ">=14.0.0 <15.0.0"
detection_priority: 1
features:
app_router: true
pages_router: true
turbopack: "stable"
server_actions: "stable"
partial_prerendering: "experimental"
router_detection:
app_router: "app/ directory exists"
pages_router: "pages/ directory exists"
templates:
config: "templates/nextjs/next-v14.config.js"
page_app: "templates/nextjs/page-v14-app.tsx"
page_pages: "templates/nextjs/page-v14-pages.tsx"
breaking_changes:
- "Server Actions stable (syntax changes from v13)"
- "Turbopack stable for dev (replaces webpack in dev mode)"
- "Minimum Node.js version: 18.17"
migration_guide: "https://nextjs.org/docs/app/building-your-application/upgrading/version-14"
notes: "Stable Server Actions and Turbopack"
# ========== VITE ==========
vite:
v4:
version_range: ">=4.0.0 <5.0.0"
detection_priority: 2
features:
default_port: 5173
rollup_version: 3
css_code_split: true
templates:
config: "templates/vite/vite-v4.config.ts"
notes: "Stable Vite 4 with Rollup 3"
v5:
version_range: ">=5.0.0 <6.0.0"
detection_priority: 1
features:
default_port: 5173
rollup_version: 4
css_code_split: true
improved_hmr: true
templates:
config: "templates/vite/vite-v5.config.ts"
breaking_changes:
- "Rollup 4 (plugin compatibility check needed)"
- "Minimum Node.js version: 18.0"
- "Some deprecated options removed"
migration_guide: "https://vitejs.dev/guide/migration"
notes: "Vite 5 with Rollup 4 and improved HMR"
# ========== PLAYWRIGHT ==========
playwright:
v1_40_plus:
version_range: ">=1.40.0"
detection_priority: 1
features:
component_testing: true
trace_viewer: true
codegen: true
auto_waiting: true
templates:
config: "templates/playwright.config.template.ts"
test_spec: "templates/test-spec.template.ts"
page_object: "templates/page-object.template.ts"
notes: "Modern Playwright with full feature set"
# ========== POSTCSS ==========
postcss:
v8:
version_range: ">=8.0.0"
detection_priority: 1
notes: "Standard PostCSS v8 - most common version"
# ========== VERSION DETECTION STRATEGIES ==========
detection_strategies:
priority_order:
description: "Check versions in priority order (higher priority first)"
example: "For Tailwind: check v4 rules before v3 rules"
semver_matching:
description: "Use semver.satisfies() for version range matching"
library: "semver npm package"
fallback:
description: "If no version matches, warn and use sensible defaults"
default_strategy: "Use latest stable templates with warning"
multi_framework:
description: "Detect multiple frameworks simultaneously"
example: "React 18 + Vite 5 + Tailwind 4 + Playwright 1.40"
# ========== COMMON BREAKING CHANGE PATTERNS ==========
breaking_change_categories:
syntax_changes:
examples:
- "Tailwind v3→v4: @tailwind → @import"
- "React v17→v18: Optional JSX import"
impact: "Critical - app won't build/run"
detection: "Parse error, syntax error in build output"
configuration_changes:
examples:
- "Tailwind v4: PostCSS plugin rename"
- "Next.js v14: next.config.js options"
impact: "High - build fails or misconfigured"
detection: "Build error, plugin not found"
api_changes:
examples:
- "React v19: ref as prop"
- "Next.js v13: getServerSideProps in App Router"
impact: "Medium - runtime errors or warnings"
detection: "TypeScript errors, runtime warnings"
deprecation_warnings:
examples:
- "React v18: ReactDOM.render → createRoot"
- "Vite: Legacy options removed"
impact: "Low - works but will break in future"
detection: "Console warnings, deprecation notices"
# ========== USAGE NOTES ==========
usage:
detection_flow:
- "Read package.json dependencies and devDependencies"
- "For each framework, iterate through versions in priority order"
- "Use semver.satisfies(installedVersion, versionRange)"
- "First match wins (highest priority)"
- "If no match, use fallback and warn"
template_selection:
- "Based on detected version, select appropriate template files"
- "Templates use version-specific syntax and best practices"
- "Combine multiple framework templates (e.g., React + Tailwind + Vite)"
error_prevention:
- "Check for known breaking changes before generating config"
- "Warn user if mixing incompatible versions"
- "Provide migration guides for major version differences"
maintenance:
- "Add new versions as they're released"
- "Update breaking_changes based on real-world issues"
- "Keep migration_guide links current"
- "Test template compatibility regularly"

456
skills/playwright-e2e-automation/data/playwright-best-practices.md Normal file
View File

@@ -0,0 +1,456 @@
# Playwright Best Practices
Official best practices for Playwright test automation, optimized for LLM-assisted development.
## Test Structure
### Use Page Object Models (POM)
**Why**: Separates page structure from test logic, improves maintainability
```typescript
// Good: Page Object Model
// pages/login.page.ts
export class LoginPage {
constructor(private page: Page) {}
async goto() {
await this.page.goto('/login');
}
async login(username: string, password: string) {
await this.page.getByLabel('Username').fill(username);
await this.page.getByLabel('Password').fill(password);
await this.page.getByRole('button', { name: 'Sign in' }).click();
}
}
// specs/login.spec.ts
test('user can login', async ({ page }) => {
const loginPage = new LoginPage(page);
await loginPage.goto();
await loginPage.login('user@example.com', 'password123');
await expect(page).toHaveURL('/dashboard');
});
```
### Use Semantic Selectors
**Priority order** (most stable → least stable):
1. **getByRole** - Accessible role (button, heading, textbox, etc.)
2. **getByLabel** - Form inputs with associated labels
3. **getByPlaceholder** - Input placeholder text
4. **getByText** - User-visible text content
5. **getByTestId** - data-testid attributes (last resort)
```typescript
// Best: Role-based (accessible and stable)
await page.getByRole('button', { name: 'Submit' }).click();
// Good: Label-based (for forms)
await page.getByLabel('Email address').fill('user@example.com');
// Acceptable: Text-based
await page.getByText('Continue to checkout').click();
// Avoid: CSS selectors (brittle)
await page.click('.btn-primary'); // ❌ Breaks if class changes
// Last resort: Test IDs (when semantic selectors don't work)
await page.getByTestId('checkout-button').click();
```
## Screenshot Best Practices
### When to Capture Screenshots
1. **Initial page load** - Baseline visual state
2. **Before interaction** - Pre-state for comparison
3. **After interaction** - Result of user action
4. **Error states** - When validation fails or errors occur
5. **Success states** - Confirmation screens, success messages
6. **Test failures** - Automatic capture for debugging
### Screenshot Naming Convention
```typescript
// Pattern: {test-name}-{viewport}-{state}-{timestamp}.png
await page.screenshot({
path: `screenshots/current/login-desktop-initial-${Date.now()}.png`,
fullPage: true
});
await page.screenshot({
path: `screenshots/current/checkout-mobile-error-${Date.now()}.png`,
fullPage: true
});
```
### Full-Page vs Element Screenshots
```typescript
// Full-page: For layout and overall UI analysis
await page.screenshot({
path: 'homepage-full.png',
fullPage: true // Captures entire scrollable page
});
// Element-specific: For component testing
const button = page.getByRole('button', { name: 'Submit' });
await button.screenshot({
path: 'submit-button.png'
});
```
## Waiting and Timing
### Auto-Waiting
Playwright automatically waits for:
- Element to be attached to DOM
- Element to be visible
- Element to be stable (not animating)
- Element to receive events (not obscured)
- Element to be enabled
```typescript
// This automatically waits for button to be clickable
await page.getByRole('button', { name: 'Submit' }).click();
```
### Explicit Waits (when needed)
```typescript
// Wait for navigation
await page.waitForURL('/dashboard');
// Wait for network idle (good before screenshots)
await page.waitForLoadState('networkidle');
// Wait for specific element
await page.waitForSelector('img[alt="Profile picture"]');
// Wait for custom condition
await page.waitForFunction(() => window.scrollY === 0);
```
### Avoid Fixed Timeouts
```typescript
// Bad: Arbitrary delays
await page.waitForTimeout(3000); // ❌ Flaky, slow
// Good: Wait for specific condition
await expect(page.getByText('Success')).toBeVisible(); // ✅ Fast and reliable
```
## Test Isolation
### Independent Tests
Each test should be completely independent:
```typescript
// Good: Test is self-contained
test('user can add item to cart', async ({ page }) => {
// Set up: Create user, log in
await page.goto('/');
await login(page, 'user@example.com', 'password');
// Action: Add to cart
await page.getByRole('button', { name: 'Add to cart' }).click();
// Assert: Item in cart
await expect(page.getByTestId('cart-count')).toHaveText('1');
// Cleanup happens automatically with new page context
});
// Bad: Depends on previous test state
test('user can checkout', async ({ page }) => {
// ❌ Assumes cart already has items from previous test
await page.goto('/checkout');
// ...
});
```
### Use test.beforeEach for Common Setup
```typescript
test.describe('Shopping cart', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
await login(page, 'user@example.com', 'password');
});
test('can add item to cart', async ({ page }) => {
// Setup already done
await page.getByRole('button', { name: 'Add to cart' }).click();
await expect(page.getByTestId('cart-count')).toHaveText('1');
});
test('can remove item from cart', async ({ page }) => {
// Setup already done, fresh state
await page.getByRole('button', { name: 'Add to cart' }).click();
await page.getByRole('button', { name: 'Remove' }).click();
await expect(page.getByTestId('cart-count')).toHaveText('0');
});
});
```
## Visual Regression Testing
### Snapshot Testing
```typescript
// Basic snapshot
await expect(page).toHaveScreenshot('homepage.png');
// With threshold (allow minor differences)
await expect(page).toHaveScreenshot('homepage.png', {
maxDiffPixelRatio: 0.05 // Allow 5% difference
});
// Element snapshot
const card = page.getByRole('article').first();
await expect(card).toHaveScreenshot('product-card.png');
```
### Updating Baselines
```bash
# Update all snapshots
npx playwright test --update-snapshots
# Update specific test
npx playwright test login.spec.ts --update-snapshots
```
### Baseline Management
- **Store baselines in git** - Commit to repository for consistency
- **Review diffs carefully** - Not all changes are bugs
- **Update deliberately** - Only update when changes are intentional
- **Use CI checks** - Fail pipeline on unexpected visual changes
## Configuration Best Practices
### playwright.config.ts Essentials
```typescript
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
// Timeout for each test
timeout: 30 * 1000,
// Global setup/teardown
globalSetup: require.resolve('./tests/setup/global-setup.ts'),
// Fail fast on CI, retry locally
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
// Parallel execution
workers: process.env.CI ? 1 : undefined,
// Reporter
reporter: process.env.CI ? 'github' : 'html',
use: {
// Base URL
baseURL: 'http://localhost:5173',
// Screenshot on failure
screenshot: 'only-on-failure',
// Trace on first retry
trace: 'on-first-retry',
// Video on failure
video: 'retain-on-failure',
},
// Projects for multi-browser testing
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
{
name: 'mobile-chrome',
use: { ...devices['Pixel 5'] },
},
{
name: 'mobile-safari',
use: { ...devices['iPhone 13'] },
},
],
// Web server for dev
webServer: {
command: 'npm run dev',
url: 'http://localhost:5173',
reuseExistingServer: !process.env.CI,
},
});
```
## Debugging
### Playwright Inspector
```bash
# Debug specific test
npx playwright test --debug login.spec.ts
# Debug from specific line
npx playwright test --debug --grep "user can login"
```
### VS Code Debugger
```json
// .vscode/launch.json
{
"configurations": [
{
"name": "Debug Playwright Tests",
"type": "node",
"request": "launch",
"program": "${workspaceFolder}/node_modules/@playwright/test/cli.js",
"args": ["test", "--headed", "${file}"],
"console": "integratedTerminal"
}
]
}
```
### Trace Viewer
```bash
# Run with trace
npx playwright test --trace on
# View trace
npx playwright show-trace trace.zip
```
## Performance Optimization
### Parallel Execution
```typescript
// Run tests in parallel (default)
test.describe.configure({ mode: 'parallel' });
// Run tests serially (when needed)
test.describe.configure({ mode: 'serial' });
```
### Reuse Authentication State
```typescript
// global-setup.ts
import { chromium } from '@playwright/test';
export default async function globalSetup() {
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('http://localhost:5173/login');
await page.getByLabel('Username').fill('admin');
await page.getByLabel('Password').fill('password');
await page.getByRole('button', { name: 'Sign in' }).click();
// Save authentication state
await page.context().storageState({ path: 'auth.json' });
await browser.close();
}
// Use in tests
test.use({ storageState: 'auth.json' });
```
## Common Pitfalls to Avoid
### 1. Not Waiting for Network Idle Before Screenshots
```typescript
// Bad: Screenshot may capture loading state
await page.goto('/dashboard');
await page.screenshot({ path: 'dashboard.png' });
// Good: Wait for content to load
await page.goto('/dashboard');
await page.waitForLoadState('networkidle');
await page.screenshot({ path: 'dashboard.png' });
```
### 2. Using Non-Stable Selectors
```typescript
// Bad: Position-based (breaks if order changes)
await page.locator('button').nth(2).click();
// Good: Content-based
await page.getByRole('button', { name: 'Submit' }).click();
```
### 3. Not Handling Dynamic Content
```typescript
// Bad: Assumes content is already loaded
const text = await page.getByTestId('user-name').textContent();
// Good: Wait for element first
await expect(page.getByTestId('user-name')).toBeVisible();
const text = await page.getByTestId('user-name').textContent();
```
### 4. Overly Broad Assertions
```typescript
// Bad: Fails on any minor change
await expect(page).toHaveScreenshot({ maxDiffPixelRatio: 0 });
// Good: Allow reasonable tolerance
await expect(page).toHaveScreenshot({ maxDiffPixelRatio: 0.02 });
```
## Summary Checklist
- [ ] Use Page Object Models for test organization
- [ ] Prefer semantic selectors (getByRole, getByLabel)
- [ ] Capture screenshots at key interaction points
- [ ] Wait for network idle before screenshots
- [ ] Use auto-waiting instead of fixed timeouts
- [ ] Make tests independent and isolated
- [ ] Configure proper retry logic (2-3 retries in CI)
- [ ] Store authentication state for reuse
- [ ] Use trace viewer for debugging
- [ ] Review visual diffs before updating baselines
- [ ] Run tests in parallel for performance
- [ ] Enable screenshot/video on failure
- [ ] Store baselines in version control
- [ ] Use meaningful screenshot names with timestamps
- [ ] Configure appropriate visual diff thresholds
---
**References:**
- [Playwright Official Docs](https://playwright.dev/)
- [Best Practices Guide](https://playwright.dev/docs/best-practices)
- [Locators Guide](https://playwright.dev/docs/locators)