311 lines
9.9 KiB
Markdown
311 lines
9.9 KiB
Markdown
# Element Interaction Workflows
|
|
|
|
User input simulation for clicking, typing, form filling, and drag & drop operations.
|
|
|
|
## Tools in This Group
|
|
|
|
### click
|
|
Clicks on an element identified by its UID.
|
|
|
|
**Required:** `--uid UID`
|
|
**Optional:** `--dblClick [true|false]`
|
|
|
|
```bash
|
|
# Single click
|
|
node scripts/click.js --uid button_submit_abc123
|
|
|
|
# Double click
|
|
node scripts/click.js --uid file_icon_xyz789 --dblClick true
|
|
```
|
|
|
|
### fill
|
|
Types text into inputs, textareas, or selects options in dropdown menus.
|
|
|
|
**Required:** `--uid UID --value VALUE`
|
|
|
|
```bash
|
|
# Fill text input
|
|
node scripts/fill.js --uid input_username_abc --value john.doe
|
|
|
|
# Fill textarea
|
|
node scripts/fill.js --uid textarea_comment_def --value "This is a comment"
|
|
|
|
# Select dropdown option
|
|
node scripts/fill.js --uid select_country_ghi --value "United States"
|
|
```
|
|
|
|
### fill_form
|
|
Fills multiple form fields at once using a JSON array.
|
|
|
|
**Required:** `--elements JSON_ARRAY`
|
|
|
|
```bash
|
|
node scripts/fill_form.js --elements '[{"uid":"input_email","value":"test@example.com"},{"uid":"input_password","value":"secret123"}]'
|
|
```
|
|
|
|
### hover
|
|
Hovers the mouse over an element.
|
|
|
|
**Required:** `--uid UID`
|
|
|
|
```bash
|
|
node scripts/hover.js --uid tooltip_trigger_abc
|
|
```
|
|
|
|
### drag
|
|
Drags one element onto another element.
|
|
|
|
**Required:** `--from-uid FROM_UID --to-uid TO_UID`
|
|
|
|
```bash
|
|
node scripts/drag.js --from-uid draggable_item_abc --to-uid dropzone_xyz
|
|
```
|
|
|
|
### upload_file
|
|
Uploads a file through a file input element.
|
|
|
|
**Required:** `--uid UID --filePath FILEPATH`
|
|
|
|
```bash
|
|
node scripts/upload_file.js --uid input_file_upload --filePath /Users/username/documents/resume.pdf
|
|
```
|
|
|
|
### press_key
|
|
Presses a key or key combination (for keyboard shortcuts, navigation, special keys).
|
|
|
|
**Required:** `--key KEY`
|
|
|
|
```bash
|
|
# Single key
|
|
node scripts/press_key.js --key Enter
|
|
|
|
# Key combination
|
|
node scripts/press_key.js --key "Control+S"
|
|
|
|
# Navigation
|
|
node scripts/press_key.js --key ArrowDown
|
|
```
|
|
|
|
## Workflows
|
|
|
|
### Workflow: Login Form Automation
|
|
|
|
Complete login process with validation:
|
|
|
|
- [ ] Open login page: `node scripts/new_page.js --url https://example.com/login`
|
|
- [ ] Wait for form: `node scripts/wait_for.js --text "Sign In" --timeout 5000`
|
|
- [ ] Get element UIDs: `node scripts/take_snapshot.js`
|
|
- [ ] Fill username: `node scripts/fill.js --uid input_username_a1b2 --value testuser`
|
|
- [ ] Fill password: `node scripts/fill.js --uid input_password_c3d4 --value mypassword123`
|
|
- [ ] Submit form: `node scripts/click.js --uid button_submit_e5f6`
|
|
- [ ] Wait for redirect: `node scripts/wait_for.js --text "Dashboard" --timeout 10000`
|
|
- [ ] Verify login: `node scripts/take_screenshot.js --filePath logged_in.png`
|
|
|
|
**Input Example:**
|
|
```
|
|
Username field UID: input_username_a1b2
|
|
Password field UID: input_password_c3d4
|
|
Submit button UID: button_submit_e5f6
|
|
```
|
|
|
|
**Expected Output:**
|
|
User logged in successfully, redirected to dashboard.
|
|
|
|
### Workflow: Multi-Field Form with Dropdowns
|
|
|
|
Fill complex form with various input types:
|
|
|
|
- [ ] Navigate to form: `node scripts/new_page.js --url https://example.com/signup`
|
|
- [ ] Get structure: `node scripts/take_snapshot.js`
|
|
- [ ] Fill first name: `node scripts/fill.js --uid input_firstname --value John`
|
|
- [ ] Fill last name: `node scripts/fill.js --uid input_lastname --value Doe`
|
|
- [ ] Fill email: `node scripts/fill.js --uid input_email --value john.doe@example.com`
|
|
- [ ] Select country: `node scripts/fill.js --uid select_country --value "United States"`
|
|
- [ ] Select state: `node scripts/fill.js --uid select_state --value California`
|
|
- [ ] Fill textarea: `node scripts/fill.js --uid textarea_bio --value "Software engineer with 5 years experience"`
|
|
- [ ] Submit: `node scripts/click.js --uid button_register`
|
|
- [ ] Verify: `node scripts/wait_for.js --text "Registration successful" --timeout 10000`
|
|
|
|
**Expected Output:**
|
|
All form fields filled correctly, registration completed.
|
|
|
|
### Workflow: Bulk Form Filling
|
|
|
|
Use fill_form for efficient multi-field updates:
|
|
|
|
- [ ] Open form: `node scripts/new_page.js --url https://example.com/profile`
|
|
- [ ] Identify UIDs: `node scripts/take_snapshot.js`
|
|
- [ ] Fill all fields at once:
|
|
```bash
|
|
node scripts/fill_form.js --elements '[
|
|
{"uid":"input_name","value":"John Doe"},
|
|
{"uid":"input_email","value":"john@example.com"},
|
|
{"uid":"input_phone","value":"555-1234"},
|
|
{"uid":"select_timezone","value":"America/Los_Angeles"}
|
|
]'
|
|
```
|
|
- [ ] Review changes: `node scripts/take_snapshot.js`
|
|
- [ ] Save: `node scripts/click.js --uid button_save`
|
|
- [ ] Verify: `node scripts/wait_for.js --text "Profile updated" --timeout 5000`
|
|
|
|
**Expected Output:**
|
|
All fields updated in single operation, profile saved successfully.
|
|
|
|
### Workflow: File Upload with Validation
|
|
|
|
Upload a file and verify success:
|
|
|
|
- [ ] Navigate to upload page: `node scripts/new_page.js --url https://example.com/upload`
|
|
- [ ] Get file input UID: `node scripts/take_snapshot.js`
|
|
- [ ] Select file: `node scripts/upload_file.js --uid input_file_abc123 --filePath /Users/username/documents/report.pdf`
|
|
- [ ] Wait for preview: `node scripts/wait_for.js --text "report.pdf" --timeout 5000`
|
|
- [ ] Verify file name appears: `node scripts/take_snapshot.js`
|
|
- [ ] Click upload button: `node scripts/click.js --uid button_upload_xyz`
|
|
- [ ] Wait for completion: `node scripts/wait_for.js --text "Upload successful" --timeout 30000`
|
|
- [ ] Capture confirmation: `node scripts/take_screenshot.js --filePath upload_complete.png`
|
|
|
|
**Input Example:**
|
|
File path: `/Users/username/documents/report.pdf`
|
|
File input UID: `input_file_abc123`
|
|
|
|
**Expected Output:**
|
|
File uploaded successfully, confirmation message displayed.
|
|
|
|
### Workflow: Drag and Drop Interface
|
|
|
|
Interact with drag-and-drop components:
|
|
|
|
- [ ] Open drag interface: `node scripts/new_page.js --url https://example.com/kanban`
|
|
- [ ] Get board structure: `node scripts/take_snapshot.js`
|
|
- [ ] Drag task 1: `node scripts/drag.js --from-uid task_item_1 --to-uid column_in_progress`
|
|
- [ ] Wait for update: `node scripts/wait_for.js --text "Task moved" --timeout 3000`
|
|
- [ ] Drag task 2: `node scripts/drag.js --from-uid task_item_2 --to-uid column_done`
|
|
- [ ] Verify final state: `node scripts/take_snapshot.js`
|
|
- [ ] Save changes: `node scripts/click.js --uid button_save_board`
|
|
|
|
**Expected Output:**
|
|
Tasks moved between columns, board state updated and saved.
|
|
|
|
### Workflow: Keyboard Navigation
|
|
|
|
Use keyboard shortcuts and navigation keys:
|
|
|
|
- [ ] Open page: `node scripts/new_page.js --url https://example.com/editor`
|
|
- [ ] Focus text area: `node scripts/click.js --uid textarea_editor_abc`
|
|
- [ ] Type content: `node scripts/fill.js --uid textarea_editor_abc --value "Hello world"`
|
|
- [ ] Select all: `node scripts/press_key.js --key "Control+A"`
|
|
- [ ] Copy: `node scripts/press_key.js --key "Control+C"`
|
|
- [ ] Navigate down: `node scripts/press_key.js --key ArrowDown`
|
|
- [ ] Paste: `node scripts/press_key.js --key "Control+V"`
|
|
- [ ] Save: `node scripts/press_key.js --key "Control+S"`
|
|
- [ ] Verify save: `node scripts/wait_for.js --text "Saved" --timeout 5000`
|
|
|
|
**Expected Output:**
|
|
Keyboard shortcuts execute correctly, content manipulated and saved.
|
|
|
|
### Workflow: Hover Tooltips and Menus
|
|
|
|
Interact with hover-triggered UI elements:
|
|
|
|
- [ ] Open page: `node scripts/new_page.js --url https://example.com/dashboard`
|
|
- [ ] Get elements: `node scripts/take_snapshot.js`
|
|
- [ ] Hover over help icon: `node scripts/hover.js --uid icon_help_abc`
|
|
- [ ] Wait for tooltip: `node scripts/wait_for.js --text "Click for help" --timeout 2000`
|
|
- [ ] Capture tooltip: `node scripts/take_screenshot.js --filePath tooltip.png`
|
|
- [ ] Hover over menu: `node scripts/hover.js --uid menu_trigger_xyz`
|
|
- [ ] Wait for dropdown: `node scripts/wait_for.js --text "Settings" --timeout 2000`
|
|
- [ ] Click menu item: `node scripts/click.js --uid menu_item_settings`
|
|
|
|
**Expected Output:**
|
|
Tooltips and dropdowns appear on hover, menu items clickable.
|
|
|
|
## Common Patterns
|
|
|
|
### Pattern: Always Snapshot Before Interaction
|
|
|
|
Get fresh UIDs for every interaction:
|
|
|
|
```bash
|
|
# 1. Navigate to page
|
|
node scripts/new_page.js --url https://example.com/form
|
|
|
|
# 2. Wait for page load
|
|
node scripts/wait_for.js --text "Submit" --timeout 5000
|
|
|
|
# 3. Get current element UIDs
|
|
node scripts/take_snapshot.js
|
|
|
|
# 4. Use UIDs from snapshot output
|
|
node scripts/fill.js --uid <uid-from-snapshot> --value "data"
|
|
```
|
|
|
|
### Pattern: Form Validation Handling
|
|
|
|
Handle form validation errors:
|
|
|
|
```bash
|
|
# 1. Fill form
|
|
node scripts/fill.js --uid input_email --value "invalid-email"
|
|
|
|
# 2. Submit
|
|
node scripts/click.js --uid button_submit
|
|
|
|
# 3. Check for error message
|
|
node scripts/wait_for.js --text "Invalid email" --timeout 3000
|
|
|
|
# 4. Correct the input
|
|
node scripts/fill.js --uid input_email --value "valid@example.com"
|
|
|
|
# 5. Resubmit
|
|
node scripts/click.js --uid button_submit
|
|
```
|
|
|
|
### Pattern: Wait After Each Interaction
|
|
|
|
Ensure UI updates before next action:
|
|
|
|
```bash
|
|
# 1. Click button
|
|
node scripts/click.js --uid button_load_more
|
|
|
|
# 2. Wait for new content
|
|
node scripts/wait_for.js --text "Showing 20 items" --timeout 5000
|
|
|
|
# 3. Now interact with new elements
|
|
node scripts/take_snapshot.js
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
**Problem:** Element UID not found
|
|
|
|
**Solution:** UIDs are dynamic. Always take a fresh snapshot before interaction. Don't reuse old UIDs.
|
|
|
|
**Problem:** Click doesn't trigger action
|
|
|
|
**Solution:**
|
|
- Ensure element is visible and clickable (not hidden)
|
|
- Try waiting for page to fully load first with `wait_for.js`
|
|
- Check if element requires hover first: use `hover.js` then `click.js`
|
|
|
|
**Problem:** Fill doesn't work on input
|
|
|
|
**Solution:**
|
|
- Verify the element is an input/textarea/select
|
|
- For some custom components, use `click.js` to focus, then `press_key.js` to type
|
|
- Check if page has JavaScript that prevents fill
|
|
|
|
**Problem:** File upload fails
|
|
|
|
**Solution:**
|
|
- Use absolute file paths (not relative)
|
|
- Verify file exists at the specified path
|
|
- Use forward slashes in paths: `/Users/username/file.pdf`
|
|
|
|
**Problem:** Drag and drop doesn't work
|
|
|
|
**Solution:**
|
|
- Ensure both UIDs are valid and current
|
|
- Some drag implementations require specific timing - add `wait_for.js` between actions
|
|
- Verify dropzone accepts the dragged element type
|