Initial commit

This commit is contained in:
Zhongwei Li
2025-11-29 18:30:46 +08:00
commit ca9abd0543
12 changed files with 419 additions and 0 deletions

View File

@@ -0,0 +1,62 @@
---
name: experiment-design-kit
description: Toolkit for structuring hypotheses, variants, guardrails, and measurement
plans.
---
# Experiment Design Kit Skill
## When to Use
- Translating raw ideas into testable hypotheses with clear success metrics.
- Ensuring experiment briefs include guardrails, instrumentation, and rollout details.
- Coaching pods on best practices for multi-variant or multi-surface tests.
## Framework
1. **Problem Framing** define user problem, business impact, and north-star metric.
2. **Hypothesis Structure** "If we do X for Y persona, we expect Z change" with assumptions.
3. **Measurement Plan** primary metric, guardrails, min detectable effect, power calc.
4. **Variant Strategy** control definition, variant catalog, targeting, and exclusion rules.
5. **Operational Plan** owners, timeline, dependencies, QA/rollback steps.
## Templates
- Experiment brief (context, hypothesis, design, metrics, launch checklist).
- Guardrail register with thresholds + alerting rules.
- Variant matrix for surfaces, messaging, and states.
- **GTM Agents Growth Backlog Board** capture idea → sizing → prioritization scoring (ICE/RICE) @puerto/README.md#183-212.
- **Weekly Experiment Packet** includes KPI guardrails, qualitative notes, and next bets for Marketing Director + Sales Director.
- **Rollback Playbook** pre-built checklist tied to lifecycle-mapping rip-cord procedures.
## Tips
- Pressure-test hypotheses with counter-metrics to avoid local optima.
- Document data constraints early to avoid rework during build.
- Pair with `guardrail-scorecard` to ensure sign-off before launch.
- Apply GTM Agents cadence: Monday backlog groom, Wednesday build review, Friday learnings sync.
- Require KPI guardrails per stage (activation, engagement, monetization) before authorizing build.
- If a test risks Sales velocity, include Sales Director in approval routing per GTM Agents governance.
## GTM Agents Experiment Operating Model
1. **Backlog Intake** ideas flow from GTM pods; Growth Marketer tags theme, objective, expected impact.
2. **Prioritization** score with RICE + qualitative "strategic fit" modifier; surface top 3 bets weekly.
3. **Design & Instrumentation** reference Serena/Context7 to patch code + confirm documentation.
4. **Launch & Monitor** use guardrail-scorecard to watch leading indicators (churn, complaints, latency).
5. **Learning Loop** run Sequential Thinking retro; document hypothesis, result, decision, follow-up in backlog card.
## KPI Guardrails (GTM Agents Reference)
- Activation rate change must stay within ±3% of baseline for Tier-1 segments.
- Revenue per visitor cannot drop more than 2% for more than 48h.
- Support tickets tied to experiment variant must remain <5% of total volume.
## Weekly Experiment Packet Outline
```
Week Ending: <Date>
1. Portfolio Snapshot tests live, status, KPI trend (guardrail vs actual)
2. Key Wins hypothesis, uplift, next action (ship, iterate, expand)
3. Guardrail Alerts what tripped, mitigation taken (rollback? scope adjust?)
4. Pipeline Impact SQLs, ARR influenced, notable customer anecdotes
5. Upcoming Launches dependencies, owners, open questions
```
Share packet with Growth, Marketing Director, Sales Director, and RevOps to mirror GTM Agents's cross-functional communication rhythm.
---

View File

@@ -0,0 +1,31 @@
---
name: guardrail-scorecard
description: Framework for defining, monitoring, and enforcing guardrail metrics across
experiments.
---
# Guardrail Scorecard Skill
## When to Use
- Setting non-negotiable metrics (stability, churn, latency, compliance) before launching tests.
- Monitoring live experiments to ensure guardrails stay within thresholds.
- Reporting guardrail status in launch packets and post-test readouts.
## Framework
1. **Metric Inventory** list guardrail metrics, owners, data sources, refresh cadence.
2. **Threshold Matrix** define warning vs critical bands per metric / persona / region.
3. **Alerting & Escalation** map notification channels, DRI, and decision timelines.
4. **Exception Handling** document when guardrail overrides are acceptable and required approvals.
5. **Retrospective Loop** log breaches, mitigations, and rule updates for future tests.
## Templates
- Guardrail register (metric, threshold, owner, alert channel).
- Live monitoring dashboard layout.
- Exception memo structure for approvals.
## Tips
- Tie guardrails to downstream systems (billing, support) to catch second-order impacts.
- Keep thresholds dynamic for seasonality but document logic.
- Pair with `launch-experiment` to ensure readiness before flipping flags.
---

View File

@@ -0,0 +1,31 @@
---
name: hypothesis-library
description: Curated repository of experiment hypotheses, assumptions, and historical
learnings.
---
# Hypothesis Library Skill
## When to Use
- Capturing new experiment ideas with consistent metadata.
- Referencing past wins/losses before prioritizing the backlog.
- Sharing reusable learnings across pods and channels.
## Framework
1. **Metadata Schema** hypothesis ID, theme, persona, funnel stage, metrics.
2. **Assumptions Matrix** belief statements, supporting evidence, confidence rating.
3. **Status Tracking** idea → scoped → running → decided → archived.
4. **Learning Tags** impact summary, guardrail notes, follow-up ideas.
5. **Governance Hooks** approvals, owners, review cadence.
## Templates
- Intake form for new hypotheses.
- Learning card format (context, result, recommendation).
- Portfolio dashboard summarizing mix by theme/metric.
## Tips
- Require at least one supporting data point before moving to prioritization.
- Use consistent tagging so search/filtering works across teams.
- Link to `synthesize-learnings` outputs to keep narratives fresh.
---