Initial commit
This commit is contained in:
Zhongwei Li
391
skills/spring-boot/spring-boot-resilience4j/SKILL.md
Normal file
391
skills/spring-boot/spring-boot-resilience4j/SKILL.md
Normal file
@@ -0,0 +1,391 @@
|
||||
---
|
||||
name: spring-boot-resilience4j
|
||||
description: This skill should be used when implementing fault tolerance and resilience patterns in Spring Boot applications using the Resilience4j library. Apply this skill to add circuit breaker, retry, rate limiter, bulkhead, time limiter, and fallback mechanisms to prevent cascading failures, handle transient errors, and manage external service dependencies gracefully in microservices architectures.
|
||||
allowed-tools: Read, Write, Edit, Bash
|
||||
category: backend
|
||||
tags: [spring-boot, resilience4j, circuit-breaker, fault-tolerance, retry, bulkhead, rate-limiter]
|
||||
version: 1.1.0
|
||||
---
|
||||
|
||||
# Spring Boot Resilience4j Patterns
|
||||
|
||||
## When to Use
|
||||
|
||||
To implement resilience patterns in Spring Boot applications, use this skill when:
|
||||
- Preventing cascading failures from external service unavailability with circuit breaker pattern
|
||||
- Retrying transient failures with exponential backoff
|
||||
- Rate limiting to protect services from overload or downstream service capacity constraints
|
||||
- Isolating resources with bulkhead pattern to prevent thread pool exhaustion
|
||||
- Adding timeout controls to async operations with time limiter
|
||||
- Combining multiple patterns for comprehensive fault tolerance
|
||||
|
||||
Resilience4j is a lightweight, composable library for adding fault tolerance without requiring external infrastructure. It provides annotation-based patterns that integrate seamlessly with Spring Boot's AOP and Actuator.
|
||||
|
||||
## Instructions
|
||||
|
||||
### 1. Setup and Dependencies
|
||||
|
||||
Add Resilience4j dependencies to your project. For Maven, add to `pom.xml`:
|
||||
|
||||
```xml
|
||||
<dependency>
|
||||
<groupId>io.github.resilience4j</groupId>
|
||||
<artifactId>resilience4j-spring-boot3</artifactId>
|
||||
<version>2.2.0</version> // Use latest stable version
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-aop</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-actuator</artifactId>
|
||||
</dependency>
|
||||
```
|
||||
|
||||
For Gradle, add to `build.gradle`:
|
||||
|
||||
```gradle
|
||||
implementation "io.github.resilience4j:resilience4j-spring-boot3:2.2.0"
|
||||
implementation "org.springframework.boot:spring-boot-starter-aop"
|
||||
implementation "org.springframework.boot:spring-boot-starter-actuator"
|
||||
```
|
||||
|
||||
Enable AOP annotation processing with `@EnableAspectJAutoProxy` (auto-configured by Spring Boot).
|
||||
|
||||
### 2. Circuit Breaker Pattern
|
||||
|
||||
Apply `@CircuitBreaker` annotation to methods calling external services:
|
||||
|
||||
```java
|
||||
@Service
|
||||
public class PaymentService {
|
||||
private final RestTemplate restTemplate;
|
||||
|
||||
public PaymentService(RestTemplate restTemplate) {
|
||||
this.restTemplate = restTemplate;
|
||||
}
|
||||
|
||||
@CircuitBreaker(name = "paymentService", fallbackMethod = "paymentFallback")
|
||||
public PaymentResponse processPayment(PaymentRequest request) {
|
||||
return restTemplate.postForObject("http://payment-api/process",
|
||||
request, PaymentResponse.class);
|
||||
}
|
||||
|
||||
private PaymentResponse paymentFallback(PaymentRequest request, Exception ex) {
|
||||
return PaymentResponse.builder()
|
||||
.status("PENDING")
|
||||
.message("Service temporarily unavailable")
|
||||
.build();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Configure in `application.yml`:
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
circuitbreaker:
|
||||
configs:
|
||||
default:
|
||||
registerHealthIndicator: true
|
||||
slidingWindowSize: 10
|
||||
minimumNumberOfCalls: 5
|
||||
failureRateThreshold: 50
|
||||
waitDurationInOpenState: 10s
|
||||
instances:
|
||||
paymentService:
|
||||
baseConfig: default
|
||||
```
|
||||
|
||||
See @references/configuration-reference.md for complete circuit breaker configuration options.
|
||||
|
||||
### 3. Retry Pattern
|
||||
|
||||
Apply `@Retry` annotation for transient failure recovery:
|
||||
|
||||
```java
|
||||
@Service
|
||||
public class ProductService {
|
||||
private final RestTemplate restTemplate;
|
||||
|
||||
public ProductService(RestTemplate restTemplate) {
|
||||
this.restTemplate = restTemplate;
|
||||
}
|
||||
|
||||
@Retry(name = "productService", fallbackMethod = "getProductFallback")
|
||||
public Product getProduct(Long productId) {
|
||||
return restTemplate.getForObject(
|
||||
"http://product-api/products/" + productId,
|
||||
Product.class);
|
||||
}
|
||||
|
||||
private Product getProductFallback(Long productId, Exception ex) {
|
||||
return Product.builder()
|
||||
.id(productId)
|
||||
.name("Unavailable")
|
||||
.available(false)
|
||||
.build();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Configure retry in `application.yml`:
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
retry:
|
||||
configs:
|
||||
default:
|
||||
maxAttempts: 3
|
||||
waitDuration: 500ms
|
||||
enableExponentialBackoff: true
|
||||
exponentialBackoffMultiplier: 2
|
||||
instances:
|
||||
productService:
|
||||
baseConfig: default
|
||||
maxAttempts: 5
|
||||
```
|
||||
|
||||
See @references/configuration-reference.md for retry exception configuration.
|
||||
|
||||
### 4. Rate Limiter Pattern
|
||||
|
||||
Apply `@RateLimiter` to control request rates:
|
||||
|
||||
```java
|
||||
@Service
|
||||
public class NotificationService {
|
||||
private final EmailClient emailClient;
|
||||
|
||||
public NotificationService(EmailClient emailClient) {
|
||||
this.emailClient = emailClient;
|
||||
}
|
||||
|
||||
@RateLimiter(name = "notificationService",
|
||||
fallbackMethod = "rateLimitFallback")
|
||||
public void sendEmail(EmailRequest request) {
|
||||
emailClient.send(request);
|
||||
}
|
||||
|
||||
private void rateLimitFallback(EmailRequest request, Exception ex) {
|
||||
throw new RateLimitExceededException(
|
||||
"Too many requests. Please try again later.");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Configure in `application.yml`:
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
ratelimiter:
|
||||
configs:
|
||||
default:
|
||||
registerHealthIndicator: true
|
||||
limitForPeriod: 10
|
||||
limitRefreshPeriod: 1s
|
||||
timeoutDuration: 500ms
|
||||
instances:
|
||||
notificationService:
|
||||
baseConfig: default
|
||||
limitForPeriod: 5
|
||||
```
|
||||
|
||||
### 5. Bulkhead Pattern
|
||||
|
||||
Apply `@Bulkhead` to isolate resources. Use `type = SEMAPHORE` for synchronous methods:
|
||||
|
||||
```java
|
||||
@Service
|
||||
public class ReportService {
|
||||
private final ReportGenerator reportGenerator;
|
||||
|
||||
public ReportService(ReportGenerator reportGenerator) {
|
||||
this.reportGenerator = reportGenerator;
|
||||
}
|
||||
|
||||
@Bulkhead(name = "reportService", type = Bulkhead.Type.SEMAPHORE)
|
||||
public Report generateReport(ReportRequest request) {
|
||||
return reportGenerator.generate(request);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use `type = THREADPOOL` for async/CompletableFuture methods:
|
||||
|
||||
```java
|
||||
@Service
|
||||
public class AnalyticsService {
|
||||
@Bulkhead(name = "analyticsService", type = Bulkhead.Type.THREADPOOL)
|
||||
public CompletableFuture<AnalyticsResult> runAnalytics(
|
||||
AnalyticsRequest request) {
|
||||
return CompletableFuture.supplyAsync(() ->
|
||||
analyticsEngine.analyze(request));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Configure in `application.yml`:
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
bulkhead:
|
||||
configs:
|
||||
default:
|
||||
maxConcurrentCalls: 10
|
||||
maxWaitDuration: 100ms
|
||||
instances:
|
||||
reportService:
|
||||
baseConfig: default
|
||||
maxConcurrentCalls: 5
|
||||
|
||||
thread-pool-bulkhead:
|
||||
instances:
|
||||
analyticsService:
|
||||
maxThreadPoolSize: 8
|
||||
```
|
||||
|
||||
### 6. Time Limiter Pattern
|
||||
|
||||
Apply `@TimeLimiter` to async methods to enforce timeout boundaries:
|
||||
|
||||
```java
|
||||
@Service
|
||||
public class SearchService {
|
||||
@TimeLimiter(name = "searchService", fallbackMethod = "searchFallback")
|
||||
public CompletableFuture<SearchResults> search(SearchQuery query) {
|
||||
return CompletableFuture.supplyAsync(() ->
|
||||
searchEngine.executeSearch(query));
|
||||
}
|
||||
|
||||
private CompletableFuture<SearchResults> searchFallback(
|
||||
SearchQuery query, Exception ex) {
|
||||
return CompletableFuture.completedFuture(
|
||||
SearchResults.empty("Search timed out"));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Configure in `application.yml`:
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
timelimiter:
|
||||
configs:
|
||||
default:
|
||||
timeoutDuration: 2s
|
||||
cancelRunningFuture: true
|
||||
instances:
|
||||
searchService:
|
||||
baseConfig: default
|
||||
timeoutDuration: 3s
|
||||
```
|
||||
|
||||
### 7. Combining Multiple Patterns
|
||||
|
||||
Stack multiple patterns on a single method for comprehensive fault tolerance:
|
||||
|
||||
```java
|
||||
@Service
|
||||
public class OrderService {
|
||||
@CircuitBreaker(name = "orderService")
|
||||
@Retry(name = "orderService")
|
||||
@RateLimiter(name = "orderService")
|
||||
@Bulkhead(name = "orderService")
|
||||
public Order createOrder(OrderRequest request) {
|
||||
return orderClient.createOrder(request);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Execution order: Retry → CircuitBreaker → RateLimiter → Bulkhead → Method
|
||||
|
||||
All patterns should reference the same named configuration instance for consistency.
|
||||
|
||||
### 8. Exception Handling and Monitoring
|
||||
|
||||
Create a global exception handler using `@RestControllerAdvice`:
|
||||
|
||||
```java
|
||||
@RestControllerAdvice
|
||||
public class ResilienceExceptionHandler {
|
||||
|
||||
@ExceptionHandler(CallNotPermittedException.class)
|
||||
@ResponseStatus(HttpStatus.SERVICE_UNAVAILABLE)
|
||||
public ErrorResponse handleCircuitOpen(CallNotPermittedException ex) {
|
||||
return new ErrorResponse("SERVICE_UNAVAILABLE",
|
||||
"Service currently unavailable");
|
||||
}
|
||||
|
||||
@ExceptionHandler(RequestNotPermitted.class)
|
||||
@ResponseStatus(HttpStatus.TOO_MANY_REQUESTS)
|
||||
public ErrorResponse handleRateLimited(RequestNotPermitted ex) {
|
||||
return new ErrorResponse("TOO_MANY_REQUESTS",
|
||||
"Rate limit exceeded");
|
||||
}
|
||||
|
||||
@ExceptionHandler(BulkheadFullException.class)
|
||||
@ResponseStatus(HttpStatus.SERVICE_UNAVAILABLE)
|
||||
public ErrorResponse handleBulkheadFull(BulkheadFullException ex) {
|
||||
return new ErrorResponse("CAPACITY_EXCEEDED",
|
||||
"Service at capacity");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Enable Actuator endpoints for monitoring resilience patterns in `application.yml`:
|
||||
|
||||
```yaml
|
||||
management:
|
||||
endpoints:
|
||||
web:
|
||||
exposure:
|
||||
include: health,metrics,circuitbreakers,retries,ratelimiters
|
||||
endpoint:
|
||||
health:
|
||||
show-details: always
|
||||
health:
|
||||
circuitbreakers:
|
||||
enabled: true
|
||||
ratelimiters:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
Access monitoring endpoints:
|
||||
- `GET /actuator/health` - Overall health including resilience patterns
|
||||
- `GET /actuator/circuitbreakers` - Circuit breaker states
|
||||
- `GET /actuator/metrics` - Custom resilience metrics
|
||||
|
||||
## Best Practices
|
||||
|
||||
- **Always provide fallback methods**: Ensure graceful degradation with meaningful responses rather than exceptions
|
||||
- **Use exponential backoff for retries**: Prevent overwhelming recovering services with aggressive backoff (`exponentialBackoffMultiplier: 2`)
|
||||
- **Choose appropriate failure thresholds**: Set `failureRateThreshold` between 50-70% depending on acceptable error rates
|
||||
- **Use constructor injection exclusively**: Never use field injection for Resilience4j dependencies
|
||||
- **Enable health indicators**: Set `registerHealthIndicator: true` for all patterns to integrate with Spring Boot health
|
||||
- **Separate failure vs. client errors**: Retry only transient errors (network timeouts, 5xx); skip 4xx and business exceptions
|
||||
- **Size bulkheads based on load**: Calculate thread pool and semaphore sizes from expected concurrent load and latency
|
||||
- **Monitor and adjust**: Continuously review metrics and adjust timeouts/thresholds based on production behavior
|
||||
- **Document fallback behavior**: Make fallback logic clear and predictable to users and maintainers
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
Refer to `references/testing-patterns.md` for:
|
||||
- Testing circuit breaker state transitions
|
||||
- Simulating transient failures with WireMock
|
||||
- Validating fallback method signatures
|
||||
- Avoiding common misconfiguration errors
|
||||
|
||||
Refer to `references/configuration-reference.md` for:
|
||||
- Complete property reference for all patterns
|
||||
- Configuration validation rules
|
||||
- Exception handling configuration
|
||||
|
||||
## References and Examples
|
||||
|
||||
- [Complete property reference and configuration patterns](references/configuration-reference.md)
|
||||
- [Unit and integration testing strategies](references/testing-patterns.md)
|
||||
- [Real-world e-commerce service example using all patterns](references/examples.md)
|
||||
- [Resilience4j Documentation](https://resilience4j.readme.io/)
|
||||
- [Spring Boot Actuator Skill](/skills/spring-boot-actuator/SKILL.md) - Monitoring resilience patterns with Actuator
|
||||
@@ -0,0 +1,360 @@
|
||||
# Resilience4j Configuration Reference
|
||||
|
||||
## Circuit Breaker Configuration
|
||||
|
||||
### Complete Properties List
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
circuitbreaker:
|
||||
configs:
|
||||
default:
|
||||
registerHealthIndicator: true # Default: false
|
||||
slidingWindowType: COUNT_BASED # COUNT_BASED or TIME_BASED
|
||||
slidingWindowSize: 100 # Default: 100 (calls or seconds)
|
||||
minimumNumberOfCalls: 10 # Default: 100
|
||||
failureRateThreshold: 50 # Default: 50 (percentage)
|
||||
slowCallRateThreshold: 100 # Default: 100 (percentage)
|
||||
slowCallDurationThreshold: 60s # Default: 60000ms
|
||||
waitDurationInOpenState: 60s # Default: 60000ms
|
||||
automaticTransitionFromOpenToHalfOpenEnabled: false # Default: false
|
||||
permittedNumberOfCallsInHalfOpenState: 10 # Default: 10
|
||||
maxWaitDurationInHalfOpenState: 0s # Default: 0 (unlimited)
|
||||
recordExceptions:
|
||||
- java.io.IOException
|
||||
- java.util.concurrent.TimeoutException
|
||||
ignoreExceptions:
|
||||
- java.lang.IllegalArgumentException
|
||||
eventConsumerBufferSize: 100 # Default: 100
|
||||
instances:
|
||||
myService:
|
||||
baseConfig: default
|
||||
failureRateThreshold: 60
|
||||
```
|
||||
|
||||
### Circuit Breaker States
|
||||
|
||||
1. **CLOSED**: Normal operation, calls pass through
|
||||
2. **OPEN**: Circuit is open, calls immediately fail with `CallNotPermittedException`
|
||||
3. **HALF_OPEN**: Testing if service recovered, allows limited test calls
|
||||
4. **DISABLED**: Circuit breaker disabled, all calls pass through
|
||||
5. **FORCED_OPEN**: Manually forced to open state for emergency situations
|
||||
|
||||
### Sliding Window Types
|
||||
|
||||
**COUNT_BASED** (Default)
|
||||
- Aggregates outcome of last N calls
|
||||
- Better for services with consistent traffic
|
||||
- `slidingWindowSize` = number of calls to track
|
||||
|
||||
**TIME_BASED**
|
||||
- Aggregates outcome of calls in last N seconds
|
||||
- Better for services with variable traffic
|
||||
- `slidingWindowSize` = time in seconds
|
||||
|
||||
## Retry Configuration
|
||||
|
||||
### Complete Properties List
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
retry:
|
||||
configs:
|
||||
default:
|
||||
maxAttempts: 3 # Default: 3
|
||||
waitDuration: 500ms # Default: 500ms
|
||||
enableExponentialBackoff: false # Default: false
|
||||
exponentialBackoffMultiplier: 2 # Default: 2
|
||||
exponentialMaxWaitDuration: 10s # Default: no limit
|
||||
enableRandomizedWait: false # Default: false
|
||||
randomizedWaitFactor: 0.5 # Default: 0.5
|
||||
retryExceptions:
|
||||
- java.io.IOException
|
||||
- org.springframework.web.client.ResourceAccessException
|
||||
ignoreExceptions:
|
||||
- java.lang.IllegalArgumentException
|
||||
failAfterMaxAttempts: false # Default: false
|
||||
eventConsumerBufferSize: 100 # Default: 100
|
||||
instances:
|
||||
myService:
|
||||
baseConfig: default
|
||||
maxAttempts: 5
|
||||
```
|
||||
|
||||
### Exponential Backoff Example
|
||||
|
||||
```yaml
|
||||
waitDuration: 500ms
|
||||
enableExponentialBackoff: true
|
||||
exponentialBackoffMultiplier: 2.0
|
||||
exponentialMaxWaitDuration: 10s
|
||||
```
|
||||
|
||||
Attempt waits:
|
||||
- Attempt 1: 500ms
|
||||
- Attempt 2: 1000ms (500 × 2)
|
||||
- Attempt 3: 2000ms (1000 × 2)
|
||||
- Attempt 4: 4000ms (2000 × 2)
|
||||
- Attempt 5: 8000ms (4000 × 2)
|
||||
- Maximum: 10000ms
|
||||
|
||||
## Rate Limiter Configuration
|
||||
|
||||
### Complete Properties List
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
ratelimiter:
|
||||
configs:
|
||||
default:
|
||||
limitForPeriod: 50 # Default: 50
|
||||
limitRefreshPeriod: 500ns # Default: 500ns
|
||||
timeoutDuration: 5s # Default: 5s
|
||||
registerHealthIndicator: true # Default: false
|
||||
allowHealthIndicatorToFail: true # Default: false
|
||||
instances:
|
||||
myService:
|
||||
baseConfig: default
|
||||
limitForPeriod: 10
|
||||
limitRefreshPeriod: 1s
|
||||
```
|
||||
|
||||
### Common Rate Limit Patterns
|
||||
|
||||
**10 requests per second**
|
||||
```yaml
|
||||
limitForPeriod: 10
|
||||
limitRefreshPeriod: 1s
|
||||
timeoutDuration: 0s # Fail immediately if no permits
|
||||
```
|
||||
|
||||
**100 requests per minute**
|
||||
```yaml
|
||||
limitForPeriod: 100
|
||||
limitRefreshPeriod: 1m
|
||||
timeoutDuration: 500ms # Wait up to 500ms for permit
|
||||
```
|
||||
|
||||
**5 requests per second with queuing**
|
||||
```yaml
|
||||
limitForPeriod: 5
|
||||
limitRefreshPeriod: 1s
|
||||
timeoutDuration: 2s # Wait up to 2s for permit
|
||||
```
|
||||
|
||||
## Bulkhead Configuration
|
||||
|
||||
### Semaphore Bulkhead
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
bulkhead:
|
||||
configs:
|
||||
default:
|
||||
maxConcurrentCalls: 25 # Default: 25
|
||||
maxWaitDuration: 0ms # Default: 0
|
||||
eventConsumerBufferSize: 100 # Default: 100
|
||||
instances:
|
||||
myService:
|
||||
baseConfig: default
|
||||
maxConcurrentCalls: 10
|
||||
maxWaitDuration: 100ms
|
||||
```
|
||||
|
||||
### Thread Pool Bulkhead
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
thread-pool-bulkhead:
|
||||
configs:
|
||||
default:
|
||||
maxThreadPoolSize: 4 # Default: Runtime.availableProcessors()
|
||||
coreThreadPoolSize: 2 # Default: Runtime.availableProcessors() - 1
|
||||
queueCapacity: 100 # Default: 100
|
||||
keepAliveDuration: 20ms # Default: 20ms
|
||||
writableStackTraceEnabled: true # Default: true
|
||||
instances:
|
||||
myService:
|
||||
baseConfig: default
|
||||
maxThreadPoolSize: 8
|
||||
coreThreadPoolSize: 4
|
||||
queueCapacity: 200
|
||||
```
|
||||
|
||||
## Time Limiter Configuration
|
||||
|
||||
### Complete Properties List
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
timelimiter:
|
||||
configs:
|
||||
default:
|
||||
timeoutDuration: 1s # Default: 1s
|
||||
cancelRunningFuture: true # Default: true
|
||||
instances:
|
||||
myService:
|
||||
baseConfig: default
|
||||
timeoutDuration: 3s
|
||||
```
|
||||
|
||||
## Annotation Reference
|
||||
|
||||
### @CircuitBreaker
|
||||
|
||||
```java
|
||||
@CircuitBreaker(
|
||||
name = "serviceName", // Required: Instance name from config
|
||||
fallbackMethod = "fallbackMethodName" // Optional: Fallback method name
|
||||
)
|
||||
|
||||
// Fallback method signature
|
||||
public String fallback(Long id, Exception ex) { }
|
||||
```
|
||||
|
||||
### @Retry
|
||||
|
||||
```java
|
||||
@Retry(
|
||||
name = "serviceName", // Required: Instance name from config
|
||||
fallbackMethod = "fallbackMethodName" // Optional: Fallback method name
|
||||
)
|
||||
```
|
||||
|
||||
### @RateLimiter
|
||||
|
||||
```java
|
||||
@RateLimiter(
|
||||
name = "serviceName",
|
||||
fallbackMethod = "fallbackMethodName"
|
||||
)
|
||||
```
|
||||
|
||||
### @Bulkhead
|
||||
|
||||
```java
|
||||
@Bulkhead(
|
||||
name = "serviceName",
|
||||
fallbackMethod = "fallbackMethodName",
|
||||
type = Bulkhead.Type.SEMAPHORE // SEMAPHORE or THREADPOOL
|
||||
)
|
||||
```
|
||||
|
||||
### @TimeLimiter
|
||||
|
||||
```java
|
||||
@TimeLimiter(
|
||||
name = "serviceName",
|
||||
fallbackMethod = "fallbackMethodName"
|
||||
)
|
||||
// Works only with CompletableFuture<T> or reactive types (Mono<T>, Flux<T>)
|
||||
```
|
||||
|
||||
## Annotation Execution Order
|
||||
|
||||
When combining annotations on a method, execution order from outermost to innermost:
|
||||
|
||||
1. `@Retry`
|
||||
2. `@CircuitBreaker`
|
||||
3. `@RateLimiter`
|
||||
4. `@TimeLimiter`
|
||||
5. `@Bulkhead`
|
||||
6. Actual method call
|
||||
|
||||
## Exception Reference
|
||||
|
||||
| Pattern | Exception | HTTP Status | Meaning |
|
||||
|---------|-----------|-------------|---------|
|
||||
| Circuit Breaker | `CallNotPermittedException` | 503 | Circuit is OPEN or FORCED_OPEN |
|
||||
| Rate Limiter | `RequestNotPermitted` | 429 | No permits available |
|
||||
| Bulkhead | `BulkheadFullException` | 503 | Bulkhead at capacity |
|
||||
| Time Limiter | `TimeoutException` | 408 | Operation exceeded timeout |
|
||||
|
||||
## Programmatic Configuration
|
||||
|
||||
### Circuit Breaker
|
||||
|
||||
```java
|
||||
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
|
||||
.failureRateThreshold(50)
|
||||
.waitDurationInOpenState(Duration.ofSeconds(10))
|
||||
.slowCallDurationThreshold(Duration.ofSeconds(2))
|
||||
.permittedNumberOfCallsInHalfOpenState(3)
|
||||
.minimumNumberOfCalls(10)
|
||||
.slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
|
||||
.slidingWindowSize(100)
|
||||
.recordExceptions(IOException.class, TimeoutException.class)
|
||||
.ignoreExceptions(IllegalArgumentException.class)
|
||||
.build();
|
||||
|
||||
CircuitBreakerRegistry registry = CircuitBreakerRegistry.of(config);
|
||||
CircuitBreaker circuitBreaker = registry.circuitBreaker("myService");
|
||||
```
|
||||
|
||||
### Retry
|
||||
|
||||
```java
|
||||
RetryConfig config = RetryConfig.custom()
|
||||
.maxAttempts(3)
|
||||
.waitDuration(Duration.ofMillis(500))
|
||||
.intervalFunction(IntervalFunction.ofExponentialBackoff(
|
||||
Duration.ofMillis(500),
|
||||
2.0
|
||||
))
|
||||
.retryExceptions(IOException.class, TimeoutException.class)
|
||||
.ignoreExceptions(IllegalArgumentException.class)
|
||||
.build();
|
||||
|
||||
RetryRegistry registry = RetryRegistry.of(config);
|
||||
Retry retry = registry.retry("myService");
|
||||
```
|
||||
|
||||
## Actuator Endpoints
|
||||
|
||||
Access monitoring endpoints when management endpoints are enabled:
|
||||
|
||||
| Endpoint | Description |
|
||||
|----------|-------------|
|
||||
| `GET /actuator/circuitbreakers` | List all circuit breakers and states |
|
||||
| `GET /actuator/circuitbreakerevents` | List circuit breaker events |
|
||||
| `GET /actuator/retryevents` | List retry events |
|
||||
| `GET /actuator/ratelimiters` | List rate limiters |
|
||||
| `GET /actuator/bulkheads` | List bulkhead status |
|
||||
| `GET /actuator/timelimiters` | List time limiters |
|
||||
| `GET /actuator/metrics` | Custom resilience metrics |
|
||||
|
||||
## Micrometer Metrics
|
||||
|
||||
Resilience4j exposes the following metrics:
|
||||
|
||||
**Circuit Breaker Metrics**
|
||||
- `resilience4j.circuitbreaker.calls{name, kind}`
|
||||
- `resilience4j.circuitbreaker.state{name, state}`
|
||||
- `resilience4j.circuitbreaker.failure.rate{name}`
|
||||
- `resilience4j.circuitbreaker.slow.call.rate{name}`
|
||||
|
||||
**Retry Metrics**
|
||||
- `resilience4j.retry.calls{name, kind}`
|
||||
|
||||
**Rate Limiter Metrics**
|
||||
- `resilience4j.ratelimiter.available.permissions{name}`
|
||||
- `resilience4j.ratelimiter.waiting_threads{name}`
|
||||
|
||||
**Bulkhead Metrics**
|
||||
- `resilience4j.bulkhead.available.concurrent.calls{name}`
|
||||
- `resilience4j.bulkhead.max.allowed.concurrent.calls{name}`
|
||||
|
||||
## Version Compatibility
|
||||
|
||||
| Resilience4j | Spring Boot | Java | Spring Framework |
|
||||
|--------------|-------------|------|------------------|
|
||||
| 2.2.x | 3.x | 17+ | 6.x |
|
||||
| 2.1.x | 3.x | 17+ | 6.x |
|
||||
| 2.0.x | 2.7.x | 8+ | 5.3.x |
|
||||
| 1.7.x | 2.x | 8+ | 5.x |
|
||||
|
||||
## References
|
||||
|
||||
- [Resilience4j Official Documentation](https://resilience4j.readme.io/)
|
||||
- [Spring Boot Integration Guide](https://resilience4j.readme.io/docs/getting-started-3)
|
||||
- [Micrometer Metrics Guide](https://resilience4j.readme.io/docs/micrometer)
|
||||
@@ -0,0 +1,483 @@
|
||||
# Resilience4j Real-World Examples
|
||||
|
||||
## E-Commerce Order Service
|
||||
|
||||
Complete example demonstrating all Resilience4j patterns in a microservices environment.
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
order-service/
|
||||
├── src/main/java/com/ecommerce/order/
|
||||
│ ├── config/
|
||||
│ │ ├── ResilienceConfig.java
|
||||
│ │ └── RestTemplateConfig.java
|
||||
│ ├── controller/
|
||||
│ │ ├── OrderController.java
|
||||
│ │ └── GlobalExceptionHandler.java
|
||||
│ ├── service/
|
||||
│ │ ├── OrderService.java
|
||||
│ │ ├── PaymentService.java
|
||||
│ │ ├── InventoryService.java
|
||||
│ │ └── NotificationService.java
|
||||
│ ├── domain/
|
||||
│ │ ├── Order.java
|
||||
│ │ ├── OrderStatus.java
|
||||
│ │ └── Payment.java
|
||||
└── src/main/resources/
|
||||
└── application.yml
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
```yaml
|
||||
server:
|
||||
port: 8080
|
||||
|
||||
spring:
|
||||
application:
|
||||
name: order-service
|
||||
|
||||
resilience4j:
|
||||
circuitbreaker:
|
||||
configs:
|
||||
default:
|
||||
registerHealthIndicator: true
|
||||
slidingWindowSize: 10
|
||||
minimumNumberOfCalls: 5
|
||||
failureRateThreshold: 50
|
||||
waitDurationInOpenState: 30s
|
||||
instances:
|
||||
paymentService:
|
||||
baseConfig: default
|
||||
waitDurationInOpenState: 60s
|
||||
inventoryService:
|
||||
baseConfig: default
|
||||
|
||||
retry:
|
||||
configs:
|
||||
default:
|
||||
maxAttempts: 3
|
||||
waitDuration: 500ms
|
||||
enableExponentialBackoff: true
|
||||
exponentialBackoffMultiplier: 2
|
||||
instances:
|
||||
paymentService:
|
||||
maxAttempts: 5
|
||||
waitDuration: 1s
|
||||
|
||||
ratelimiter:
|
||||
configs:
|
||||
default:
|
||||
limitForPeriod: 100
|
||||
limitRefreshPeriod: 1s
|
||||
instances:
|
||||
emailService:
|
||||
limitForPeriod: 10
|
||||
limitRefreshPeriod: 1m
|
||||
|
||||
bulkhead:
|
||||
configs:
|
||||
default:
|
||||
maxConcurrentCalls: 10
|
||||
maxWaitDuration: 100ms
|
||||
instances:
|
||||
orderProcessing:
|
||||
maxConcurrentCalls: 5
|
||||
|
||||
timelimiter:
|
||||
configs:
|
||||
default:
|
||||
timeoutDuration: 3s
|
||||
instances:
|
||||
paymentService:
|
||||
timeoutDuration: 5s
|
||||
|
||||
management:
|
||||
endpoints:
|
||||
web:
|
||||
exposure:
|
||||
include: '*'
|
||||
endpoint:
|
||||
health:
|
||||
show-details: always
|
||||
health:
|
||||
circuitbreakers:
|
||||
enabled: true
|
||||
ratelimiters:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
### Order Service Implementation
|
||||
|
||||
```java
|
||||
@Slf4j
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class OrderService {
|
||||
|
||||
private final PaymentService paymentService;
|
||||
private final InventoryService inventoryService;
|
||||
private final NotificationService notificationService;
|
||||
|
||||
@Bulkhead(name = "orderProcessing", type = Bulkhead.Type.SEMAPHORE)
|
||||
@Transactional
|
||||
public Order processOrder(OrderRequest request) {
|
||||
log.info("Processing order for customer: {}", request.getCustomerId());
|
||||
|
||||
Order order = createOrder(request);
|
||||
|
||||
try {
|
||||
// Reserve inventory
|
||||
inventoryService.reserveInventory(order);
|
||||
|
||||
// Process payment
|
||||
String paymentId = paymentService.processPayment(order).get();
|
||||
order = order.toBuilder().paymentId(paymentId).build();
|
||||
|
||||
// Send confirmation (async, best effort)
|
||||
notificationService.sendOrderConfirmation(order);
|
||||
|
||||
log.info("Order processed successfully: {}", order.getId());
|
||||
return order;
|
||||
|
||||
} catch (Exception ex) {
|
||||
log.error("Order processing failed", ex);
|
||||
compensateFailedOrder(order);
|
||||
throw new OrderProcessingException("Failed to process order", ex);
|
||||
}
|
||||
}
|
||||
|
||||
private void compensateFailedOrder(Order order) {
|
||||
try {
|
||||
inventoryService.releaseInventory(order);
|
||||
if (order.getPaymentId() != null) {
|
||||
paymentService.refundPayment(order.getPaymentId());
|
||||
}
|
||||
} catch (Exception ex) {
|
||||
log.error("Compensation failed", ex);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Payment Service with Multiple Patterns
|
||||
|
||||
```java
|
||||
@Slf4j
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class PaymentService {
|
||||
|
||||
private final PaymentClient paymentClient;
|
||||
|
||||
@CircuitBreaker(name = "paymentService", fallbackMethod = "processPaymentFallback")
|
||||
@Retry(name = "paymentService")
|
||||
@TimeLimiter(name = "paymentService")
|
||||
public CompletableFuture<String> processPayment(Order order) {
|
||||
return CompletableFuture.supplyAsync(() -> {
|
||||
Payment payment = Payment.builder()
|
||||
.orderId(order.getId())
|
||||
.amount(order.getTotalAmount())
|
||||
.build();
|
||||
|
||||
PaymentResponse response = paymentClient.processPayment(payment);
|
||||
|
||||
if (!response.isSuccess()) {
|
||||
throw new PaymentFailedException(response.getErrorMessage());
|
||||
}
|
||||
|
||||
return response.getPaymentId();
|
||||
});
|
||||
}
|
||||
|
||||
private CompletableFuture<String> processPaymentFallback(
|
||||
Order order, Exception ex) {
|
||||
log.error("Payment processing failed for order: {}", order.getId(), ex);
|
||||
throw new PaymentServiceUnavailableException(
|
||||
"Payment service unavailable", ex);
|
||||
}
|
||||
|
||||
@CircuitBreaker(name = "paymentService")
|
||||
@Retry(name = "paymentService")
|
||||
public void refundPayment(String paymentId) {
|
||||
paymentClient.refundPayment(paymentId);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Exception Handler
|
||||
|
||||
```java
|
||||
@Slf4j
|
||||
@RestControllerAdvice
|
||||
public class GlobalExceptionHandler {
|
||||
|
||||
@ExceptionHandler(CallNotPermittedException.class)
|
||||
@ResponseStatus(HttpStatus.SERVICE_UNAVAILABLE)
|
||||
public ErrorResponse handleCircuitOpen(CallNotPermittedException ex) {
|
||||
log.error("Circuit breaker is open", ex);
|
||||
return ErrorResponse.builder()
|
||||
.code("SERVICE_UNAVAILABLE")
|
||||
.message("Service is temporarily unavailable")
|
||||
.status(HttpStatus.SERVICE_UNAVAILABLE.value())
|
||||
.build();
|
||||
}
|
||||
|
||||
@ExceptionHandler(RequestNotPermitted.class)
|
||||
@ResponseStatus(HttpStatus.TOO_MANY_REQUESTS)
|
||||
public ErrorResponse handleRateLimited(RequestNotPermitted ex) {
|
||||
return ErrorResponse.builder()
|
||||
.code("TOO_MANY_REQUESTS")
|
||||
.message("Rate limit exceeded")
|
||||
.status(HttpStatus.TOO_MANY_REQUESTS.value())
|
||||
.build();
|
||||
}
|
||||
|
||||
@ExceptionHandler(BulkheadFullException.class)
|
||||
@ResponseStatus(HttpStatus.SERVICE_UNAVAILABLE)
|
||||
public ErrorResponse handleBulkheadFull(BulkheadFullException ex) {
|
||||
return ErrorResponse.builder()
|
||||
.code("SERVICE_BUSY")
|
||||
.message("Service at capacity")
|
||||
.status(HttpStatus.SERVICE_UNAVAILABLE.value())
|
||||
.build();
|
||||
}
|
||||
|
||||
@ExceptionHandler(TimeoutException.class)
|
||||
@ResponseStatus(HttpStatus.REQUEST_TIMEOUT)
|
||||
public ErrorResponse handleTimeout(TimeoutException ex) {
|
||||
return ErrorResponse.builder()
|
||||
.code("REQUEST_TIMEOUT")
|
||||
.message("Request timed out")
|
||||
.status(HttpStatus.REQUEST_TIMEOUT.value())
|
||||
.build();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Testing Patterns
|
||||
|
||||
### Unit Test for Circuit Breaker
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
class PaymentServiceCircuitBreakerTest {
|
||||
|
||||
@Autowired
|
||||
private PaymentService paymentService;
|
||||
|
||||
@Autowired
|
||||
private CircuitBreakerRegistry circuitBreakerRegistry;
|
||||
|
||||
@MockBean
|
||||
private PaymentClient paymentClient;
|
||||
|
||||
private CircuitBreaker circuitBreaker;
|
||||
|
||||
@BeforeEach
|
||||
void setup() {
|
||||
circuitBreaker = circuitBreakerRegistry.circuitBreaker("paymentService");
|
||||
circuitBreaker.reset();
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldOpenCircuitAfterFailures() {
|
||||
Order order = createTestOrder();
|
||||
when(paymentClient.processPayment(any()))
|
||||
.thenThrow(new RuntimeException("Service error"));
|
||||
|
||||
// Trigger failures to exceed threshold
|
||||
for (int i = 0; i < 5; i++) {
|
||||
try {
|
||||
paymentService.processPayment(order).get();
|
||||
} catch (Exception ignored) {}
|
||||
}
|
||||
|
||||
assertThat(circuitBreaker.getState())
|
||||
.isEqualTo(CircuitBreaker.State.OPEN);
|
||||
|
||||
// Next call should fail immediately
|
||||
assertThatThrownBy(() -> paymentService.processPayment(order).get())
|
||||
.hasRootCauseInstanceOf(PaymentServiceUnavailableException.class);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Integration Test with WireMock
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
@AutoConfigureWireMock(port = 0)
|
||||
class OrderServiceIntegrationTest {
|
||||
|
||||
@Autowired
|
||||
private OrderService orderService;
|
||||
|
||||
@Test
|
||||
void shouldRetryOnTransientFailure() {
|
||||
// First two calls fail, third succeeds
|
||||
stubFor(post("/payment/process")
|
||||
.inScenario("Retry")
|
||||
.whenScenarioStateIs(STARTED)
|
||||
.willReturn(serverError())
|
||||
.willSetStateTo("First Retry"));
|
||||
|
||||
stubFor(post("/payment/process")
|
||||
.inScenario("Retry")
|
||||
.whenScenarioStateIs("First Retry")
|
||||
.willReturn(serverError())
|
||||
.willSetStateTo("Second Retry"));
|
||||
|
||||
stubFor(post("/payment/process")
|
||||
.inScenario("Retry")
|
||||
.whenScenarioStateIs("Second Retry")
|
||||
.willReturn(ok().withBody("{\"paymentId\":\"PAY-123\"}")));
|
||||
|
||||
Order order = orderService.processOrder(createOrderRequest());
|
||||
|
||||
assertThat(order.getPaymentId()).isEqualTo("PAY-123");
|
||||
verify(exactly(3), postRequestedFor(urlEqualTo("/payment/process")));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Advanced Scenarios
|
||||
|
||||
### Reactive WebFlux Example
|
||||
|
||||
```java
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
public class ReactiveProductService {
|
||||
|
||||
private final WebClient webClient;
|
||||
private final CircuitBreaker circuitBreaker;
|
||||
private final Retry retry;
|
||||
|
||||
public Mono<Product> getProduct(String productId) {
|
||||
return webClient.get()
|
||||
.uri("/products/{id}", productId)
|
||||
.retrieve()
|
||||
.bodyToMono(Product.class)
|
||||
.transformDeferred(CircuitBreakerOperator.of(circuitBreaker))
|
||||
.transformDeferred(RetryOperator.of(retry))
|
||||
.onErrorResume(throwable ->
|
||||
Mono.just(Product.unavailable(productId))
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Resilience Configuration
|
||||
|
||||
```java
|
||||
@Configuration
|
||||
@Slf4j
|
||||
public class ResilienceConfig {
|
||||
|
||||
@Bean
|
||||
public CircuitBreakerRegistry circuitBreakerRegistry() {
|
||||
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
|
||||
.failureRateThreshold(50)
|
||||
.waitDurationInOpenState(Duration.ofSeconds(30))
|
||||
.slowCallDurationThreshold(Duration.ofSeconds(2))
|
||||
.permittedNumberOfCallsInHalfOpenState(3)
|
||||
.minimumNumberOfCalls(5)
|
||||
.slidingWindowSize(10)
|
||||
.build();
|
||||
|
||||
CircuitBreakerRegistry registry = CircuitBreakerRegistry.of(config);
|
||||
|
||||
// Register event consumer
|
||||
registry.getEventPublisher()
|
||||
.onEntryAdded(event ->
|
||||
log.info("CircuitBreaker added: {}",
|
||||
event.getAddedEntry().getName())
|
||||
);
|
||||
|
||||
return registry;
|
||||
}
|
||||
|
||||
@Bean
|
||||
public RegistryEventConsumer<CircuitBreaker> circuitBreakerEventConsumer() {
|
||||
return new RegistryEventConsumer<>() {
|
||||
@Override
|
||||
public void onEntryAddedEvent(EntryAddedEvent<CircuitBreaker> event) {
|
||||
CircuitBreaker cb = event.getAddedEntry();
|
||||
cb.getEventPublisher()
|
||||
.onStateTransition(e ->
|
||||
log.warn("CircuitBreaker {} state changed: {} -> {}",
|
||||
cb.getName(),
|
||||
e.getStateTransition().getFromState(),
|
||||
e.getStateTransition().getToState())
|
||||
)
|
||||
.onError(e ->
|
||||
log.error("CircuitBreaker {} error: {}",
|
||||
cb.getName(),
|
||||
e.getThrowable().getMessage())
|
||||
);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onEntryRemovedEvent(EntryRemovedEvent<CircuitBreaker> event) {
|
||||
log.info("CircuitBreaker removed: {}",
|
||||
event.getRemovedEntry().getName());
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onEntryReplacedEvent(EntryReplacedEvent<CircuitBreaker> event) {
|
||||
log.info("CircuitBreaker replaced: {}",
|
||||
event.getNewEntry().getName());
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Monitoring and Metrics
|
||||
|
||||
```java
|
||||
@RestController
|
||||
@RequestMapping("/api/monitoring")
|
||||
@RequiredArgsConstructor
|
||||
public class ResilienceMonitoringController {
|
||||
|
||||
private final CircuitBreakerRegistry circuitBreakerRegistry;
|
||||
|
||||
@GetMapping("/circuit-breakers")
|
||||
public List<CircuitBreakerStatus> getStatus() {
|
||||
return circuitBreakerRegistry.getAllCircuitBreakers().stream()
|
||||
.map(this::toStatus)
|
||||
.collect(Collectors.toList());
|
||||
}
|
||||
|
||||
private CircuitBreakerStatus toStatus(CircuitBreaker cb) {
|
||||
CircuitBreaker.Metrics metrics = cb.getMetrics();
|
||||
|
||||
return CircuitBreakerStatus.builder()
|
||||
.name(cb.getName())
|
||||
.state(cb.getState().name())
|
||||
.failureRate(metrics.getFailureRate())
|
||||
.slowCallRate(metrics.getSlowCallRate())
|
||||
.numberOfBufferedCalls(metrics.getNumberOfBufferedCalls())
|
||||
.numberOfFailedCalls(metrics.getNumberOfFailedCalls())
|
||||
.numberOfSuccessfulCalls(metrics.getNumberOfSuccessfulCalls())
|
||||
.build();
|
||||
}
|
||||
}
|
||||
|
||||
@Value
|
||||
@Builder
|
||||
class CircuitBreakerStatus {
|
||||
String name;
|
||||
String state;
|
||||
float failureRate;
|
||||
float slowCallRate;
|
||||
int numberOfBufferedCalls;
|
||||
int numberOfFailedCalls;
|
||||
int numberOfSuccessfulCalls;
|
||||
}
|
||||
```
|
||||
|
||||
See testing-patterns.md for comprehensive testing strategies and configuration-reference.md for complete configuration options.
|
||||
@@ -0,0 +1,491 @@
|
||||
# Resilience4j Testing Patterns
|
||||
|
||||
## Circuit Breaker Testing
|
||||
|
||||
### Testing State Transitions
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
class CircuitBreakerStateTest {
|
||||
|
||||
@Autowired
|
||||
private PaymentService paymentService;
|
||||
|
||||
@MockBean
|
||||
private RestTemplate restTemplate;
|
||||
|
||||
@Test
|
||||
void shouldTransitionToOpenAfterFailures() {
|
||||
// Simulate repeated failures
|
||||
when(restTemplate.postForObject(anyString(), any(), eq(PaymentResponse.class)))
|
||||
.thenThrow(new HttpServerErrorException(HttpStatus.INTERNAL_SERVER_ERROR));
|
||||
|
||||
// Trigger failures to exceed threshold
|
||||
for (int i = 0; i < 5; i++) {
|
||||
assertThatThrownBy(() -> paymentService.processPayment(new PaymentRequest()))
|
||||
.isInstanceOf(HttpServerErrorException.class);
|
||||
}
|
||||
|
||||
// Circuit should be open - fallback executes
|
||||
PaymentResponse response = paymentService.processPayment(new PaymentRequest());
|
||||
assertThat(response.getStatus()).isEqualTo("PENDING");
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldExecuteFallbackWhenCircuitOpen() {
|
||||
when(restTemplate.postForObject(anyString(), any(), eq(PaymentResponse.class)))
|
||||
.thenThrow(new RuntimeException("Service unavailable"));
|
||||
|
||||
// Force failures to open circuit
|
||||
for (int i = 0; i < 5; i++) {
|
||||
try {
|
||||
paymentService.processPayment(new PaymentRequest());
|
||||
} catch (Exception ignored) {}
|
||||
}
|
||||
|
||||
// Circuit is open, fallback provides response
|
||||
PaymentResponse response = paymentService.processPayment(new PaymentRequest());
|
||||
assertThat(response.getStatus()).isEqualTo("PENDING");
|
||||
assertThat(response.getMessage()).contains("temporarily unavailable");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Testing Circuit States Directly
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
class CircuitBreakerDirectStateTest {
|
||||
|
||||
@Autowired
|
||||
private CircuitBreakerRegistry circuitBreakerRegistry;
|
||||
|
||||
@Test
|
||||
void shouldManuallyOpenAndCloseCircuit() {
|
||||
CircuitBreaker circuitBreaker = circuitBreakerRegistry.circuitBreaker("paymentService");
|
||||
|
||||
assertThat(circuitBreaker.getState()).isEqualTo(CircuitBreaker.State.CLOSED);
|
||||
|
||||
// Manually open circuit
|
||||
circuitBreaker.transitionToOpenState();
|
||||
assertThat(circuitBreaker.getState()).isEqualTo(CircuitBreaker.State.OPEN);
|
||||
|
||||
// Manually close circuit
|
||||
circuitBreaker.transitionToClosedState();
|
||||
assertThat(circuitBreaker.getState()).isEqualTo(CircuitBreaker.State.CLOSED);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Retry Testing
|
||||
|
||||
### Testing Retry Attempts
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
@AutoConfigureWireMock(port = 0)
|
||||
class RetryTest {
|
||||
|
||||
@Autowired
|
||||
private OrderService orderService;
|
||||
|
||||
@Test
|
||||
void shouldRetryOnTransientFailure() {
|
||||
// Setup: First two calls fail, third succeeds
|
||||
stubFor(post("/orders")
|
||||
.inScenario("Retry Scenario")
|
||||
.whenScenarioStateIs(STARTED)
|
||||
.willReturn(serverError())
|
||||
.willSetStateTo("First Failure"));
|
||||
|
||||
stubFor(post("/orders")
|
||||
.inScenario("Retry Scenario")
|
||||
.whenScenarioStateIs("First Failure")
|
||||
.willReturn(serverError())
|
||||
.willSetStateTo("Second Failure"));
|
||||
|
||||
stubFor(post("/orders")
|
||||
.inScenario("Retry Scenario")
|
||||
.whenScenarioStateIs("Second Failure")
|
||||
.willReturn(ok().withBody("""
|
||||
{"id":1,"status":"CREATED"}
|
||||
""")));
|
||||
|
||||
Order order = orderService.createOrder(new OrderRequest());
|
||||
|
||||
assertThat(order.getId()).isEqualTo(1L);
|
||||
assertThat(order.getStatus()).isEqualTo("CREATED");
|
||||
|
||||
// Verify exactly 3 calls were made
|
||||
verify(exactly(3), postRequestedFor(urlEqualTo("/orders")));
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldThrowExceptionAfterMaxRetries() {
|
||||
stubFor(post("/orders").willReturn(serverError()));
|
||||
|
||||
assertThatThrownBy(() -> orderService.createOrder(new OrderRequest()))
|
||||
.isInstanceOf(Exception.class);
|
||||
|
||||
// Verify retry attempts (maxAttempts = 3)
|
||||
verify(atLeast(3), postRequestedFor(urlEqualTo("/orders")));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Rate Limiter Testing
|
||||
|
||||
### Testing Rate Limit Enforcement
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
class RateLimiterTest {
|
||||
|
||||
@Autowired
|
||||
private NotificationService notificationService;
|
||||
|
||||
@Test
|
||||
void shouldRejectRequestsExceedingRateLimit() {
|
||||
// Configuration: 5 permits per second
|
||||
|
||||
// First 5 requests should succeed
|
||||
for (int i = 0; i < 5; i++) {
|
||||
notificationService.sendEmail(createEmailRequest(i));
|
||||
}
|
||||
|
||||
// 6th request should fail immediately (no timeout)
|
||||
assertThatThrownBy(() ->
|
||||
notificationService.sendEmail(createEmailRequest(6))
|
||||
).isInstanceOf(RequestNotPermitted.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldAlowRequestsAfterWindowReset() throws InterruptedException {
|
||||
// First batch of requests
|
||||
for (int i = 0; i < 5; i++) {
|
||||
notificationService.sendEmail(createEmailRequest(i));
|
||||
}
|
||||
|
||||
// Wait for refresh period (1 second)
|
||||
Thread.sleep(1100);
|
||||
|
||||
// Should succeed - window has reset
|
||||
notificationService.sendEmail(createEmailRequest(5));
|
||||
}
|
||||
|
||||
private EmailRequest createEmailRequest(int id) {
|
||||
return EmailRequest.builder()
|
||||
.to("user" + id + "@example.com")
|
||||
.subject("Test " + id)
|
||||
.build();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Bulkhead Testing
|
||||
|
||||
### Testing Semaphore Bulkhead
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
class BulkheadSemaphoreTest {
|
||||
|
||||
@Autowired
|
||||
private ReportService reportService;
|
||||
|
||||
@Test
|
||||
void shouldLimitConcurrentCalls() {
|
||||
// Configuration: maxConcurrentCalls = 5
|
||||
|
||||
CountDownLatch latch = new CountDownLatch(5);
|
||||
List<CompletableFuture<Report>> futures = new ArrayList<>();
|
||||
|
||||
// Submit 5 concurrent calls
|
||||
for (int i = 0; i < 5; i++) {
|
||||
futures.add(CompletableFuture.supplyAsync(() -> {
|
||||
latch.countDown();
|
||||
return reportService.generateReport(new ReportRequest());
|
||||
}));
|
||||
}
|
||||
|
||||
// 6th call should be rejected
|
||||
assertThatThrownBy(() ->
|
||||
reportService.generateReport(new ReportRequest())
|
||||
).isInstanceOf(BulkheadFullException.class);
|
||||
|
||||
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Testing Thread Pool Bulkhead
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
class BulkheadThreadPoolTest {
|
||||
|
||||
@Autowired
|
||||
private AnalyticsService analyticsService;
|
||||
|
||||
@Test
|
||||
void shouldUseThreadPoolForAsync() {
|
||||
// Configuration: threadPoolSize = 2, queueCapacity = 100
|
||||
|
||||
List<CompletableFuture<AnalyticsResult>> futures = new ArrayList<>();
|
||||
|
||||
for (int i = 0; i < 10; i++) {
|
||||
futures.add(analyticsService.runAnalytics(new AnalyticsRequest()));
|
||||
}
|
||||
|
||||
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
|
||||
|
||||
for (CompletableFuture<AnalyticsResult> future : futures) {
|
||||
assertThat(future.join()).isNotNull();
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Time Limiter Testing
|
||||
|
||||
### Testing Timeout Enforcement
|
||||
|
||||
```java
|
||||
@SpringBootTest
|
||||
class TimeLimiterTest {
|
||||
|
||||
@Autowired
|
||||
private SearchService searchService;
|
||||
|
||||
@Test
|
||||
void shouldTimeoutExceededOperations() {
|
||||
// Configuration: timeoutDuration = 1s
|
||||
|
||||
SearchQuery slowQuery = new SearchQuery();
|
||||
slowQuery.setSimulatedDelay(Duration.ofSeconds(2));
|
||||
|
||||
assertThatThrownBy(() ->
|
||||
searchService.search(slowQuery).get()
|
||||
).hasCauseInstanceOf(TimeoutException.class);
|
||||
}
|
||||
|
||||
@Test
|
||||
void shouldReturnFallbackOnTimeout() {
|
||||
SearchQuery slowQuery = new SearchQuery();
|
||||
slowQuery.setSimulatedDelay(Duration.ofSeconds(2));
|
||||
|
||||
CompletableFuture<SearchResults> result = searchService.search(slowQuery);
|
||||
SearchResults results = result.join();
|
||||
|
||||
assertThat(results).isNotNull();
|
||||
assertThat(results.isTimedOut()).isTrue();
|
||||
assertThat(results.getMessage()).contains("timed out");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Fallback Method Signature Validation
|
||||
|
||||
### Correct Fallback Signatures
|
||||
|
||||
```java
|
||||
@Service
|
||||
public class PaymentService {
|
||||
|
||||
@CircuitBreaker(name = "payment", fallbackMethod = "paymentFallback")
|
||||
public PaymentResponse processPayment(PaymentRequest request) {
|
||||
// method body
|
||||
}
|
||||
|
||||
// CORRECT: Matches return type and parameters + Exception
|
||||
private PaymentResponse paymentFallback(PaymentRequest request, Exception ex) {
|
||||
// fallback logic
|
||||
}
|
||||
|
||||
@Retry(name = "product")
|
||||
public Product getProduct(String productId) {
|
||||
// method body
|
||||
}
|
||||
|
||||
// CORRECT: Can omit Exception parameter
|
||||
private Product getProductFallback(String productId) {
|
||||
// fallback logic
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Common Fallback Signature Errors
|
||||
|
||||
```java
|
||||
@CircuitBreaker(name = "service", fallbackMethod = "fallback")
|
||||
public String processData(Long id) { }
|
||||
|
||||
// WRONG: Missing parameter
|
||||
public String fallback(Exception ex) { }
|
||||
|
||||
// WRONG: Wrong return type
|
||||
public void fallback(Long id, Exception ex) { }
|
||||
|
||||
// WRONG: Wrong parameter type
|
||||
public String fallback(String id, Exception ex) { }
|
||||
|
||||
// CORRECT:
|
||||
public String fallback(Long id, Exception ex) { }
|
||||
```
|
||||
|
||||
## Integration Testing Configuration
|
||||
|
||||
### Test Configuration Profile
|
||||
|
||||
```yaml
|
||||
# application-test.yml
|
||||
resilience4j:
|
||||
circuitbreaker:
|
||||
instances:
|
||||
testService:
|
||||
registerHealthIndicator: false
|
||||
slidingWindowSize: 5
|
||||
minimumNumberOfCalls: 3
|
||||
failureRateThreshold: 50
|
||||
waitDurationInOpenState: 100ms
|
||||
|
||||
retry:
|
||||
instances:
|
||||
testService:
|
||||
maxAttempts: 2
|
||||
waitDuration: 10ms
|
||||
|
||||
ratelimiter:
|
||||
instances:
|
||||
testService:
|
||||
limitForPeriod: 10
|
||||
limitRefreshPeriod: 1s
|
||||
timeoutDuration: 10ms
|
||||
```
|
||||
|
||||
### Test Helper Methods
|
||||
|
||||
```java
|
||||
@TestConfiguration
|
||||
public class ResilienceTestConfig {
|
||||
|
||||
public static void openCircuitBreaker(CircuitBreaker circuitBreaker) {
|
||||
circuitBreaker.transitionToOpenState();
|
||||
}
|
||||
|
||||
public static void closeCircuitBreaker(CircuitBreaker circuitBreaker) {
|
||||
circuitBreaker.transitionToClosedState();
|
||||
}
|
||||
|
||||
public static void simulateFailures(
|
||||
CircuitBreaker circuitBreaker,
|
||||
int numberOfFailures) {
|
||||
for (int i = 0; i < numberOfFailures; i++) {
|
||||
try {
|
||||
circuitBreaker.executeSupplier(() -> {
|
||||
throw new RuntimeException("Simulated failure");
|
||||
});
|
||||
} catch (Exception ignored) {}
|
||||
}
|
||||
}
|
||||
|
||||
public static void resetCircuitBreaker(CircuitBreaker circuitBreaker) {
|
||||
circuitBreaker.transitionToClosedState();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Common Testing Mistakes
|
||||
|
||||
### Mistake 1: Not Waiting for Sliding Window
|
||||
|
||||
```java
|
||||
// WRONG: Circuit might not open yet
|
||||
for (int i = 0; i < 3; i++) {
|
||||
try { service.call(); } catch (Exception e) {}
|
||||
}
|
||||
assertThat(circuit.getState()).isEqualTo(CircuitBreaker.State.OPEN); // May fail!
|
||||
|
||||
// CORRECT: Exceed minimumNumberOfCalls before checking
|
||||
for (int i = 0; i < 5; i++) { // minimumNumberOfCalls = 5
|
||||
try { service.call(); } catch (Exception e) {}
|
||||
}
|
||||
assertThat(circuit.getState()).isEqualTo(CircuitBreaker.State.OPEN);
|
||||
```
|
||||
|
||||
### Mistake 2: Incorrect Fallback Method Access
|
||||
|
||||
```java
|
||||
// WRONG: Fallback method is private, not accessible by AOP
|
||||
@CircuitBreaker(name = "service", fallbackMethod = "fallback")
|
||||
public String process(String data) { }
|
||||
|
||||
private String fallback(String data, Exception ex) { } // Private - won't work!
|
||||
|
||||
// CORRECT: Package-private or protected
|
||||
protected String fallback(String data, Exception ex) { }
|
||||
```
|
||||
|
||||
### Mistake 3: Not Mocking External Dependencies
|
||||
|
||||
```java
|
||||
// WRONG: Circuit breaker might open due to real network calls
|
||||
@SpringBootTest
|
||||
class ServiceTest {
|
||||
@Autowired
|
||||
private ServiceWithCircuitBreaker service;
|
||||
|
||||
// Missing @MockBean for external service
|
||||
|
||||
@Test
|
||||
void test() {
|
||||
// Real network calls - unpredictable
|
||||
}
|
||||
}
|
||||
|
||||
// CORRECT: Mock external dependencies
|
||||
@SpringBootTest
|
||||
class ServiceTest {
|
||||
@Autowired
|
||||
private ServiceWithCircuitBreaker service;
|
||||
|
||||
@MockBean
|
||||
private ExternalService externalService;
|
||||
|
||||
@Test
|
||||
void test() {
|
||||
when(externalService.call()).thenThrow(new RuntimeException());
|
||||
// Predictable failure
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Memory Usage in Tests
|
||||
|
||||
- **COUNT_BASED sliding window**: Stores last N call outcomes in memory
|
||||
- **TIME_BASED sliding window**: May require more memory for high-throughput services
|
||||
- Use smaller `slidingWindowSize` in tests to reduce memory footprint
|
||||
|
||||
### Timeout Configuration for Tests
|
||||
|
||||
```yaml
|
||||
resilience4j:
|
||||
timelimiter:
|
||||
instances:
|
||||
testService:
|
||||
timeoutDuration: 2s # Longer timeout for slower CI/CD environments
|
||||
|
||||
circuitbreaker:
|
||||
instances:
|
||||
testService:
|
||||
waitDurationInOpenState: 100ms # Shorter for faster test execution
|
||||
```
|
||||
|
||||
### Avoiding Test Flakiness
|
||||
|
||||
- Set deterministic timeouts based on CI/CD environment
|
||||
- Use `@ActiveProfiles("test")` for test-specific configurations
|
||||
- Reset circuit breaker state between tests when needed
|
||||
- Mock external services consistently
|
||||
Reference in New Issue
Block a user