Spring Boot and Caffeine Cache with Microservices
Caching is a very important part of a system, especially when we work with microservices. A cache is a storage layer between the application and a data source, such as a database or any other service.
So, by keeping frequently accessed data in a cache, we can have major advantages such as improving performance, giving a better user experience, reducing network load, increasing scalability (by reducing the hit rate of original data sources), et cetera.
Depending on your requirements, you can use different types of caching mechanisms for the application.
- In-memory cache
- Database caching
- Disk caching
- DNS caching
- CDN caching
So, in this article, the focus is on the caffeine cache, which is one of the in-memory caching mechanisms.
Caffeine Cache
Caffeine is a high-performance Java caching library that provides a near-optimal hit rate, as mentioned in its page. A cache can be configured to evict stored items, while a Map keeps all the items that are stored. And also, caffeine caches can be initiated and configured to match different use cases. Automatic eviction, automatic loading of items, and automatic refreshing mechanisms are some major functionalities that are very useful.
We can learn more about caffeine cache and its functionalities by referring to its documentation here. (https://github.com/ben-manes/caffeine/wiki)
Implementation
So, from this article, let’s see how we can implement a caching mechanism, especially with microservices.
For explanation, let’s have two microservices named Order-Service and Product-Service. Also, assume that, product-service has product details and order-service needs product details for client requests.
Let’s create the order-service with one endpoint to request an order for a given product id, while creating the product-service with two endpoints that will return the product details for the given product id and a list of products. And these two services are going to communicate with each other over HTTP calls.
Dependencies
Here, I’m going to use reactive web applications with Spring WebFlux and Netty.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>And caffeine
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
<version>3.1.6</version>
</dependency>Product-Service
As mentioned, let’s have a controller and two simple endpoints to have product details in product-service.
@RestController
@RequestMapping("/product-service")
public class ProductController {
@GetMapping(value = "/product/{id}", produces = MediaType.APPLICATION_JSON_VALUE)
public Mono<String> requestProduct(@PathVariable String id) {
return id.equals("6") ? Mono.error(new ResponseStatusException(HttpStatus.NOT_FOUND, "Product is not found")) : Mono.just("Product " + id);
}
@GetMapping(value = "/products",produces = MediaType.APPLICATION_JSON_VALUE)
public Mono<Map<String,String>> getProducts() {
Map<String, String> productMap = new HashMap();
productMap.put("1", "Product 1");
productMap.put("2", "Product 2");
productMap.put("3", "Product 3");
productMap.put("4", "Product 4");
productMap.put("5", "Product 5");
return Mono.just(productMap);
}
}Order-Service
Caching functionalities are going to be implemented in this service.
As a first step, let’s create a bean for WebClient.
@Bean
public WebClient getWebClient(){
return WebClient.builder().build();
}A simple way to understand first
Let’s create a Controller and a Service interface to process client requests for a given ID.
Controller
@AllArgsConstructor
@RestController
@RequestMapping("/order-service")
public class OrderController {
private final OrderService orderService;
@GetMapping(value = "/order/first-way/{id}", produces = MediaType.APPLICATION_JSON_VALUE)
public Mono<String> requestOrderFirstWay(@PathVariable String id) {
return orderService.requestOrderFirstWay(id);
}
}Service
public interface OrderService {
Mono<String> requestOrderFirstWay(String id);
}Service-Implementation
Then, WebClient should be injected into the service implementation, as we are going to communicate between two services through HTTP.
Then, we can initiate Caffeine Cache as a member variable of this implementation. In this example, I’m going to initiate the cache with time-base eviction, and the field type for both key and value is going to be string. (It provides three types of eviction: size-based eviction, time-based eviction, and reference-based eviction. It can be configured depending on our use case and the original data source. So, if the data source is being updated frequently, we can give a shorter expiration time or implement a way to update the cache whenever data is updated.)
@Slf4j
@Service
@AllArgsConstructor
public class OrderServiceImpl implements OrderService {
private final WebClient webClient;
private final Cache<String, String> productCache = Caffeine.newBuilder()
.expireAfterAccess(Duration.ofSeconds(600L))
.maximumSize(10000L)
.evictionListener((key, item, cause) -> log.info(this.getClass().getName().concat(": key {} was evicted "), key))
.removalListener((key, item, cause) -> log.info(this.getClass().getName().concat(": Key {} was removed "), key))
.scheduler(Scheduler.systemScheduler())
.build();
}First Thinking: Handle in the method
Now, we can implement the request-order method with the cache. As it is elaborated in the [1] diagram, we need to have product details from product-service in the order-service. When a request comes to the application, it first checks the cache. If matching data is already in the cache, it will be returned from the cache without any HTTP calls. If matching data is not in the cache, it will be retrieved from the product-service through an HTTP request and stored in the cache for future use. Let’s see how this method can be implemented.
@Override
public Mono<String> requestOrderFirstWay(String id) {
String productName = productCache.getIfPresent(id);
return Mono.justOrEmpty(productName != null ? "Order requested on : " + productName : null)
.switchIfEmpty(
webClient.get()
.uri("http://localhost:8081/product-service/product/" + id) // an endpoint the product-service
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.accept(MediaType.APPLICATION_JSON)
.exchangeToMono(clientResponse -> {
if (clientResponse.statusCode().is2xxSuccessful()) {
log.info("success from product-service for id {}", id);
return clientResponse.bodyToMono(String.class)
.doOnNext(product -> productCache.put(id, product)) // add into the cache
.map(product -> "Order requested on : " + product);
} else if (clientResponse.statusCode().is4xxClientError()) {
log.error("error from product-service for id {}", id);
return Mono.error(new ResponseStatusException(HttpStatus.NOT_FOUND, "Product is not available"));
} else {
log.error("error from product-service for id {}", id);
return Mono.error(new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR, HttpStatus.INTERNAL_SERVER_ERROR.getReasonPhrase()));
}
})
.switchIfEmpty(Mono.just("Product is not available"))
);
}As you can see here, it is handled if the product or service returns a not-found error, a success, or any other error as well.
Note: For these scenarios, the system architecture of microservices should be handled well for fault tolerance and stability of services.
Second Thinking: Add an initial data fetch for the cache.
It is very similar to the above but just needs to add a post-constructor. (retry mechanism can also be added, like below).
@PostConstruct
private void init() {
webClient.get()
.uri("http://localhost:8081/product-service/products")
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.accept(MediaType.APPLICATION_JSON)
.exchangeToMono(response -> {
if (response.statusCode().is2xxSuccessful()) {
log.info("success from product-service for products");
return response.bodyToMono(HashMap.class)
.doOnNext(productMap -> productMap.forEach((k, v) -> productCache.put(k.toString(), v.toString())));
} else {
log.error("error from product-service for products : code {}", response.statusCode());
return Mono.error(new ResponseStatusException(HttpStatus.NOT_FOUND, "Products are not available"));
}
})
.retryWhen(Retry.backoff(3, Duration.ofSeconds(5L)))
.subscribe();
}Third Thinking: How to do this in a generic way
We can think of this in different ways. The simple way is to implement a generic cache complement that can be injected wherever we want.
@Component
@Slf4j
public class CacheManager<K,V> {
private final Cache<K, V> cache = Caffeine.newBuilder()
.expireAfterAccess(Duration.ofSeconds(600L))
.maximumSize(10000L)
.evictionListener((key, item, cause) -> log.info(this.getClass().getName().concat(": key {} was evicted "), key))
.removalListener((key, item, cause) -> log.info(this.getClass().getName().concat(": Key {} was removed "), key))
.scheduler(Scheduler.systemScheduler())
.build();
public Mono<V> get(K key, Mono<V> handler) {
return Mono.justOrEmpty(this.cache.getIfPresent(key))
.switchIfEmpty(Mono.defer(() -> handler.flatMap(it -> this.put(key, it))));
}
public Mono<V> put(K key, V object) {
this.cache.put(key, object);
return Mono.just(object);
}
public void remove(K key) {
this.cache.invalidate(key);
}
}Simple magic is included in the “get(K key, Mono<V> handler)” method of this component. The second parameter of this method is a publisher. It will check the data in the cache for a given key. If matching data is not in the cache, it will subscribe to the publisher and retrieve the data. Therefore, we just need to pass the correct publisher when this component is used.
Then this component can be injected into the service implementation, and you just need to implement fetching data logic like in previous steps.
private final CacheManager<String, String> productCacheManager;
@Override
public Mono<String> requestOrderGenericWay(String id) {
Mono<String> productResponseMono = webClient.get()
.uri("http://localhost:8081/product-service/product/" + id)
.header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(HttpStatus::is4xxClientError, clientResponse -> Mono.error(new ResponseStatusException(HttpStatus.NOT_FOUND, "Product is not available")))
.bodyToMono(String.class);
return this.productCacheManager.get(id, productResponseMono).map(product -> "Order requested on : " + product);
}I hope you gain some knowledge from this article.
You can see the source code here. https://github.com/yasas1/cache-demo
You can see my previous articles here.https://medium.com/@ranawaka.y
