Redis Cache - A String story

In one of my past projects, we were working on a project that made extensive use of Redis. This is a distributed cache system used as an alternative to Hazelcast. There are always several disputes between the usage of both among developers, but I'm not going to go there in this post. Where I am going to go is the solution to a problem I found today in less than 30 minutes, which at the time cost so much time and we ended up not really developing the best solution.

In my current project buy-odd-yucca-concert, I'm now using Micronaut in combination with Redis in order do develop a simple pub-sub system to provide listeners for the fulfilling of several segments of a concert reservation. One of those segments is a TicketDTO:

data class TicketDto(
 val name: String,
 val address: String,
 val birthDate: LocalDate,
 val concertDays: List<ConcertDayDto> = emptyList(),
 val meals: List<MealDto> = emptyList(),
 val drinks: List<DrinkDto> = emptyList(),
 val parkingReservation: ParkingReservationDto? = null,
 val createdAt: LocalDateTime? = LocalDateTime.now(),
) : Serializable

The details of the goal of this ticket are pretty much irrelevant at this point. The only thing we are interested in known is how to put in in a Redis cache, or in my case, publish this to a Pub-Sub system.

In the past the discussion started out from a very strange assumption that Redis "does not support anything other than Strings" in the cache. One quick look at their cache info page suggest the complete opposite: https://redis.com/solutions/use-cases/caching/. And quoting what they say there, "Redis is designed around the concept of data-structures and can store your dataset across Strings, Hashes, Sorted Sets, Sets, Lists, Streams, and other data structures or Redis modules". Unfortunately not only was this ignored, but the notion that Redis only supported Strings gained steam when going head first into the code, we saw that there was only a StringCodec, but not really an ObjectCodec or anything related. Someone then came up with the idea of using lombok's @ToString and then cache this string value and use it. I personally was never too happy with this, but hey, it worked and the code is now running! Good right? 😁... Not today in 2022 though.

Flash forward and in current days reaching this part of my project brought me back the memories of that discussion. To avoid this kind of discussions, that only waste time and monbey, in other developments I'm sharing with you the possibility of actually serialising whatever you want into the Redis cache, pub-sub systems or anything else in the Redis ecosystem. In my project I'm using lettuce, which is one of the most common Redis clients out there. Here is the dependency:

<dependency>
 <groupId>io.micronaut.redis</groupId>
 <artifactId>micronaut-redis-lettuce</artifactId>
 <version>5.2.0</version>
</dependency>

It is a micronaut dependency but it contains all needed lettuce dependencies.

Digging into this library, I found out that next to StringCodec there is this ByteArrayCodec, which should allow for some sort of leverage to achieve our goals. After a huge amount of time spent in discovering and re-descovering how buffering, serialising, and streaming works I finally came up with the following in Kotlin:

class TicketCodec : RedisCodec<String, TicketDto> {

 override fun decodeKey(byteBuffer: ByteBuffer): String {
 return defaultCharset().decode(byteBuffer).toString()
 }

 override fun decodeValue(byteBuffer: ByteBuffer): TicketDto =
 ObjectInputStream(
 ByteArrayInputStream(byteArrayCodec.decodeValue(byteBuffer))
 ).use { it.readObject() as TicketDto }

 override fun encodeKey(key: String): ByteBuffer {
 return defaultCharset().encode(key)
 }

 override fun encodeValue(ticketDto: TicketDto): ByteBuffer =
 ByteArrayOutputStream().use { baos ->
 ObjectOutputStream(baos).use { oos ->
 oos.writeObject(ticketDto)
 byteArrayCodec.encodeValue(baos.toByteArray())
 }
 }

 companion object {
 val byteArrayCodec = ByteArrayCodec()
 }
}

The way codecs work in redis is actually very very simple. Everything, literally everythin is stored in a ByteBuffer! It's not even a String. You encode your values as ByteBuffers. These buffers keep your values stored in cache. Not Strings. You may be able in the code to make only String operations, but that's not what happens underwater. So essentially, and for my example this is what happens when you publish and then listen to your value:

1. Publish

   1. - the key gets serialised and the encodeKey method is called.

   2. - the value gets serialised and the encodeValue method is called.

2. Listener (Subscriber)

   1. - the key gets deserialised and the decodeKey method is called

   2. - the value gets deserialised and the decodeValue method is called

In my first attempts I kept on trying to do even more manual steps. And this is because I kept looking at the StringCodec example. Take a look at this class in your IDE to be amazed on how complex the actual String processing really is. Because I wasn't getting anywhere with pure low level methods, I decided to try the ByteArrayCodec directly. And voila. Now it works and I can serialise my objet correctly.

Finally was able to apply this codec which make code declaration slightly different than usual. This is the bean factory modified:

@Factory
class RedisBeanFactory {
 @Singleton
 fun pubSubCommands(redisClient: RedisClient): RedisPubSubAsyncCommands<String, TicketDto> =
 redisClient.connectPubSub(TicketCodec()).async()
}

And this is the way I'm publishing the ticket now. Again, still purely experimental code:

redisClient.connectPubSub(TicketCodec()).async().publish("ticketsChannel", ticketDto)

It is entirely up to you, whatever you want to cache and whatever data structures you should use. Even plain old strings. And I'm also not saying with this article that using a string instead of an object representation is per se a bad thing. But in order to make good decisions, we need to know what is possible and what it's not possible and a common mistake is to assume what is not possible. A good philosophy maybe is to assume that anything is possible until proven otherwise. This always goes back to the same principle: "Knowledge is power" and "The pen is mightier than the sword". In this case, It would have made me really happy to know that the reason we used @ToString from, among so many other ideas, lombok, had been something in the lines of "it improves performance". I hope with this bite, to have broaden your horizons in your work with Redis, if you didn't know this already.