The Java Cache Disaster That Drove Me Crazy ??

The world of software engineering is wonderfully unpredictable. Some days pass by smoothly, while others can be grueling and leave a lasting impression. However, it is these challenging days that teach us lessons for a lifetime.

Today, I’m sharing one such incident I faced last year and how it taught me a new lesson about Java.

Background ??

On a beautiful Monday morning, our day began with a dose of Monday blues.

An inexplicable bug appeared in our system—one that had been working fine for so long that we could barely remember when it wasn't. But isn’t that the story of every bug? It’s just waiting for someone to notice, and for this particular one, the day had finally arrived.

To summarize, here is the pseudocode of what caused the issue:

if (simple condition) {
	performOperation();
}        

We discovered that this piece of code was executing sometimes and not at other times, even in seemingly similar scenarios—almost identical, one might say.

At first glance, this code seemed perfectly fine—there was nothing unusual about it. It had been working flawlessly until now, so what suddenly went wrong?

So, what really happened?

Keep reading to find out.

The Puzzling Hunt ??

We had the logs, we had the payload and we definitely had our clocks ticking. What we did not have was the root cause of this issue. So we started the debugging by following the flow of request, capturing the state at every crucial juncture, and eliminating possibilities.

We hadn’t pinpointed the issue yet, but somehow, amid this multi-service interaction, we were able to narrow it down to one service. However, there was still much more to explore.

Every issue has a root cause. It might seem hidden until you find it, but it's always present. You just need to look a bit harder.

The good news was that we had at least limited our search scope.

With this revised search space, we donned our debugger’s hats again, navigating logs, dry-running code, and reproducing the issue in lower environments.

Finally, we narrowed it down to the piece of code I shared above:

if (simple condition) {
	performOperation();
}        

The issue was straightforward:

This operation was sometimes executed and sometimes omitted. However, our understanding of the underlying code suggested that it should have always been executed given the input provided.

So why was it bypassed in some instances?

There could be only one reason: the simple condition was not so simple after all.

But if I told you it was just a basic integer equality check, would you believe me?

Believe it or not, when I said “simple condition,” I really meant it.

So let’s take a break and see how Java handles these cases, and then we’ll continue the story.

The Pitfalls of Equality in Java: == vs. .equals()

First, let’s start by comparing two primitive int values.

int firstNumber = 1;
int secondNumber = 1;

if (firstNumber == secondNumber) {
    System.out.println("Equal");
} else {
    System.out.println("Not equal");
}        

Output
------- 
1 and 1 are Equal        

Well that was quite primitive (literally as we were comparing primitives) and the result also seems to be as expected. After all, 1 is equal to 1.

But did you know that each primitive data type has a corresponding wrapper class? For example, for int, we have Integer.

Let’s see what happens when we perform the same comparison with these wrapper classes.

Integer firstNumber = 1;
Integer secondNumber = 1;

if (firstNumber == secondNumber) {
    System.out.printf("%s and %s are Equal", firstNumber, secondNumber);
} else {
    System.out.printf("%s and %s are Not Equal", firstNumber, secondNumber);
}        

Output
------- 
1 and 1 are Equal        

1 is still equal to 1, and that’s to be expected.

However, here’s something interesting: as we increase the numbers (not too high, just up to 128 or beyond), the result might surprise you.

Integer firstNumber = 128;
Integer secondNumber = 128;

if (firstNumber == secondNumber) {
    System.out.printf("%s and %s are Equal", firstNumber, secondNumber);
} else {
    System.out.printf("%s and %s are Not Equal", firstNumber, secondNumber);
}        

Output
------- 
128 and 128 are Not Equal        

And you thought 128 is equal to 128. How naive!

Before diving into what actually happened, let’s take a look at a small extract from the IntegerCache present inside the Integer.java class, as the title suggests: caching.

public static Integer valueOf(int i) {
        if (i >= IntegerCache.low && i <= IntegerCache.high)
            return IntegerCache.cache[i + (-IntegerCache.low)];
        return new Integer(i);
    }        

with

IntegerCache.low = -128
IntegerCache.high = 127 
(high can be modified using VM args but for now treat it as default)        

So when we are dealing with Integer wrapper instead of primitives, Java does not create a new object each time you set a value like following

Integer firstNumber = 1;        

During the initialization of IntegerCache, Java pre-populates the cache and stores objects for values in the range of -128 to 127.

Later, when an Integer is needed, Java checks this cache (which contains numbers from -128 to 127) and reuses the existing objects.

But you might wonder, what about numbers outside this range?

For values beyond -128 to 127, a new Integer object is created each time.

So, what’s the benefit of this caching? The range of -128 to 127 is chosen because these values are frequently used, allowing Java to reduce the overhead of creating new objects for this common range.

I’m sure you can already see where this is headed.

But before we return to the story, let me clarify the correct way to perform this comparison.

For value comparison, you should never use == with objects.

The == operator checks for reference equality, meaning it compares memory addresses rather than the actual values. Thus, two objects are considered equal using == only if both references point to the same object in memory. This is why Integer(1) is considered equal (since it’s cached), but Integer(128) or larger values are not, as they are not cached.

The proper way to compare values is to use the object.equals() method. This method checks for logical equality, which means it compares the values within the objects. If the equals() method is overridden in a user-defined class, it will use that implementation for comparison.

So, when we repeat the two examples from earlier using the equals() method, both will correctly indicate that the two objects are equal.

Integer firstNumber = 1;
Integer secondNumber = 1;

if (firstNumber.equals(secondNumber)) {
    System.out.printf("%s and %s are Equal", firstNumber, secondNumber);
} else {
    System.out.printf("%s and %s are Not Equal", firstNumber, secondNumber);
}        

Output
------- 
1 and 1 are Equal        

Integer firstNumber = 128;
Integer secondNumber = 128;

if (firstNumber.equals(secondNumber)) {
    System.out.printf("%s and %s are Equal", firstNumber, secondNumber);
} else {
    System.out.printf("%s and %s are Not Equal", firstNumber, secondNumber);
}        

Output
------- 
128 and 128 are Equal        

Story Continues… ??

I’m sure you’ve already grasped what went wrong in our code, but for completeness, let me lay it out:

if (simple condition) {
	performOperation();
}        

So this simple and innocuous looking condition was comparing two Integer objects (yes objects) using ==, which led to inconsistent behavior of the application due to caching scenario we just discussed.

Once we pinned down the issue, the fix was straightforward. The real challenge in this whole process was to wrap our head around this behavior but once we did it, I can safely say that this learning will stay lifelong with me.

Final Thoughts and Learnings ??

As a Java developer, I knew that equals() should always be used for value comparison, while == is for reference comparison.

Now call it our fortune or misfortune, but the values had always been within a very short range (I think that is why this short range of cache has been chosen by Java. Most use cases of int are covered in them).

Nevertheless, the key takeaway from this experience is the importance of understanding the subtleties. This experience taught me a valuable lesson: sometimes, the most insidious bugs arise from aspects of the language that seem straightforward at first. It’s only by diving deeper into the code that one can emerge as a better software engineer.

Did you learn something new or found this article interesting.

Leave a like below ??, share your thoughts in the comments section ??, and don’t forget to follow for more such content! ????

Keep showering your love on this one, and I’ll see you next time with something new. Till then, have fun and keep the Joy of Engineering alive within you! ???

#SoftwareEngineering #Learning #Design #Java #Innovation #JoyOfEngineering