What's it all about, Alfie?

I've got two large Java web apps that I'm working on - one I wrote from the ground up and another I inherited. Both have the problem that they are getting very large for the boxes we have, and I need to be doing something about their memory footprint. The problem is, in both cases, the values are relatively important and thinning the data footprint out is not as trivial as it seems.

Multi-Level LRU Caches

In one of the apps, I had recently put an LRU Cache so that I would not have to save every value, only those that had shown themselves to be of use to the users. Sounds like a good idea. But there's a limit to that. Each operation on the BKLRUCache is going to re-order the aging list, and if a lot of updates come into the system in short order, there's a possibility that they will purge the cache of "good" values.

Sure, you can increase the size of the cache, but that defeats the purpose. You can also buffer up the changes and treat them as a single batch, but that has negative effects as well. No, the idea is to somehow "flag" the requested entries so that they stay over the "untouched" ones in the cache. But what's the easiest way to accomplish that?

What I came up with, after much hassle and arguing with myself, was a two-level LRU cache. The primary level is the same, fixed size. When the user asks for a value, that value is first looked for in the secondary LRU Cache, and if it's not there, the primary cache is checked. If it's there, it's put into the secondary cache so that it's not competing with the other entries that may never be touched by a client.

I don't have to worry about the size issues - they are independent caches, and I can size them to suit my needs. I also don't need to worry about the flood of updates wiping out the one previously asked for. It only has to compete against the other requests. Very nice.

H2 File System Database Tests

In my other web app I've got a very large H2 in-memory database. This is getting to be a real problem. So much so that I'm starting to think about alternatives to the in-memory database. There's MySQL, PostgreSQL, SQLite, even H2 has a file-system database mode. So as I looked at each of these I realized that there's no way I would be able to tell the difference until I started running some tests.

Given that H2 has good specs and I already had it in the project, it seemed like a good enough place to start. What I did was just to make it not in-memory, but based on the local filesystem (simple disk), and run the application and see what the results were.

I was stunned to say the least.

I tried a lot of things, but I kept coming back to the factor of 6x to 10x in speed. It's just not possible to have a 3 second response time to the queries that I need to process. No way. So I must come up with something that makes it possible to get better speed while still not taking up a hundred GB of RAM.

No solution yet.

This afternoon I got a tweet about the newest release of Acorn 2.1 from Gus at Flying Meat. There's a lot of new stuff in this release - AppleScript support, color picker, rulers, etc. It's a nice update, and he's been busy. It's not been that long since 2.0 was released. Pretty impressive.

This morning I saw that iConquer 3.0 was released as a 64-bit app with full Snow Leopard support. I haven't played it in quite a while, but it's still my favorite implementation of Risk on the Mac. I've seen one on the iPhone, and have been tempted, but haven't taken the plunge. For now, this is how I play the game, and it's great to see that at least some work is being done on the code. For a while there, I was thinking it had been sold off and mothballed.

Flip4Mac WMV 2.3.0.14 was finally released today with Snow Leopard support. It's been in beta for a while, and I've been using it, but it's nice to see the guys actually feel it's "done" and release a final version. I don't look at that much windows media, but it's always nice to know that I can, if I need to.

This morning I was hoping to see a nice, stable, web app that I had to patch together last night. Instead, I had a crashing app that wasn't working at all. Crud.

It seems that the report updating thread - that worker thread that updates all the reports for new data arriving in the system, was dead. And after a restart, it died in only a few minutes. Not good.

So the first thing I did was to put the HashMap back into the code for the LRUCache which gave us the stability, but it was a ticking time-bomb. Around 2:00 this afternoon it was going to run into that same 12 GB limit and we were going to be in a mess of trouble. Not good, but I had no choice.

Once the HashMap was back in the code, I started looking for the fall-out of using the LRUCache. Because, in theory, there's no reason that the LRUCache should not have worked just like the HashMap - it's a Map, after all. It's just a question of removing some aged entries.

The code was telling me that once again, we had optimistic coding and the problems were not as easily removed as switching out one Map for another. The problems came about because the code assumed that everything was going to exist - no checks whatsoever. With the LRUCache not having some data that was once there, we had an easy NullPointerException. With no try/catch block on the thread, it simply died. Wonderful.

The changes were that horrible - basically just putting in the checks that should have been in the code in the first place, but the idea of not having something like a try/catch block on the Thread's run() method is just amazing.

Why?

Oh... I know. It's never going to fail.

I finally got something that stays up, looks like it's all OK, and most importantly, the memory footprint is stable. It's been running in NYC for a few hours and the memory is flattening out nicely. One issue was that the LRUCache was really only needed on the archive storage for the reports. The other HashMap was really just a mapping of the report description to the last report instance. That makes sense to leave as a HashMap as it won't be growing without bound like the archive will.

The only way to know is to find out what tomorrow will bring.

Blender 2.49b was released this morning and I had to get it. Yeah, it was all about fixes for the game engine part, but I still love it. It's one of the only places that still has a download link for the Irix OS. Cool.

There's been a problem with the 500GB drives in Unibody MacBook Pros where there's a hardware delay due to the drive and the machine not quite understanding the spin-down and spin-up cycles. Well... this is the second update I know of for the MBP to try and fix this. I haven't seen it, but I'll certainly update on the off-chance that I have seen it and just not noticed it. That would be a funny one.

With another round of security and stability fixes Firefox 3.5.3 is out. Pretty standard stuff, just needed to update and we're done. Simple, but very effective. Use it every day.

I've been working on this new deployment of this app I inherited and today I was nailed with a design decision that really shouldn't have been in the code in the first place, but was, and it had an ripple-effect that was really quite spectacular.

The design decision was to save every generated report for the lifetime of the web app. This was done so that anytime during the day a difference report could be made (on the server) between any two points in time. The reason for these difference reports was to enable the client to essentially freeze the report and then track the changes to the frozen state as time progresses.

One might say that this was the job of the client code - face it, it's already getting the data - it just needs to hold on to one dataset and then difference the incoming set to the saved set. It'd be minimal coding, easy, but it wasn't done. Nope.

So... when we added a lot of fields to this release, each report is now much larger, and we've added the roll-ups by product which only adds to the data per report. What happened was that by 2:20 pm the memory usage of the Tomcat instance was at 12 GB! It was slugging through Garbage Collection and I had to restart it. That helped, but I had to get a solution, and fast.

I talked to the original author and he suggested an LRU Cache on the data and not hold all the results - only the last n that have been accessed. It turns out, the implementation of an LRU Cache was pretty simple. I added that in place of the HashMap data structures for the retention of the reports and it appeared to work just fine. I checked that things worked, and that was what I put into production for tomorrow.

This should help a lot.

I have to admit that I'm not impressed by Java a lot. It's just the level of familiarity I have with the language. It's just not that often that something really surprises me. So when it happens, it really blows me away. Today is one of those days.

I was trying to implement a Least Recently Used (LRU) Cache, and a co-worker said it would be easy. I doubted it, it's not like it's that easy to make one. But he Googled a few places, and when you base it on the Java LinkedHashMap, it really is easy:

package one.bkit.util;
 
/**
 * Java System-level Imports
 */
import java.util.*;
import java.util.Map.*;
 
/**
 * Superclass Imports
 */
 
/**
 * Class Imports
 */
 
/**
 * This class is a simple implementation a Map where only the last
 * recently used entries are going to stay in the map. You can use
 * the default size of 100, or you can give it a size. In either case,
 * every <tt>get()</tt> and <tt>put()</tt> are put through the test
 * the 'most recently used' filter. If the next <tt>put()</tt> pushes
 * one out of the map, then it wasn't in the most recently used.
 */
public class BKLRUCache<K, V> extends LinkedHashMap<K, V> {
 
    /**
     * This is the size of the cache. No more elements will be held
     * in this map than this. After this, the oldest goes to make room
     * for the newest.
     */
    private int             _maxEntries = 100;
 
    // this is the serial version tag for this class
    private static final long serialVersionUID = 20091014;
 
 
    /**
     * ----------------------------------------------------------
     *		Constructors
     * ----------------------------------------------------------
     */
    /**
     * The default constructor assumes a default maximum size of 100
     * elements such that <b>only</b> the most recently used 100 entries
     * will be maintained in the map. After that, the oldest is discarded
     * to make room for the newer entries.
     */
    public BKLRUCache() {
        this(100);
    }
 
 
    /**
     * The general form of the constructor takes the maximum size of the
     * LRU cache, and uses that as opposed to any default.
     */
    public BKLRUCache(int maxEntries) {
        // fix the size, keep things flat for speed, and use access order
        super(maxEntries + 1, 1, true);
        _maxEntries = maxEntries;
    }
 
 
    /**
     * This version of the constructor takes a map and populates this
     * map with up to 100 entries - actually, it'll hold the <b>last</b>
     * 100 entries of the iterator on the argument. It's doing the
     * <tt>putAll()</tt> on the argument, and the way this works, only
     * the last 100 are saved.
     */
    public BKLRUCache(Map<? extends K, ? extends V> m) {
        this(m, 100);
    }
 
 
    /**
     * This version of the constructor takes a map and populates this
     * map with up to maxEntries entries - actually, it'll hold the
     * <b>last</b> entries of the iterator on the argument. It's doing
     * the <tt>putAll()</tt> on the argument, and the way this works,
     * only the last 'n' are saved.
     */
    public BKLRUCache(Map<? extends K, ? extends V> m, int maxEntries) {
        this(maxEntries);
        putAll(m);
    }
 
 
    /**
     * The magic happens here as the Java LinkedHashMap has a method
     * that we can intercept and tell it to keep (or not) the oldest
     * entry in the map. This is where we look at the size and then
     * see if it's a keeper or not. Simple.
     */
    @Override
    protected boolean removeEldestEntry(Entry<K, V> eldest) {
        return (size() > _maxEntries);
    }
}

This is the kind of code I love to see: simple... elegent... compact. It does something very useful and it does it without a lot of grief. Amazing.

Days like this make me want to do more Java coding and learn more of the newest language additions. It might be nice.