What's it all about, Alfie?

Today I finally had time to finish my move to the new development 'server' box in the server-room. It's a little 4-core 1U box, but it's what the "standard" development server is. So be it. I needed access to more memory than my desktop would hold, and this guy is capable of holding 192GB - which is nice, but it'll start with 32GB and see how we go from there.

Since I have no root access, I have to check a few things, ask a few things to be changed, and repeat until everything is set up as I need it. Not exactly hard, but it's detail work that needs to be done right so that should I need to build another box, they'll be able to use this as a template and stamp them out quite nicely.

After I got done with the move, I made a few much needed GUI changes to my web app. The fixed calendar was just taking up too much room and it had to go. Thankfully, it's also easily made into a pop-up calendar, with an INPUT tag associated with the value. It took a little messing with divs, but I got something that looks pretty nice, and now there's a lot more room on the page for new GUI features. Not bad.

I was reading the feeds this morning and came across this little guy, Quiet Read, and thought it might be just the thing I needed to hold onto a few web sites that I might need, or want to hold onto - but not put them in my bookmarks as I wouldn't need them for that long a time.

It's a clever little idea. I might have done the GUI a little differently, but it's an early effort, and if he keeps up with the changes, I'm guessing it'll be very handy to have. We'll see how I use it in the weeks to come.

This morning I noticed that Firefox 3.5.5 was out with a list of several stability fixes - some of which were considered critical. OK, I'll upgrade, sounds reasonable.

Today has been a really hard day. I've worked all day trying to get the speed of this one process hitting a legacy Windows app up enough to make the users happy enough so that they don't ask for features cut to get the speed where they need it to be. I totally understand why they need the speed. I also know what's causing it to be slower - it's the additional features, but there has to be a balance between the two - for there are people that the new features are the more important part, and others that the speed is the more important part.

So I'm trying to get speed and features, but today was very disheartening.

For ten full hours I tried to get the speed up. I tried everything I could think of, and then thought up new ideas, and tried them. I was sure I could find the speed - but I couldn't. I ended the day with the same cycle times as I started with. Sure, they were a lot better than the production values, but that was hardware. Run the process on faster hardware and you get faster times.

Obvious, but it's all I had.

That's what we'll run with until we get even faster hardware for this guy. It's not really rocket science, but it's more than a little frustrating. But like so much in life, there's nothing I can do about it past what I've already done.

I got a tweet this morning from Adium saying that 1.4b14 is out fixing the Facebook problem that popped up yesterday. It's got a lot more in the release, and I'm constantly amazed by the work that's done on this guy, but it's still so useable. I use it every single day. Amazing app.

Today I spent a lot of time trying to figure out what the problem was with the hardware for this little feeder app to this larger webapp. I started with a desktop box, and got it set up and running. The times were nice, but still about twice what it was running on my desktop. So that spun off a big investigation as to why seemingly identical boxes are off by a factor of two.

After that for a bit, I was given a Windows Server to check. It was fine, but the times were slower - primarily due to slower processors in the server. Still, it was nice to see that I was able to run the application on Windows Server as opposed to just XP. But in the end, "nice, but no good", I was still a long way away from the marks I'd taken using my desktop.

When I went back to the desktop to try and account for the factor of 2, I noticed the network card was running at 1/10th that of my desktop, so I called in the network support guys to ask to have it switched up to 1Gbps. Turns out, they can't start until 4:15 pm. Yum.

I wish this was more clear-cut. I have something, but not enough. Have to keep at it tomorrow.

Today I got a tweet about a new release of BBEdit being out - 9.3. The release notes are extensive and shows this to be a significant update to the system. Wow. Pretty impressive.

I'm not sure about a lot of the new stuff, but the "maketags" argument is great as it means I don't have to worry about making my own ctags file. Very nice of them.

I got a nice little notice from LaunchBar that 5.0.2 was out with a few pretty serious bugs fixed. It's something I use every day, and while I'm not indexing Services a lot, it's nice to get the memory leaks fixed.

Today I've spent a lot of time trying to find performance where there was no guarantee that it should be found. I had a strong feeling that there had to be a better way, but I had no proof. I have this application that needs to talk to a legacy system through a DataSource connector (yeah, Windows) to get data out and into the app I inherited. Doesn't sound too hard. Yeah, right.

Well, this is complicated by the fact that there are a few ways to get at this data, and I was looking at different access schemes and thinking "If I could get the speed Excel has in getting the data, I'd be OK". So it became a quest for how Excel did it, and matching it's speed.

I started with the existing code - using an OleDbAdapter. This was taking a horribly long 130 sec. to get the data that Excel could return on my desktop box in about 12 sec. That factor of ten was killing me. I needed to run it on the VMs, but hey, they're supposed to be virtual machines and match the performance of my desktop - right?

So I tried an ODBC connector. That didn't get me anywhere because the methods to populate the .NET DataTable weren't implemented in the ODBC driver. After I passed this back to the developer that pointed me in this direction, he agreed that yeah, this wouldn't work. Nice.

Then I tried a more "bare bones" approach - getting the data at a low-level array and then pulling that apart to make a DataTable out of it. WHen I got that one into test, I realized that the time was even worse. Not the right direction to go in.

So finally I started running all the tests on all the machines I had at my disposal: my desktop, my Test VM, and the Production VM. The results were shocking.

so while I might see that the difference between a VM and a desktop is somewhat fixed for the Excel query, it's hardly fixed for the programmatic query.

Why the difference?

My best explanation is that VMs are horrible for anything that needs to use a disk. The two VMs have different disk subsystems and neither is really good. We are getting a new SAN, but that's going to be a few weeks away, still. The performance should then even out as all the VMs will be moved to the new SAN, but until then, we're stuck with what we have.

Additionally, the desktop seems to have consistent performance in either case. While this seems logical, it also seems to point to the fact that the Excel query is much less memory-intensive. I can see it pulling in the data in a stream. The code query has to get it all in a DataTable and then process it. I can see this being a big difference in the table sizes we're looking at.

What I'm going to try tomorrow is to get a plain desktop and try to set it up as the compute box to replace the production VM. This is not going to make me any friends in the support group as they hate having production apps on desktop boxes, but hey... this is an order of magnitude difference. We can try to get XP-compatible servers and replace the desktop with a server-room solution when we can get it in, racked and powered. But for now, the users are screaming and I need to give them a solution.

Amazing how things work out.

Saw this morning that VLC 1.0.3 is out. Since it's a core part of HandBrake and ripping DVDs, I had to upgrade. I'm a little surprised that the built-in auto-updating is not picking up these changes, but then, maybe they don't consider these to be significant updates.