We're making progress, but it's still a long way to go. This "Developer Day" project is turning out to be a lot more than a day. No question. But hey... with this part done we'll be ready to really deliver something really great. I just have to make it happen. Gotta hold out.
It's been a lot of work, and we have a little more than a week to go, but I think we can make it. I'll be close.
This afternoon I saw that Acorn 3.0.1 was out with quite a nice list of fixes and new features. It's pretty amazing, but I guess when you're on a roll, the ideas are there from all the suggestions, and you have a solid codebase to work from, so things should be a little easier. Still, it's nice to see the improvement.
This morning I saw a tweet that Twitterrific for Mac 4.1 was out, and hoped that they had done something to make the graphics switching problem I was having a thing of the past. Alas, no. But they did fix quite a few other things, and that's nice - for the day I'll be able to actually use it.
I like the new font rendering as I think it'll allow me to go back to the 'dark' color scheme and still read things. It's just a shame that I can't use it with my MacBook Pro yet.
This morning I was working on some code and it was getting a little frustrating working with the mongoDB replica sets - after all, they are meant to be able to switch the PRIMARY as will. Fair enough, but that makes it hard if you have to target the primary in your mongo command-line shell:
$ mongo --host xchi43mongo:32018
where we might have a replica set of xchi43mongo and xchi44mongo. Seems they should make the third box in the set be the one to talk to - but I don't write it, so I have to try one and then another if I'm wrong. Kinda frustrating.
So this morning I decided to figure this out, and sure enough, mongoDB allows for referencing it's replica sets:
$ mongo --host dapool/xchi43mongo:32018,xchi44mongo:32018
and it picks up the name of the replica set from your command as well as the machines in the set. Nice. I can see this - it's an alias now, and I can pretty easily deal with this.
Not bad, mongoDB.
This morning, once again, I'm getting timeouts hitting the mongoDB replica set in staging at The Shop. I know mongoDB is supposed to be this amazing piece of software, but I'm seeing it right now as a major pain in the rear. I'm looking at a database whose size right now is less than 10GB, and it's a major source of grief. We are having replication issues, and the erlang drivers had to be changed, and then retrofitted. The Java drivers had to be updated to get document sizes greater than 16MB.
But those are the known issues. It's these gotchas that are really annoying. I'm trying to get a single document from it and it's timing out. I get no idea why. One box it's fine, another box it times out. Same mongoDB, but different boxes. This worked yesterday, so what's changed overnight?
Arrggghh... I'm really beginning to hate this software.
UPDATE: it turned out to be the socket buffer size on the default socket for erlang. It was at something like 1kB and I was moving MB through. Very inefficient. When we got the socket options set right, the speed wasn't too bad. Not fantastic, but not bad.
Today was spent in a few meetings, and helping get a few developers on the right track, but when I had the time to do a little coding, I was working on getting everything glued together for the greek engine. Specifically, the first part of the engine will be a Broker service that will do all the what if scenarios for the users. This is something that the folks have wanted for a long time, and it makes a lot of sense to deliver this first, with the large-scale, center-state, reliable multicast, greek engine put together from these same components once we get this guy up and running so we know how much hardware to spend on the problem.
This phase of the project is nice in that it appears to be making a lot of progress in a very short time, but in reality, it points out where you made mistakes in the design up to this point. If you have a good design, then it indeed goes together pretty quickly. But if you missed something, then you have to go back and fix or retro-fit that, and then put things together.
Interestingly enough, I had a little of both today. My general Source and Sink objects needed to have locks provided on their lists, but that's because I was starting to violate the assumptions that had been made in their original design. I could go back and implement a lockless list, but that's a little overkill, I think at this point. If we need it, I know how to do it, I just need a little time to actually do it.
It's not done by any means, but it's getting there. Slowly, but surely.
This morning the Google Chrome team released 12.0.742.16 with what it calls "UI and performance issues" - which appears to be the theme in recent weeks. It does appear that they are taking the time now to really clean things up and not focus on new features as much as clean up the codebase and polish the app as much as possible. Good enough. Sounds like a plan.
This afternoon I've been helping the team get things straight for the next few days, and starting to pull things together in how we'll present this to the clients for their consumption. One way would be to have another ZeroMQ PUB/SUB reliable multicast system, but that's really overkill. What we really want is to have a point-to-point, one-on-one conversation with a client so that they can tell us what they want, and we can supply it for them.
Things are starting to take shape in my mind, but it's always better if I noodle on them for a while.
After my experience with the utility of indexes for mongoDB, I was not surprised to find that this extended to pretty small doc sets as well. Specifically, I had a lot of configuration data stored in a mongoDB collection as some 300-ish documents. But they were binary, and when I added an index on the appropriate key field of the documents, the access speed dropped like a rock. Once again, mongoDB wasn't storing the starting points of the docs, it was doing a complete table scan, and in this case, it was very expensive because the table it was scanning had a few very large docs.
Amazing.
This has become such an issue, we put code into The Broker so that the first time someone accesses one of these document stores, it ensures there's an index on the key field. It's going to make it impossible to have bad performance because you forgot the index. Sure, it may be calling ensureIndex() more than needed, but it's only once a runtime startup, and the benefit far outweighs the cost.
Just amazing difference.
I haven't had a lot of issues with Adium, but when they start another beta cycle, it's something I want to be a part of. They usually have significant features or fixes in the betas - so it's something to be a part of. So this morning when I read in chat that they were starting another with fixes for the MSN buddy list maintenance, I downloaded 1.4.2b1.
The release notes are already impressive. I did have one thing I needed to deal with - the security setting on AOL was causing me grief, but it was an easy fix.