I was reading a little of Wil Shipley's blog this morning - he tweeted that he had another post up about the potential Amazon/Microsoft merger, and ran across this post about how to code.
Solid Gold.
It's amazing to actually see that guys I believe are doing it right are, in fact, doing it exactly how I do it. It's a great affirmation of how I do my job.
Read it. Live it. Fantastic stuff.
I've been really slugging it out today. I mean really. Today has been a lot of work writing JSON configuration files for my ticker plant that will be read into a MongoDB that is the back-end for the configuration service. There's a lot of things I need to set up for this system - all the exchange feeds, all the distribution of the messages, all the mappings... it's a lot of stuff. In addition to all this, I really need to find a way to create the service coverage maps...
These things really got me down, I have to say. Not ashamed to admit it. There's something to be said about not getting bogged down, but when you're in the middle of it, the best thing you can do - I have found - is to just push through it. Taking a break isn't going to help - you have to come back to the exact same place. That's no good. There's no way to go around the problem, you have to go through it.
Kind of a drag, but it's what you have to do. And in the end, it's really what separates the good from the middle-of-the-road. If you push through when you're sick and tired of doing something, then you're going to get it done faster, and move on to something that you like to do. Let it really slow you down, and you're spending more time doing this thing you're going to hate even more.
Well... gotta get back to it. Pushing through it.
This morning I noticed that Google Chrome dev 7.0.544.0 was out, with several fixes including some Mac changes in the accelerated drawing code. Well, isn't that nice? Written proof that there might actually be accelerated drawing in the browser - that would be very interesting. But it really doesn't matter - it's a good browser, and it's nice to see progress.
I've spent most of today doing two things: weaving the authentication data into the Ticker Plant client code, and making a nice package (tarball) of the client include files and libraries for other groups at The Shop. Over the last few days, I've been doing the "second pass" on the code - looking at those supporting sub-systems and components that I've been "faking" with simple hard-coded values. For example, the original authentication system was a very skeletal 'client' of the authentication service and a simple 'token' that contains the important data, but the tests for the validity of the token was always true, and the client just created an empty one.
Like I said, very incomplete - but it worked and wasn't crucial to the functional aspects of the codebase. It's just the supporting stuff that the app had to have before it went live, but could always be done later. Well... now is that later.
Interestingly, putting the AuthToken in the code was surprising harder than I had expected. I simply hadn't exposed the ability to set the AuthToken at the highest levels of the code, which, it turns out, is exactly what I needed. Unfortunately, to find this, I had to do a lot of digging because I wasn't logging the requests that failed due to invalid AuthTokens.
First, I had to log things a little more, and then I saw the problem. Then I needed to see how often I was doing the same things in different places in the code. These weren't always done the same way in the code, so I took the time to standardized the way I was doing the authentication work. This took even more time. In the end, I was able to get it all working, but it took a lot more time than I had expected.
After that, I got pulled into a few meetings that pointed me in the direction that we might have a C++ client sooner rather than later. What I wanted to do was create a package - a tarball, that contained the libraries and include files so it's easily used by other groups. Making it was really a process of finding the minimum set of include files that I would need. Then I just had to stage them into a good directory structure and tar them up. I made it clean with some nice dependencies so that if we change one of the headers, we'll be able to know to make the package again.
Lots of good things done... but it was a lot of details to get done.
This afternoon was spent testing the ticker plant in all it's glory - that means loading data from the configuration service, checking the login service, and getting ticks moving. I realized that there were several parts of the system that weren't as polished as they should be - specifically in the configuration service client, and even a little in the Broker client code.
The configuration client didn't have a way for me to ask for a series of keys, and wait until all of them had returned. I had to do it one at a time. While it's not horrible, it's not as efficient as the system could be - after all, we created the asynchronous calls to the Broker just for this case.
So I added in the ability to get() a list of keys and have it returned in a variant list. The code actually sends out all the requests for the values asynchronously, and then waits for them all to return. There's a timeout, and a logging, if the complete set of values doesn't come back, but that's not something that's likely to happen.
In the end, I cut 40% off the time needed to get the configuration values this way. Nice.
This morning I finally was able to push through and finish the code for my persistence system for the ticker plant. We have a cache of messages in each ticker plant, and they are typically fed from the exchange ticker feed, but in the event that we need to restart the server, it'd be nice to be able to save and load the state of the messages (ticks) so if we have to restart, we can keep the cache in roughly the same state after the restart as before.
This is very important when you need to have a stable price flow. You can't have a restart take out the messages that came in the morning and haven't been seen since. Well... I'm counting on a service hanging off the Broker that is going to handle all the saving and loading for me. I'll simply send maps/lists of data (serialized messages) to the configuration service and when I restart, I'll read from the same. It's fine in that it'll work regardless of the machine, and we have all the tools at our disposal, but the problem is that we need the Broker and it's services up before we can start.
Minor point. If we can't get to the Broker and it's services, we're in a lot bigger problems.
So I got that written up, and a good bit of testing on the authentication service and the basic configuration service. I need to go back to the code now and start putting in calls to the configuration service so that these components and apps can load their configuration from the right source.
Never a dull moment.
While it's not my favorite graphics manipulation application on the Mac, GraphicConverter has shown itself to have some invaluable tools. Specifically, when we moved this past summer, I had a lot of documents that needed to be processed to and from the different parties. GraphicConverter did all this far simpler than anything else I had. It was simply amazing.
So when I got an email the other day about a totally revamped version - a year in the making, I thought What the heck, it's been good for me, stick with it and got it. It was a $26 upgrade, and it seems reasonable for that price. Like I said, it was a lifesaver several nights when I was dead tired and messing with these documents was not the first thing on my mind.
It's got a cleaner UI, and it seems to have everything I used. Support the folks making great Mac software. Nice.
I just got out of a meeting, and in the beginning things were going better than I thought. I was getting an idea on how their system was put together, listening to what they were doing, seeing where they thought they were going wrong. It was all constructive - up to a point. And at that point, I was very saddened to see "chips" appear on shoulders. Because I knew that the moment those chips went up, the communication broke down, and the meeting, and probably the entire collaborative project was over.
I understand the emotion. You're a professional. You care a lot about those things you've been asked to care about. When something happens that rumbles the ground you walk on, it's not a "good" feeling. It's OK to handle it for a while, but after a time, you really want to regain some sense of control. Like I said, it's understandable. I just wish people didn't get so upset about things.
But even that, I suppose, is understandable. I get upset at a lot of things. That I don't get upset about someone offering advice is my little filter. Maybe these other folks don't get upset by being in glass elevators - one of my phobias. It's just the want people are wired together.
The problem is that as soon as the offensive behavior goes up, the blinders go on and no amount of talk is going to bring them down. This person is now shut-off to me, and they aren't about to open up again until I agree I'm wrong, or apologize, or am reprimanded - something that means I'm addressed in the proper format, as they define 'appropriate'. It's just so counter-productive.
Sad to see it happen.
After getting the authorization client done this morning, I took a little time to write a configuration client for the Broker. One of the things we are going to need is a way to store all the configuration data for the different applications and clients in the system. Additionally, we need a place to store the current state of the ticker plants so that if they need to be restarted mid-day, we have a place to load them and come back to very nearly the same state.
Now nothing's going to be perfect, so there's going to be some data loss in the event of a restart, but the idea is that we need something that's going to have a recent history of the state of the machine so that we can get it back to the same place after restart. And after hours, it becomes perfect backup because the data isn't changing, so restarts for new code, etc. are easy to implement.
The scheme we've adopted is that there are several 'levels' to the configuration scheme - global values, local values, and user values, which makes it easy to read the same value at different levels and have something that personalizes the value as you get closer to the user. For example, we can set a global value for "timeout", but then a user value that's greater than that - or less than, based on the user's specific needs.
We also have it in a very simple key/value store so it's easy to put complex data structures in the config system, and give that it's read/write, you don't have to make a particularly fancy GUI to maintain it as the app itself can set it's state to some defaults, and then write this out to the config service and it becomes the values to read back in.
The problem I have right now is that it's written, but the configuration service itself isn't. So I'm a little ahead of the curve. As soon as it's done I'll fix any differences and then start using it in my code. No time like the present.
We're getting to the point that we're putting things together at a higher level and starting to look at the larger (near final) ticker plant and it's clients. Friday I started work on taking the authentication client (AuthClient) code and taking the API we created and implementing it to mate up with the Broker's loginservice which is our Gold Standard for authentication.
The exchange with the service is pretty nice, and the work we've done up to this point makes it a lot easier to work with. You basically have to pass it a map with the key/value pairs of the login name and password, and it'll return to you a uuid_t that is the authenticated value. If it's not a uuid_t, then it failed - for whatever reason. I end up logging the failed attempts, and clearing out the AuthToken. It's not really rocket science, but it's very effective.
I've been able to remove all the hard-coded values and "debug" code so it's really working. It's nice to get one more thing really hooked up. It's getting closer, day by day.