Working with XML-RPC, Mac OS X 10.7 and Xcode 4.3
For the last week or so I've been working getting something going with an old friend that involves a centralized service, Macs and iOS devices all sharing data. Most of this data is numeric, but that's only a current design consideration, and what we basically needed was something that was similar to iCloud, but not file or key/value based. We needed to have a specific local database (SQLite3 for Mac and iOS), and have that be a local copy of some data in a more centralized database (PostgreSQL) on a centralized server (Apache, PHP).
So the question was How to do it?
My initial thoughts went back to The Broker codec that I worked with back at The Shop, but then I realized that it was a really bad idea to use that as there's no proven need for that level of complexity, and that a far easier solution - and one I've used on the Mac in the past, is to have a simple XML-RPC client and server, and allow the server to be in PHP, and the client in the native frameworks and libraries on Mac OS X (and iOS). So I went that direction instead, and I think I've got something that works really well - and stays away from the entire marshalling/demarshalling issues that The Broker had.
But the XML-RPC route was not without it's trials and tribulations. Neither was using the wonderful FMDB Open Source library for Mac OS X and SQLite3. I had several things to figure out, and while it's all working now, it's not simple to figure all these things out, so I wanted to write this all down in a nice, concise way.
Step 1 - The Changes in Xcode 4.3
One of the firs things I wanted to do was to start a new Xcode project from scratch. Most of the projects I have weren't started with Xcode 4, and I wanted to see what I was going to get if I started clean, and put everything into git right from the start.
The first thing was that the new Xcode projects allow for a simple unit test framework that's very similar to the JUnit framework. You have a nice pair of set-up and tear-down methods, and then anything you define in the implementation file (.m), that starts with test is going to get run in the alphabetical order in which it was defined.
This is very nice and clean, and it means that you really don't have to mess around with the interface (header) file for the tests. Sure, you have to put in the ivars that you might need, but there than that, it's pretty bare. It's all in the implementation file.
There are also a set of methods for assertions like true and not nil, and all kinds of tests that you can sprinkle in the test code to make sure it's running the way you think it should. Very simple to use.
Running the tests is equally easy as it's just a simple, single key-stroke in Xcode to build the project, and the tests, and run the tests. I was very impressed as it made it very easy to run the tests until I got everything working just the way I wanted it.
Step 2 - FMDB as a Framework (well… almost)
I knew I wanted to use Flying Meat's FMDB library on GitHub for access to the SQLite3 database. I've read a lot about this library, and it looks to be set up exactly as I might have done it - but I didn't have to do it. Excellent.
But it wasn't without a little bit of grief. First off, it's not a Framework. Or a Library. It's a group of files. More than that, it's a group of files with a test app that's put together in a way that makes it a little unclear how to actually use it. I had to do a little googling to see that what you needed to do was to extract the files you needed from the project, place them into your project, and then start using them. In short, it's a library that's really a bunch of files.
This isn't bad, but a few sentences in the README would have made it very clear what I needed to do, and based on the things I've seen in Xcode 4.3 - converting it to Xcode 4.3 and using the unit test framework would make this all unnecessary. It could be a library, and it could have unit tests, and it would be the same code, it just would be a lot easier to use. But so it goes.
I added the files to my project, placed them in a 'group' (a folder in Xcode 4, but not on the disk itself), and then I was ready to go. Not horrible, but again, a few sentences, or a migration to Xcode 4 and this would all have been unnecessary.
Step 3 - XML-RPC Service in PHP
I had some code from a while back when I used a different XML-RPC implementation than what now is standard in PHP 5. So I was a little careful about how much to expect to be the same, and how much was implementation-dependent. Turns out, the built-in XML-RPC in PHP 5 is far nicer than the previous version I was using, and far faster as well. Win-win.
I have had no problems writing simple XML-RPC code in PHP 5, and it's nicely compact as well:
<?php /** * This is the simple 'ping' service that let's the client know that we * are here and answering requests. */ function doPing($method, $args) { return "PONG"; } /** * At this point, we need to set up the XML-RPC server, bind the method * calls to the individual functions in this file, and then let the * user's call get processed and return the results. It's all pretty * simple, and handles all the ugly parts of the marshalling/ * de-marshalling and method invocation. */ // first off, make an XML-RPC server $server = xmlrpc_server_create() or die("Can't create rpc server!"); // next, let's register the PHP functions as methods xmlrpc_server_register_method($server, 'ping', 'doPing') or die("Can't register the 'ping' method!"); // now let's get the request and options... $request = $GLOBALS['HTTP_RAW_POST_DATA']; $options = array('output_type' => 'xml', 'version' => 'xmlrpc'); // ...and make the call, returning the data to the client. print xmlrpc_server_call_method($server, $request, NULL, $options) or die("Can't call the method!"); // finally, we can shut down the XML-RPC server. xmlrpc_server_destroy($server); ?>
This is a simple ping service that was the first thing I built to make sure that I could communicate with the PHP service and the Obj-C client. It's not bad at all in PHP, and since it's implemented in C, it's as fast as we're going to get.
So far, looking very good
Step 4 - XML-RPC Client in Objective-C
The client side was a little more code, but logically, it was very simple as well. I wanted to encapsulate the XML-RPC call into a simple method that would take Foundation objects and return Foundation objects so that I wouldn't have to worry about data type conversions, or any marshaling or demarshalling issues. What I came up with was a general 'call' method:
- (NSObject*) call:(NSURL*)aURL method:(NSString*)aMethod with:(NSDictionary*)aParams orderedBy:(NSArray*)aKeyOrder { BOOL error = NO; NSObject* retval = nil; // first, make sure that we have something to do if (!error) { if (aURL == nil) { error = YES; NSLog(@"[SKPipe(Protected) -call:method:with:orderedBy:] - the provided URL is nil and that means that there's nothing I can do!"); } } if (!error) { if (aMethod == nil) { error = YES; NSLog(@"[SKPipe(Protected) -call:method:with:orderedBy:] - the provided method name is nil and that means that there's nothing I can do!"); } } // make the XML-RPC method invocation as it's the basis of the call WSMethodInvocationRef rpcCall; if (!error) { rpcCall = WSMethodInvocationCreate((__bridge CFURLRef)aURL, (__bridge CFStringRef)aMethod, kWSXMLRPCProtocol); if (rpcCall == NULL) { error = YES; NSLog(@"[SKPipe(Protected) -call:method:with:orderedBy:] - the XML-RPC method invocation to '%@' with method '%@' could not be created and this is a real problem. Please check on it.", aURL, aMethod); } else { // set the parameters for the invocation - if we have any if (aParams != nil) { // see if the user has given us the key order to use if (aKeyOrder != nil) { WSMethodInvocationSetParameters(rpcCall, (__bridge CFDictionaryRef)aParams, (__bridge CFArrayRef)aKeyOrder); } else { WSMethodInvocationSetParameters(rpcCall, (__bridge CFDictionaryRef)aParams, NULL); } } } } // do the actual XML-RPC call and get the results if (!error) { NSDictionary* pkg = (__bridge NSDictionary *) (WSMethodInvocationInvoke(rpcCall)); if (pkg == nil) { error = YES; NSLog(@"[SKPipe(Protected) -call:method:with:orderedBy:] - the XML-RPC call returned nothing at all and that's a serious problem. Check on it."); } else if (WSMethodResultIsFault((__bridge CFDictionaryRef)pkg)) { error = YES; NSLog(@"[SKPipe(Protected) -call:method:with:orderedBy:] - the XML-RPC invocation of '%@' to '%@' returned a fault: %@", aMethod, aURL, [pkg objectForKey:(__bridge NSString *)kWSFaultString]); } else { // looks OK... get the results retval = [pkg objectForKey:(__bridge NSString *) kWSMethodInvocationResult]; } } return (error ? nil : retval); }
The code is pretty simple - we first check to see if we have the necessary arguments to the method, and then we construct the method invocation. If there are parameters to pass to the call, then we add them, and if there is an array defining the order of the arguments in the dictionary, then we supply that as well.
We then first off the method invocation and get the return value. Pretty clean, but you have to know all the little tricks to get it to work. Thankfully, once we have this one method, we don't have to mess with XMl-RPC any more and can just stay in Objective-C and Foundation objects.
Calling the ping method isn't too bad… in fact, one of my tests does exactly that:
- (void) test_1_Ping { if (![_pipe ping]) { STFail(@"The server ping failed!"); } }
where:
- (BOOL) ping:(NSURL*)aURL { BOOL error = NO; NSObject* ans = [self call:aURL method:@"ping"]; if (![ans isEqual:@"PONG"]) { error = YES; NSLog(@"[SKPipe(Protected) -pingServiceAt:] - the XML-RPC response was not good: %@", ans); } return !error; }
It's really pretty simple to use for strings, integers, arrays, dictionaries, and most data types and values. But there are two things that XML-RPC doesn't do really well, and I needed them both: NULL and Date/Time values.
Step 5 - Dealing with NULLs
XML-RPC just can't send a NULL. No way to tag it. I found a reasonable solution in the idea of sending a special value and then interpreting that special value as a NULL in the different parts of the code. For instance, if we sent a string: "__NULL__", then we could test for the data being a string, and it's value being '__NULL__'. It's possible that a call would have that as an argument, but not very likely.
If we decode this as the true NULL, then we're OK. Not easy given all the places that I'd like to have an argument or return value be NULL, but it's possible.
Step 6 - Dealing with Dates
NSDate values can be passed through - but they are passed through as string values where the NSDate is expanded out to it's human-readable value. This is OK, but it means that we need to have quite a bit of standardization on the format across different machines and platforms, and it means that we have to take the time to decode it and encode it on every call. Non-ideal.
Far nicer would be to send milliseconds since epoch as the date and then reconstruct the dates on the other side. This is easily done in Obj-C using the NSDate class:
// build up the parameters to pass into the call - the local 'as of' NSMutableDictionary* params = [NSMutableDictionary dictionaryWithCapacity:3]; [params setObject:[NSNumber numberWithInt:aPID] forKey:@"pid"]; [params setObject:[NSNumber numberWithDouble: [aDate timeIntervalSince1970]] forKey:@"asof"];
I essentially turn it into a simple 'number' and then pass it through without any problems. On the other end, we have to worry about how to interpret this data properly. After all, it's not a date, it's a number of milliseconds since epoch, and that means that in PHP and PostgreSQL we need to account for this. What I found was that it was fairly simple to do something like:
function removeData($method, $args) { // first, get the SQL that we need based on the method name invoked if ('removeIt' == $method) { $sql = "delete from meas where pid=" . (int)$args[0] . " and mid=" . (int)$args[1] . " and extract(epoch from taken)=" . (double)$args[2]; } // now we need to open up the database, make the call, get the response $conn = pg_connect('host=localhost port=5432 dbname=primary'); if (!$conn) { $retval = "no db"; } else { $result = pg_exec($conn, $sql); if (!$result) { $retval = "no result for: " . $sql; } else { $result = pg_exec($conn, $sql); if (!$result) { $retval = "BAD"; } else { $retval = "OK"; // make sure to free up the result pg_free_result($result); } // make sure to close up the database connection pg_close($conn); } // make sure to close up the database connection pg_close($conn); } return $retval; }
Finishing Up
This has a lot of partial code, but it's all in the git repos for the project, and if you really need it, let me know. However, I think this is a great start for what you need, and a little experimentation should get you where you need to be. It's not perfect, but it's pretty close.