Back as solo mobile developer - not easy

Being the solo mobile developer at a job is not easy. I had just gotten used to having a couple of developers around me at my previous contract position. Everyone had bits and pieces of experience in different areas so you would quickly get answers to questions without doing a Google search. They might have some sample code laying about from another project or just know how to do it.

Sure the big boss here wrote the initial iOS code that I have been cleaning up and adding features against but I rarely get to speak with him and he has no Android background. I have been working on the Android port this week getting stuck on silly things that the guy I worked with at the last place would have handled with ease as he had a lot of background in those areas. Now that the core networking is in place I can start moving forward at a pretty good clip.

A real feeling of isolation when you have developers all around you but they either don't know the language (Objective C) or the framework (Android SDK) thus you can't really bounce any ideas off of them. Sure I can hit them up for general business knowledge or Java questions but not for the meat of the development cycle.

For my Android work I have been using Android Studio. I know it is a 0.1 release but that is really just the Android plug-in side of things. The over all IDE is at version 12 and I have enjoyed using IntelliJ in the past. I like the dracula theme as far as colors go.

The ability to edit layout XML and see the results side by side in the preview window is super handy. I have gotten that to generate crash reports from time to time but they did not kill the IDE or lose any of my work.

What I miss from Eclipse is the new class wizard. It asks for the base class and interfaces and lets you quickly build things out. I have asked about this on the IntelliJ forms and they basically stated you can do all of that via a number of steps. I want a Wizard. I also want the auto add of the Activity to the manifest. I should not have to do all that work manually every time I add a new activity to the project. Since this is a new project that has been happening a lot.

The Log Cat filter seems pretty iffy too. I turn it on to filter out things but it forgets about that between program runs so I have to go back to "No Filters" then back to my filter again. It is silly how much junk the emulator kicks out into Log Cat. Since I am using the HAX high speed emulator I get a ton of extra stuff.

I was happy to get the upgrade to 0.11 this morning. Good to know there is active work on Android Studio and I am sure it will continue to improve. I have been using it at home on my Win7 64 bit PC and my Macbook Pro at work. Both seems to run equally well.

Core Data stinks for bulk inserts - direct SQLLite is much faster

I was handed an iOS app that was using the basic JSON handlers and keeping everything in memory. This was slow and running out of memory for large requests.

I converted the project to use stream based JSON parsing stored to a Core Data database backed by SQLLite. This solved the memory issues allowing all requests to work. There are 5 tables involved with a total of 239,000 records if you requested all the data from the server.

The problem - that request took 5 minutes and 43 seconds to process on an iPad 2. It took 33 seconds on the simulator. Let's state this again - Always test on a device. That kind of time is totally unacceptable.

After sniffing around the net and attempting to speed up Core Data I determined that Core Data sucks at doing bulk inserts. Core Data is really nice at hiding the gut level fun of SQLLite but for large inserts it is not the way to go.

I converted everything to use SQLLite directly and saw a massive speed increase. I can now process the same record set in 1 minute and 23 seconds. That cuts a whopping 4 minutes and 20 seconds off the time. The simulator went form 33 seconds to 12 seconds.

A typical user would rarely if ever request all the records but testing worst case scenario is really the best way to go. The speed improvement is seen across the board in the app with similar percentages of improvement. I am very happy with the results even though it took a couple of days to get it all sorted out.

As for the code there are areas that are more involved than the Core Data version and there are areas that are actually a lot cleaner. Everything is a trade off and there is no clear winner.

I had never used Core Data before. It was a really good learning experience to set it all up using the GUI editor in Xcode and finding out what it took to add the Core Data library and support code to an iOS project that was created without it originally. I found a lot of Core Data really straight forward and easy to use. I will not shy away from it on other projects unless they involve bulk inserts.

Having used SQLLite in the past I had a basic object set to start with and I knew my way around FMDatabase. My previous experience did bulk insert processing and I wrote both iOS and Android versions so I stuck with SQLLite to make it easier to convert code back and forth. I wanted to learn something new this time around so I went with Core Data. Good thing I could quickly fall back on my direct SQLLite experience to solve the speed issues that arose.

Mobile app usage and development - do you just have to magically know what to do?

The more mobile development I do the more I am finding you are just expected to know things magically. There are very few help screens, tips of the day or on screen tutorials. There are so many potential gestures in any app that you need to magically discover. What can we do to make this situation better?

A lot of apps allow swipes to perform an action. Swipe left to call, swipe right to text. Long presses may do something or bring up a menu. Some actions are instant and others prompt you but you are never really sure what will happen. Delete should prompt. Add a new item most likely does not need to unless it will cost money i.e. send a message or use your data plan or will take a long time to perform.

Some will answer the mobile device is so small and we all want to save on code size and people never use help or only use it once so skip it. The original apps were very small and had few features so that worked. News apps have a ton of features. We basically have computers in our pockets and we do a lot of work on them.

Do we need a standard "Cool Features" button showing quick help? Maybe it should show a standardized overlay on the screen to explain the swipes and long presses so people can get more out of our apps.

The other side of this is developer tools. They have a boat load of features too and a lot of their power is there but hidden from the coder. I know Xcode drives me crazy with its lack of right mouse button menus. I expect to be able to perform most actions a context sensitive menu. In general the menus are very terse. Want to add a view controller to a story board? Right click on the story board and you get zooming only options. You have to know to drag a view controller for the Utilities panel. But first the Utilities panel has to be visible. That is the only way to perform this operation. That is not friendly.

Speaking of the Utilities panel it has 6 tabs on it and I seem never to go to the correct one to do what I need. Do do nearly anything you shift between 2 or 3 of the tabs. Somehow I would like the most common settings on a single tab that I could define.

Right click on a segue control and you get zoom options but nothing associated to the segue. Click on the Scene Name in the left hand panel and nothing at all happens. I excepted the editor to scroll to that Scene. You have to click on a view of the scene for that to happen. Oh and it changes your zoom factor when you do that. Just scroll there! If I was zoomed out I want to stay zoomed out!

Is the layout of your windows wrong? Can you right click on something and fix it? No, you have to go to View -> Assistant Editor and pick the layout you want. I did not even know the window I was looking at was called Assistant Editor as nothing on screen indicates that. You can mouse hover over the toolbar button to find that out. Those same buttons are inconsistent. The first three toggle between modes of a view and only one can be pressed at any time. The next three toggle the visibility of views and any or none of them can be pressed but the two sets visually they look identical.

Highlight a word in the editor and hit Cmd+F most editors will automatically put the highlighted word in the search box but not Xcode. I get to copy / paste it up there. Heck I can highlight a word in Eclipse and just hit Ctrl+K to find the next occurrence skipping the Find prompt altogether.

Add files to a project and guess what? They are not flagged as needed to be added to version control. Why? Right click on them to add them and that menu is not there. I have to fire up SourceTree to add them to GIT for this to work. Xcode version control integration is crappy at best.

Windows Update error 800B0100 - I finally just gave up

I have been battling Windows Update Error 800B0100 for over a week. I searched the web over and over and tried pretty much every solution presented. I don't know how many times I ran the Windows Readiness Tool, update fixer, fixer fixer, Mr. FixIt, delete this, delete that, uninstall all of these, manually install those, etc.

Nothing made it work. I finally just blew away the VM - this was Windows 7 Professional on my Mac, and did a reinstall from DVD. After many updates it all appears to be working. I have never run into this sort of issue in the past where Windows got so messed up that I had to give up.

Another odd thing, I needed .NET 4.0 so I download that. Right away I have to download 16 patches. This means MS does not keep the installer updated. Must be the original install then you have to download all the patches. Seems a waste of bandwidth to me. I should get a pretty recent version of the .NET 4.0 installer when I download it. I don't expect to download Chrome, 7Zip, Sublime Text or others just to turn around and have its updater run and get more stuff. To me it makes MS appear lazy.

At least it is up and running now. I needed to install IIS but with the update issue that was not possible as the windows features list just came up empty. Now it fills in and I could install IIS. Turns out that was for naught because the final piece of 3rd party software was holding all its errors and only showing me one at a time. It complained about IIS but once that was installed it turned around and told me my OS was not supported. Gee, seems that is a much more important error message. You really should from the worst situation forward or show a list of issues instead of just the first one you find.

Now we have to find and configure a Win Server 2008 to get this all rolling. Guess I will go back to some iOS clean up coding while I wait for that to happen.

Using YAJL with Objective C

I have been tasked with getting an existing iOS code base ready to be released to the App Store. After loading the existing project into Xcode and resolving static library link issues I was able to run the program but not log in. I added some #ifdef DEBUG processing to allow me to log into a server without a valid certificate and got to play around in the app on the simulator.

I was quickly able to crash the app and after setting the debugger up to stop on all exceptions I found it was running out of memory. Boy, if you can run the simulator out of memory you know you can easily do that on a device.

Looking through the code I found it was using NSJSONSerialization for all JSON processing. That library is fine and it is provided by Apple so you know it is legal to use but when you have large amounts of data it is not super efficient. If you have used XML in the past and are just getting into JSON this is similar to using DOM vs SAX. DOM is great and easy and hands you a nice tree of data but if all you are doing is plucking things out of that tree to put in another format you are better off using SAX especially if there are memory concerns.

The code was parsing everything into a big NSDictionary based tree with 13,318 base records and tons more child records. This is just not going to work on a memory limited device.

I looked up information on stream based JSON parsers and landed on YAJL which you can find at https://github.com/gabriel/yajl-objc. I won't cover how to configure it for Xcode as all that information is on the base site. It was easy to set up for both Mac OSX (my test program) and iOS.

They show a streaming sample that was too basic to be very helpful. Hopefully I can fill in some of the gaps. The first issue with their sample is they read the WHOLE file into memory via NSData then run it through the parser. Not fully streaming at that point. Second they show no samples on what to do in the callbacks.

The code snippet below will help you parse your JSON by keeping track of where you are in the data. It will keep a stack of named dictionaries and arrays as you parse through the data. You can then create and act on the proper objects as you get map keys and values.

It also reads in the JSON file in chunks instead of tossing the whole things into NSData. In my final iOS code I am using a NSURLConnection and parsing the chunks in the didReceiveData call.

The original code ran out of memory when receiving 38.9 meg of data and took 5 seconds to run. I put in some cheater try / catch blocks to let it "finish" without crashing for timing results. The new code does not run out of memory and runs in 2 seconds. Of course this is all in the simulator on a fast MacBook Pro with a lot of RAM. I don't currently have a device to test on but will get one soon.

I looked around on the web for a long time before I was able to piece together enough knowledge to pull this off so hopefully it will help others get up to speed a lot faster.

Things that go in the header file

// Array used as a stack to keep track of where we are
NSMutableArray *stack; // create this in init method
NSString *mapKey;

// Actual class code

- (id) init
{
    self = [super init];
    if (self != nil) {
        stack = [[NSMutableArray alloc] init];
    }
    return self;
}

// Code to parse the file in chunks

- (void) parseFile:(NSString *)jsonFileName
{
    YAJLParser *parser = [[YAJLParser alloc] initWithParserOptions:YAJLParserOptionsAllowComments];
    parser.delegate = self;
    uint64 offset = 0;
    uint32 chunkSize = 10240;     //Read 10KB chunks.
    NSFileHandle *handle = [NSFileHandle fileHandleForReadingAtPath:jsonFileName];
    NSData *data = [handle readDataOfLength:chunkSize];
    
    NSLog(@"Starting parse of %@...", jsonFileName);
    [stack removeAllObjects];
    mapKey = nil;
    while ([data length] > 0)
    {
        [parser parse:data];
        if (parser.parserError)
        {
            NSLog(@"Error:\n%@", parser.parserError);
        }
        offset += [data length];
        
        [handle seekToFileOffset:offset];
        data = [handle readDataOfLength:chunkSize];
    }
    
    [handle closeFile];
    parser.delegate = nil;
}

// Stack adds newly named object OR last object on stack
- (void)parserDidStartDictionary:(YAJLParser *)parser
{
    NSString *dictName = mapKey;
    if (mapKey == nil)
    {
        dictName = (stack.count == 0) ? @"" : [stack lastObject];
    }
    [stack addObject:(dictName)];

    // Create new object here based on dictionary name
    // ready to hold soon to come key + value pairs
}

// End of dictionary - pop off stack
- (void)parserDidEndDictionary:(YAJLParser *)parser
{
    mapKey = nil;
    [stack removeLastObject];

    // Save object created in start Dictionary to
    // array or core data here
}

// New array staring, push name onto stack (use previous if we don't have one)
- (void)parserDidStartArray:(YAJLParser *)parser
{
    NSString *arrayName = mapKey;
    if (mapKey == nil)
    {
        arrayName = stack.count == 0 ? @"" : [stack lastObject];
    }
    [stack addObject:(arrayName)];
}

// End of array, pop off stack
- (void)parserDidEndArray:(YAJLParser *)parser
{
    mapKey = nil;
    [stack removeLastObject];
}

- (void)parser:(YAJLParser *)parser didMapKey:(NSString *)key
{
    mapKey = key;
    // Do processing here for each map key
    // Something like
    // if (!inResults && [key isEqualToString:@"results"])
    // {
    //    inResults = true;
    // }
}

- (void)parser:(YAJLParser *)parser didAdd:(id)value
{
   // Do processing here for each value (will have a mapKey)
   // Something like

   // if (inResults && [mapKey isEqualToString:@"query"])
   // {
   //   query = ((NSString *)value);
   // }
   //
   // value will be (NSString *), (NSNull *) or (NSNumber *)
   // booleans come as (NSNumber *) 0 = false, 1 = true

}

Is there really any such thing as a typical day?

I have been at my new job for a little over a month now. I have had the chance to learn a lot of new things. Generally I have been a front end developer doing the GUI and writing custom controls. Of course that does not mean I don't get to do any middle or back end coding. I use SQL where needed, write business logic handling objects, etc.

Here we are using Spring, Hibernate and Postgres for the database chores. I am finding my way about on all those annotations and have sent off and received requests for new table schemas to the DBA.

In a given day you can find yourself using little bits of each framework in a few hours time. I started the day setting up a chron job to call some existing code that calls two different 3rd party APIs to get some data. One of the vendors uses XML and the other JSon. The objects are created via Spring and we use Hibernate to interact with the database. I was using various internal SDK calls to do most of that with all of the work being done in Java.

Once that was up and running I was asked to look at some existing Objective C iOS code to see how close it was to being app store releasable. First off I was not able to login in as the servers I have access to don't have valid certificates on them and our API is HTTPS based. A browser will let you ignore the suspect certificate after a warning message. I added some #ifdef DEBUG to allow the Connection code to accept any certificate. That got me into the app and allowed to try out rest of the GUI.

Sadly I crashed the app pretty quickly. Did not know initially if it was bad memory management or if it ran out of memory. Turns out it ran out of memory. I added a @try block around things so it could push itself along a bit farther and I was able to run everything.

I then started to research why it ran out of memory. I looked at the server request it was making and tried it in a browser. A huge pile of data came back so I ran used curl to grab the data and write it to a local file. Turns out there was 38.9 meg of JSon coming back.

I wanted to sniff around in the JSon so I asked python to pretty print it and sent that to another file. I then opened it up in Sublime 2 to see what I had. Straight forward JSon, just a lot of it.

Next I wanted to see how the code was handling the JSon. Turns out the developer used the standard NSJSONSearialization calls in iOS. This is like using the DOM in XML in that you get it all in memory and they you double it up or more by converting to your internal object format. It was the easy way to go but not the most optimized way to handle things.

Time to hit Google and see if there is a streaming JSon parser. I found YAJL which I plan on experimenting around with today. I also think we should write the results out into a database instead of holding it all in memory. I have written similar code, it was XML and not JSon, storing to SQLite at a previous position so this should be fairly straight forward.

So was this a typical day? Yes, in that I ended up using a pile of different languages, data formats, command line tools and frameworks / SDKs to get my job done. You really need to branch out and learn new things.

I played around with Python in my spare time at home as I kept hearing about it and now I have used it multiple times. I used it at my previous job for a quick error log parser. I have used it here because some sample 3rd party code was written in it and I needed to convert that to Java. I was able to run the Python code and tweak it until it worked before I converted it to Java. I have used python to pretty print JSon files.

Curl was something new to me here. I have used it a lot with the 3rd party vendors as one did all their sample code it in. Curl makes it very easy to grab server results and shove them into a file. It works great with HTTPS as you can pass the user name and password in the command along with ignoring invalid certificates which seems to be a recurring need.

I have used Java for years so that is second nature. I have not used Spring or Hibernate in the past. I am really starting to see the power of the annotations. With great power comes great responsibility though. I am learning when you need to put code in other objects so it can be properly initialized. Anytime you use a new framework you find new and interesting ways to crash your code.

There is very little iOS knowledge here so I was able to step in and handle that too. A bit of a mind shift to get back onto Objective C and it will take some time to remember the syntax, framework and ins and outs of Xcode. Keeps you on your toes to shift around between languages, IDEs and frameworks. Good thing my Mac has a lot of memory, disk space and a large second monitor to keep all of this open and ready to roll.

Just to add to the mayhem I am working on updating my Android game at home. It does not scale well to tablets so I am converting the graphics to vector based allowing it hopefully work on nearly every device. At the end of most days I am pretty mentally fried.

Motorola Xoom aint what she used to be

I have had the Motorola Xoom for a couple of years. One of the early Android tablets. It is dual core but boy is it showing its age. Part of this is due to my quad core Note II. Everything on my phone is really snappy. Things on the Xoom used to be snappy but now everything seems slow. Boot up time really stinks.

Button presses to get to home screen or bring up list of running apps are pitiful. I still use the tablet a lot for content reading. That works just as well as it always has although the screen is not super bright nor does it have the great colors of the Note II. I enjoy doing crossword puzzles more on the tablet as you can see more of the clues and all of the grid.

Not ready to get a new tablet just yet though. The Xoom works for its specific purposes. The direct plug-in charger quit working but the base charger I ordered from Amazon works just fine. If that goes out then the tablet will be useless.

Next time I will look in a smaller than 10" range but I don't know if 7" is big enough. Probably something in the middle. At least on the Android side of things there appears to be a nice variety to choose from. The current size / weight can make it a bit tough to hold one handed for extended periods of time. I am usually petting a cat while reading articles or using my other hand to one finger type crossword puzzle answers.