Android vs iPhone - SQLite and Table Views

I am still in the process of converting my iPhone project to Android. Main goal is to allow doctors to create cases out in the field and submit them to the back office for case workers to finish up and send off to insurance companies to get paid. All the rest of the software is written in Java and Ruby and has been operational for a number of years.

There are currently 24 of tables involved in the process with some containing over 17,000 records. I am storing as much of the data on the device as possible but some of it needs to be requested per login session as it can change on the back end - facilities, referring physicians, providers, etc.

The basics of getting the table data in XML format, converting it via SAX parsing to table rows and executing inserts to populate the tables is very similar between the platforms. Where the fun begins is displaying the data to the user.

The iPhone has a standard Table View but it is up to you to do all the work. You have to build out the rows, you have to get them displayed, you have to deal with rows that have different amounts of data. You have to convert clicks back into rows of your table. It appears one developer did the SQLite side and another did the Table View side but neither talked to each other. Doing all the work yourself is not fun. I have hundreds of lines of code to handle what I am doing trying to be generic as possible. Even showing a simple string when you have an empty table is a pain. Finally you have the fun of performing the SQL queries in a simulated background thread. You are responsible for disabling the UI, showing the busy spinner, positioning the busy spinner and if you feel really adventurous showing some status text.

Moving over to the Android has been a pleasure. Sure, I have the whole database schema in place giving me a big head start but that is not what is making it so easy. The Android team decided to do a lot of the hard work for me and provided a SimpleCursorAdapter. I can query the table, get a cursor back and pass it to this adapter and my table automatically populates. In a ton less code I actually have better functionality. The rows automatically size to the strings I provide. If my cursor has no rows a simple addition to the XML file shows the "No matches found" message. Background activities automatically take care of disabling the UI and I get to show status strings with ease.

Things are falling into place so quickly on the Android side I am able to add extra features as I go. This equals some pain as I will get to back port those to the iPhone before shipping.

This is not the only area where I have found Android to have a more complete API. I can quickly create two views, one for landscape, one for portrait and have them appear without any special code. Doing that on the iPhone is a major hassle. You can either shuffle the controls around in code, which is what I have done, or you can attempt to have to XIB files and do double control hookups which seems like a massive waste.

The code base I am generating on the Android side is massively smaller than on the iPhone. Pretty much anything I do takes less lines and much shorter lines of code to accomplish. Objective C is not a terse language. The less I have to type the faster I can get code done even with auto-complete available on both platforms. I also don't mind refactoring code in Eclipse where it is s crap shoot in Xcode. If I have to do anything massive I do it in AppCode which helps a lot. Not having two files for every class is also nice, cuts the size of the code tree in half.

It is going to be interesting to see what the doctors think. Many have iPhones but are thinking of getting Android tablets to do their case work. Why an Android tablet? Because you can get it in a 7" form factor and that size fits in a lab coat pocket where the 10" iPad does not. My job is to be cross platform and as feature equal as possible. This gives them a wide variety of device choice between iOS and Android.

Beta testing will be straight forward on the Android. I can send them the file and have them install it or they can stop by my cube, plug in the device to my USB cable and I can copy it right over after I kick the device into debug mode. iPhone, well buddy you better tell Apple about the device, wait for it to get provisioned, build the code with the provision in place then copy it on to the device. If you want them to install it they have to play funky monkey with iTunes. Can't go over a 100 device limit either without buying a second license.

The other area of fun will be release time. The initial version of the app just handled appointments and not completing the case. I was able to release the Android Market in 10 minutes. It took a full week for Apple to approve my app and have it hit the iPhone App Store. If I want to fix a bug on the Android I fix it and release it on my own. I have to restart the process on the iPhone side. With the current, very simple app, out there I have not had to do a bug release. The new version is much more complicated which increase the odds of nasty bugs.

If we update our back end we can easily sync that with an Android release. With the iPhone we are in the total dark even if I am really sure I have not used anything in the code that Apple with frown upon. It could take weeks to get something accepted. Can't even do a proper marketing campaign when you can't give a real release date. Apple is not really enterprise friendly and they are not really all that developer friendly either.

I am pretty happy with what I have been able to pull off on the iPhone. It looks good and according to QA is working nicely.  I am very interested to see how the Android side turns out. So far it is matching the iPhone punch for punch and coming out better in a number of areas. Any time I have less code to write I know I will have less bugs and thus happier customers.

Lessons learned by programmer doing the work of a graphics artist

At my previous position we had a graphics team that handled all the icons and other art needs. Wonderful group of people and they were very open to you handing them a less than spectacular image as a base idea and then making it into something professional. They also made sure icons followed a theme. At times they threw away my idea and went another direction. No big deal to me, they are the professionals.

Jump into my current position at a different company and we had a graphics artist but he has decided to move on and they are not in a hurry to replace him. He will do some freelance work for us and the rest they will farm out. That pretty much leaves me to do all the graphics work for the iPhone and Android devices on my current project.

I know my way around various graphics programs so this is not a huge deal and it does give a nice break from coding. I don't need a ton of assets and I have just enough of an eye for visual design that everyone has been pretty happy so far with what I am churning out. I do hit the web to find as much free art as I can. I always make sure it is free for commercial use. They rest I draw myself.

What are the lessons I have learned?

1. Always find the asset in the largest format possible. Sure you will be scaling it down but scaling down beats the crap out of scaling up any day. If the download page has it already scaled in multiple sizes grab all of them as a lot of time an artist will scale from a vector image giving much better results than a simple pixel scaling.

2. Once you find the asset in the large format save it in that format and only edit copies. I have screwed this up multiple times. Maybe I worked on a copy first then deleted the original or put it in some obscure directory. At times I find something while on the Mac and copy it to the PC to edit it or do the opposite. Then I can't find the original when I want to scale it again or tweak a color. The best solution is to have an assets area in your version control system to keep the originals.

3. Be consistent. Don't have a round add button (+) and a square [-] delete button. If one is gradient the other should be gradient too. Use consistent background images and colors. Make it look like all the screens belong to an app written by a person and not like it has been written by 10 people who never met each other.

4. Don't be afraid to throw away images even if you have done a lot of work on them. They might have looked good when you first put them in but the rest of the system has surpassed them so that one asset you loved now looks like it was drawn with a crayon.

5. Get feedback and accept it. If every person you show the app too says "Dude that background is ugly" then fix it. You may love it and may have done too many hours of work on it but the feedback of the masses is generally correct. If they can't read the white text on the background image change the text color until it is readable don't question their eyesight.

6. When looking for assets download anything that you think looks cool even if you don't have a current need for it. Never know when it will come in handy and finding it again might be impossible. I have directories full of potential icons I don't need right now.

7. Learn how to use various paint programs. I use Paint.NET, PhotoShop Elements, GIMP and some other utility programs. I was transferring everything form my Mac to my PC for editing. That was getting silly especially for simple resize and clip operations. I installed GIMP on the Mac, it is free and very full featured. MS Paint really is not what you should be using especially with so many free options available. You can use GIMP on pretty much any platform for free so it is a great place to start. Each program has strengths and features others don't and they all handle the same basic file formats so learn to use more than one.

8. Layers are massive time savers so learn and use them. If you watch a real PhotoShop pro like the graphics artist at my last job they have a lot of layers going even for something as a simple icon. It allows you to tweak various aspects of the image without redoing all the work or messing up the background. For the home page of my current project I wanted icons that appeared to be sitting on shelves. I have multiple layers: background, each shelf, each icon and each piece of text under each icon. I can quickly change the text or text color without screwing up my background. I can move elements around. I can resize the shelf. If this was a flat image that would take forever. You can save as a flattened image once you are done but keep in mind lesson 2 and save the PSD (or layer format of the software you are using) first then save the flattened image as a PNG. Don't lose your layers!

9. Nine-patch is a cool format for use on the Android. Make sure you investigate it. It allows you to quickly create scalable images when working on Android devices. Android docs on nine-patch is a good place to start. I am using this format for the shelves as there are various device sizes along with landscape and portrait orientations to do layouts against. Nine-patch is perfect for this situations.

10. Don't be afraid to experiment, it is the only way to learn. You may not be a graphics god but I am sure you can get place holder art ready to at least express the action of the icon. There is plenty to grab from the web even if it is just temporary. Everyone should know how to load an image and size and scale it. I cringe when I see a web page load super slow only to find out they loaded a 1 meg image and allowed the browser to scale it to 200x200.

11. Appreciate your graphics artist if you have one available to you. Playing around in a paint program is fun from time to time but you will soon realize it takes a lot of talent to produce professional looking applications. Back in the day everything was green, white or orange text on a black background. That does not cut it any longer.

12. Don't go overboard. Everyone has seen an application that looks like someone chugged a couple of cans of paint and threw up. Every label in a dialog is bold or italic and has a unique color. Use color to indicate something like an empty required fields but don't just use colors because you can. Multiple fonts can really mess with your eyes. For the love of {deity of choice} never blink / flash anything. I will stop using software instantly when that happens. I get migraines from strobes. I can't eat at Joe's Crab Shack or go with my kids to the roller rink with the disco ball. Blink screams amateur.

Switched to libxml2 ~ 25% less time and a lot less method calling

Since I was still disappointed with the XML parsing speed on the iTouch I decided to give libxml2 a shot to replace NSXMLParser. I got a 25% boost in processing speed, less memory thrashing and less method messaging.

iTouch Rev 1 hardware
Original TouchXML code (full DOM) 79  seconds
SAX parsing NSXMLParser           42  seconds
SAX parsing libxml2               30  seconds


iTouch Rev 3 hardware
SAX parsing libxml2              7.5 seconds

This is for 17,000 XML records in NSData format in memory to be written to an SQLite table. I still wish it could run faster but a speed up from 80 seconds a table to 30 seconds is pretty darn massive. In the simulator it takes 0.5 seconds. I have a feeling it could be a bit faster if I wrote out the records in this code instead of doing the callback but this is much more generic and reusable.

This was much easier to write than I thought it would be although it took a lot of research to get just the lines I needed. The pure documentation for libxml2 is not aimed at Objective C programmers so I had to find bits and pieces here and there. One site would talk about parsing from a URL and another how to do it from memory. One would show a big structure with statics for callbacks and another with a SWITCH / CASE statement. None of them showed how to clean up memory at the end.

Here is what the final code looks like. Its job is to take XML looking like this crappy made up sample (real stuff is diagnosis codes from doctors but the structure is the same):

<itemnames>
  <itemname>
    <id>10</id>
    <name>soup</name>
    <desc>soup you fool</desc>
  </itemname>
  <itemname>
    <id>15</id>
    <name>cat food</name>
    <desc>food for your cat</desc>
  </itemname>
</itemnames>

Passing in @"itemname" for elementName your SAXParserDeletgate would get two callbacks, each with a dictionary holding the following keys: id, name, desc and their associated values. As I get the callback I convert the dictionary items to an SQLite insert command (that code is not shown).

This is a very special case where I am simply getting a list of data. The only items in my XML data are records. All the data is in the TEXT area. You should be able to look up the XML_READER_???? info to parse attributes or other aspects of the XML. Already feels like I spent more time trying to put this into a blog posting than it did to get the code working but I don't want my efforts to go to waste.

This is my first attempt a using a syntax highlight editor in the blog so sorry if any of the code is screwy when you copy / paste it out of here. I picked C++ syntax as the closest as the syntax highlighter helper code I am using did not have anything specific for Objective C.

I really hope this can help someone out and show you how easy it is to actually use nearly straight C code to help speed up Objective C when parsing large XML data record sets. I ran it against the memory leak checker and it came out clean. The key was the final call to xmlFreeTextReader(reader);


If it does help please post a simple thank you comment. Always nice to know when the effort makes the programming life of another easier.


You MUST go to Project - targets (your app) -> Build Phase -> Link Binary with Libraries -> (+) to add libxml2.2.7.3.dylib to your overall iPhone project to be able to use libxml2. I am using Xcode 4.03, don't know how to do that in older versions but I assume it means adding it to the framework area. Doing the (+) way in Xcode 4 auto added the headers to the correct path. If it is not version 2.2.7.3 I don't think that will be an issue. It appears I am using some pretty basic libxml2 calls.

Call back delegate

//  SAXParserDelegate.h

#import <foundation/foundation.h>

@protocol SAXParserDelegate 
- (void) SAXDictionaryElement:(NSDictionary *)dictionary;
@end

Header file

//  SAXParser.h

#import <foundation/foundation.h>
#import "SAXParserDelegate.h"

@interface SAXParser : NSObject {
    id<saxparserdelegate> saxDelegate;
    NSData *data;
    NSString *name;
    NSMutableDictionary *dictionary;
    
    bool inZone;
    NSString *keyName;
    NSString *valueData;
}

- (id) initWithData:(NSData *)xmlData elementName:(NSString *)elementName delegate:(id<saxparserdelegate>)delegate;
- (void) parse;

@end

And finally the parsing code itself

//  SAXParser.m

#import "SAXParser.h"
#import <libxml/parser.h>
#import <libxml/tree.h>
#import <libxml/xmlreader.h>

@implementation SAXParser

// Initial with data, element name to find and callback delegate
- (id) initWithData:(NSData *)xmlData elementName:(NSString *)elementName delegate:(id<saxparserdelegate>)delegate{
    self = [super init];
    if (self != nil) {
        data = xmlData;
        name = elementName;
        saxDelegate = delegate;
        inZone = NO;
        
        dictionary = [[NSMutableDictionary alloc] init];
    }
    return self;
}

// Clean up any memory we used
- (void) dealloc {
    [keyName release];
    [dictionary release];
    
    [super dealloc];
}

// Start the data parse
- (void) parse {
    xmlTextReaderPtr reader = xmlReaderForMemory([data bytes], [data length], NULL, NULL, 
                                                 (XML_PARSE_NOBLANKS | XML_PARSE_NOCDATA | XML_PARSE_NOERROR | XML_PARSE_NOWARNING));

    if (!reader) {
        NSLog(@"Failed to create xmlTextReader");
        return;
    } 
    
    while (xmlTextReaderRead(reader)) {
        switch (xmlTextReaderNodeType(reader)) {
            case XML_READER_TYPE_ELEMENT: {
                NSString *elementName = [NSString stringWithCString:(char *)xmlTextReaderConstName(reader) 
                                                    encoding:NSUTF8StringEncoding];                
                if (!inZone && [elementName compare:name] == NSOrderedSame) {
                    inZone = YES;
                    [dictionary removeAllObjects];
                    [keyName release];
                    keyName = nil;
                } else if (inZone) {
                    [keyName release];
                    keyName = [elementName copy];
                }
            }
            break;
                
            case XML_READER_TYPE_TEXT: {
                if (inZone && keyName != nil) {
                    NSString *string = [NSString stringWithCString:(char *)xmlTextReaderConstValue(reader) encoding:NSUTF8StringEncoding];
                    valueData = [string retain];
                }
            }
            break;
                
            case XML_READER_TYPE_END_ELEMENT: {
                if (inZone) {
                    NSString *elementName = [NSString stringWithCString:(char *)xmlTextReaderConstName(reader) 
                                                               encoding:NSUTF8StringEncoding];                
                    if ([elementName compare:name] == NSOrderedSame) {
                        [saxDelegate SAXDictionaryElement:dictionary];
                        [keyName release];
                        keyName = nil;
                        inZone = NO;
                    } else {
                        if (valueData != nil) {
                            [dictionary setObject:[[valueData copy] autorelease] forKey:keyName];
                            [valueData release];
                            valueData = nil;
                        } else {
                            [dictionary setObject:@"" forKey:keyName];
                        }
                    }
                }
            }
            break;
        }
    }
    
    xmlFreeTextReader(reader);
}

@end

Android conversion continues

I continue to pound away on my Android conversion of the iPhone app I am close to finishing up. I have been working with SQLite on both sides. The wrappers are different which hosed me over for a bit today.

iPhone
[database beginTransaction]
... db operations
[database commit]

Android - WRONG
database.beginTransaction();
   ... db operations
database.endTransaction();

Android - CORRECT

database.beginTransaction();

   ... db operations

database.setTransactionSuccessful();

database.endTransaction();

Yes, you need to begin and end but you MUST call setTransactionSuccessful or kiss all the operations goodbye.

I added a new call to the current XML Pull Parser we are using on the Android side. It allows you to send in a callback to get each row of data as it is processed. For our big boy Java app we just parse everything into memory then write from memory structure to DB table. Not a good idea on a mobile device as I discovered on the iPhone. I use the basic NSXMLParser there and build one record at a time into a NSDictionary. I added a bit of logic into our Pull Parser for the Android allowing a callback interface with a Map of values for each record. Saves a ton of memory and time under each OS. I was very happy how easy it was to do that conversion. I have begun my testing using the largest table I download so far, it has 17,401 records.

Timings
iPhone Simulator   3.7 seconds
iTouch Rev3       42.7 seconds


Android Emulator  41.0 seconds
Samsung Galaxy S  11.8 seconds

As you can see the iPhone simulator can really fool you into thinking your code is running great. On the Android side the emulator forces you to code tighter and the phone gives you a nice speed break. I am not at all happy with the 42.7 seconds it takes the iTouch to parse and store the data. Not sure what I can do to fix it but I will have to play around with it more soon.

Luckily we only need to download the big tables when they are stale, which it will be once a year. I get a list of tables and last update date / times from the server on login. I compare my local last update to the server and grab a newer copy if it is out of date. We also have session tables that I have to load post login every time but I only want to do that once a session so I have all kinds of fun logic to decide if a table is stale. Seems to be working on the iPhone so I will start on that conversion next.

I really wish the Android device emulator ran faster. Total switch from the iPhone. iPhone the simulator is very fast. Of course the simulator is using same OS as the MacBook Pro so it really has a lot less to do. On my PC the emulator has to emulate a Linux type OS and the device. It is much faster to run on Android the actual device but of course that is a pain in other ways in that I have to type my password on the silly thing over and over.

The debugger is much slower under Android too. It is more usable as I can just click on variables and see their values or highlight a section of code and do an Inspect to see the value. Typing in commands to GDB on the Mac sucks and if you want to see some UTF8 text you have to type in a long command to convert it into something usable. Don't miss that at all.

Command to convert UTF8 NSData block to NSString in Xcode
po [NSString stringWithUTF8String:(char *)[data bytes]]

Pretty simple to tag my debug output strings and filter them via LogCat. DDMS lets me look at the file system of the emulator with ease but I have to copy my DB to my local HD browse it. Much easier to use SQLite Manager under Firefox on the Mac to look into the simple, but buried rather deep, directory structure of the simulator on the Mac to grab the .DB file and check it out.

I thought some things would be more straight forward to port especially since I was using SQLite on both devices but the wrapper classes are just different enough to cause confusion. Once I get in the base calls in my helper areas I will not even have to think about it.

Just as I expected most of the code is a lot shorter on the Android side.

• Not dealing with both a H and M file cleans up what you have to see in the code directory. Calls are less wordy as you are not naming parameters. 
• Much easier to set up static variables and enums are really enums and not globals. 
• Date processing is so much easier I am able to in-line the code as it is a few lines where Objective C was too many lines so I put things in separate methods. 
• You can fully qualify what you are putting in a Map so I don't have to typecast things. 
• No matched alloc / release pairs or autorelease calls on end of temp variables

Other random things

• SVN interaction works in Eclipse but caused me numerous issues in Xcode. 
• I can keep test.jpage open in Eclipse to run small code snippets to make sure I am doing substring correctly and the like. 
• Editor tabs are not randomly reused when I am using the debugger
• If I double click on a file to open it or a search result is in an already open that tab opens instead of reusing the tab I happen to be on (tab support in Xcode is just barely acceptable)
• Auto comment generation for JavaDoc is very handy

A lot more to do but it is moving on a pretty decent clip. So far I am just on the behind the scenes processing but soon I will be dealing more on the UI side and I will have to make some changes as there are areas easier to code on iPhone due to its one resolution for all devices. I need to find a nice tiled background for the Android. Currently using one that is OK but could be more professional. For the iPhone I had two images, one portrait and one landscape but that image shows seams when you tile.

Starting my Android conversion of the iPhone app

While the iPhone app is not complete I have begun my Android conversion. Currently the iPhone app is in internal QA and there are a few areas the server team needs to get in place before I can finish things but it is a fully usable app at this time. Since we want to release both apps in sync I need to get the Android side going again.

It is always weird when you switch languages. I have been buried in Objective C for the last few months. Java is a welcome site at this time. First only having one file open per project is so much easier. My code tree looks tiny, I keep thinking I forget to convert a file because the package is so much smaller than the group.

I am starting with the DB tables as I need that full schema in place for everything else to work. What I did to was a pretty easy port and I was able to cut and paste code with minor tweaks. Some of it I have not ported yet as I need to work with lots of rows in Android tables to see how best to handle it. The core concepts will be the same but a lot of data access will be different.

Feels good to get as far along as I did today. First day in a long time I have not been using my mind all day long. This has been pretty mind numbing but you need that from time to time. Happy to be back on my MS Natural keyboard too with all the keys including the numeric keypad. The Mac Book keyboard is OK but really has a limited key set.

Being able to hit Ctrl+Alt+O again to automatically handle all the import statements is great. You don't know how much you miss that feature until you don't have it. Plus Eclipse will auto add imports if you cut / paste from one file to another.

Working on a dual screen configuration is also nice. My monitors are more at eye level so I am not as hunched over as I am on the laptop. I have the iPhone code up via PSPad (my free Windows editor of choice) on one screen and Eclipse full screen on the other.

I know I am in for a bit of a long haul on this. Somethings will not convert smoothly to the Android, some areas will have to be rethought, areas I make fancier on the Android will have to be ported back to the iPhone. I will need to stop and go back onto the Mac when the server calls are ready to I can finish that up and get ready for beta customer release.

Don't miss typing [[[ to get an allocation, init and autorelease going that is for sure. Not hitting the @ key for strings is great too. Stuff in Java feels native to Java, some things in Objective C when doing iOS development feel like "you are not doing the normal thing, you must type more to suffer". Hitting Ctl + Mouse Click to jump to a method jumps to the right method in Eclipse. In Xcode it finds methods with same name and gives you a list to pick from. I am in the M file with the method, just go there. Highlight a string and press Comand + F - screw you, type in what you want to find! So I have to Select, Cmd + C, Cmd + F instead of Select and Cmd + F.

Back from DC - new camera is like a new IDE

Took a family vacation to Washington DC. Before we left I got a new camera - Canon T2i DLSR - for my birthday / father's day. I am not a professional photographer but I want to step up my game. I really don't know how to use all the features but I plan on taking the time to learn what I can.

I have found with almost anything in life that better tools make for a better experience. If you bowl a lot you need to get a ball made for you with the holes drilled for you hand size and your bowling style. I don't doubt someone has bowled 300 games using a ball they found when they arrived at the lanes but most likely they are using a professional tool to do a professional job.

I have friends that tend to go all out no matter what they do in life, beefy gaming rigs, top notch equipment to play tennis, bowl, swim, whatever it happens to be. This does will not make you a professional overnight but it does greatly improve your game and will allow you to more quickly improve because you are not fighting the equipment.

Using the new camera I was able to take much better pictures even though I don't know how to use it fully. I put it in auto mode or maybe switched over to macro mode, adjusted the zoom level and let it do the rest of the work. I had a Canon PowerShot G1 over ten years back. Cost around $800 at the time and it was a wonderful camera, all 3.2 megapixels of it. I got my wife a Canon 510 and it got a lot of use until my son accidentally sat on it and bent the extended lens so it not longer would retract or do much of anything. We have a Canon SD1200 point and shoot camera that is so much faster to fire up and take shots than the vaunted G1 it is crazy. It does face recognition and all sorts of other cool things. Many fine pictures off of it.

There just is no comparison to the new T2i and this is due to more megapixels in small part but mainly due to the superior lens. I just have a starter lens at this point, the one that came with the kit. Even with just that the quality is a huge step ahead of anything I have done before. The camera made me a better photographer, it sparked a new interest in learning how to get even better.

To me this applies to programming as well. Bored? Try a new IDE for the language you are currently programming in. It might take away some of the drudgery. It make have new refactoring tools so you clean up some code you have always hated. Its static analysis tools may point out issues that have haunted your code forever. Maybe it is time to try a whole new language. Or you could just look at some of the menu items in your current IDE to explore areas you have never used.

If you want to be a professional use professional tools.