PockeTwit v.68 Released

After a lot of work, PockeTwit v.68 is finally ready for release. This version has quite a few new features so I thought I'd detail them here.

One of the most visible changes is the addition of "nested" side menu items. I was running out of space on the side menus and it was keeping me from being able to add some requested features, so some side menu items can now have multiple sub-items. For example, the left menu now has an "Other Timelines..." item. When clicked, that opens a submenu that allows you to search, view your favorites, or view the Public timeline.

The multimedia capabilities of pocketwit have recieved a major ovehaul thanks to @roelvandenbrand. Roel had been requesting that I add support for image posts to mobypicture.com for quite a while. Eventually he saw it just wasn't going to be a high priority on my list so he decided to implement it himself!

Once he got started, he did way more than just add mobypicture hosting-- he put in a system that lets the user choose from several image hosts. He also changed the posting process so it uploads a picture first, THEN gets the URL and puts it into the post (so you can see how many characters it will take up). He even implemented a way to fetch images from the hosts, so we can show thumbnails of images from other posts without having to actually launch a browser. Be sure to set which picture service you want to host your images in the "Other Settings" page.

I've also added a new "addressbook" for posting. Sometimes you want to include someone's @name in your post without having to find one of their posts to @reply. Now if you start typing "@p" in the post screen, it will pop-up a list of all users in your addressbook whos screen names start with "p". This makes it easy to select that username and have it inserted into your post. It also works for "d " (direct messages) as well.

Unfortunately twitter does not provide an easy way to get the screen names of everyone you are following, so the addressbook is automatically filled with users from your Friends and Messages timelines. As you receive more status updates and messages from other users, they are added to the addressbook.

Some other minor features have been added as well -- you can now copy the text of a selected item to an email. There's an option to automatically scroll to the top of the list when new items arrive.

There's also quite a few bugfixes, and it might even be a little better about memory and battery usage.

I hope you enjoy the new version. If you have any ideas you'd like to see implemented in the future, be sure to go to our Uservoice page and submit them.

(Late) Introductions

I often feel like these "who am I" blog posts are a bit self-serving and egotistical. But as PockeTwit gets more and more users I finally decided that some people might benefit from knowing a bit more about where it comes from. And since I haven't actually posted anything on the blog in a while, it would at least add some filler.

First things first -- I am not associated with Google in any way. Yes, the project is hosted on Google's servers, but they provide free hosting and source control for any open-source projects. As much as I'd LOVE to work for Google Labs, they still haven't contacted me with that sweet job offer yet.

Second, it's just me. There's no "Team PockeTwit" or startup corporation or anything like that. Just one guy who was feeling a bit bored and decided he needed a hobby project. Some folks like to pretend their software comes from a nameless company and has a staff to support their user's every need (yeah, I'm looking at you TwitPic!). Not here. I answer the emails. I answer the tweets. I write the code.

Which brings me to the final bit -- Why? I do it because it's fun. I had finished a previous project experimenting with writing .NET for my phone and wanted to try another. The iPhone 3G had just come out and everyone was excited about twitter for the iPhones, but the apps for Windows Mobile just felt drab. And since none of them were open-source, I couldn't just try to work on a new UI as a contribution. So I started my own. Then somehow I'm 64 releases into it!

So there you have it. PockeTwit is just a small one-guy project. Yes, I embrace the "beta culture" of software because I'm just doing it for the hell of it. And it's great that your hobby can generate something other people can use and enjoy.

Smooth-Scrolling Part 3 - The Sliding Window

I finally figured out the true nature of the problem (described in parts 1 and 2), but the prognosis was still the same -- rendering the entire list to a large bitmap caused an OutOfMemory exception for some devices. I could not pre-render everything to ensure smooth scrolling.

I spent some time investigating hacks to work-around it, like declaring a non-graphics block of memory to hold the large bitmap, but every one of them failed or was too much work to be practical. And when I sought out advice from development communities, the reaction was mostly that I was foolish to try something like this with WM.

I finally settled on a "sliding window" approach. The screen can hold 4 or 5 items at most. Instead of rendering all 50 or so items to a single bitmap, why not render the 5 items on screen and a few of the ones before and after that set? This way there's a bit pre-rendered buffer when the user scrolls up or down. When the user stops scrolling, I can re-calculate the items that need to be in the buffer and re-render it in a background thread.

The core concept is simple, but there's a bit more in the actual code. Deciding on the window size was difficult. 15 items was ok for most users, but still crashed a few VGA users. And ironically, QVGA devices could handle WAY more items because of their smaller resolution. I liked that my Mogul could scroll the entire set without buffering and wanted to keep it that way. So the first step in the code is to determine the number of items it can handle in the buffer. I do it by starting with the maximum number of statuses in the list, then try to create a bitmap that will fit that number. If that fails, it decreases the number and tries again until it works.

The code that re-fills the buffer after sliding is interesting too. After the UI has stopped scrolling, it determines the statuses on-screen and enough before and after to fill the buffer. It passes that whole list to the buffer. At first, the buffer would just render that whole list again, but this was inefficient. Now it determines if there is any overlap with what it currently has rendered. It moves the overlap to the top (or bottom) of the buffer, then only re-renders the new items.

One downside to this is that it is possible for the user to scroll faster than the buffer can re-fill itself. When they do, they'll see a blank area momentarily until the buffer is able to catch up again. And how often this happens depends on how small the buffer has been made.

The quest for a smooth-scrolling interface - Ouch

In my last post, I discussed my great new idea for smooth scrolling in PockeTwit. I tested it for a good while on my Sprint Mogul and several emulators. I even had several users try out the dev build before release. I felt satisfied it was ok, so I put it in the wild.

And immediately my email began filling up with crash reports. I got to work and was able to solve all the issues pretty quickly except one -- an "OutOfMemoryException" that was coming up almost immediately on startup for many users.

This one was a bit of a mystery. I figured out that most of the users with the error were using newer devices with VGA screens. Typically, these devices have WAY more memory than the old ones without a problem. To make it even more strange, the device claimed to have plenty of free Program Memory.

The problem was obviously with my new method of generating a new huge bitmap surface to render all the statuses on. Even if the device had plenty of free memory, it would often fail creating that bitmap. It would usually run without issue after a soft reset. When I looked at a task manager, the program looked like it was using the same amount of memory as before!

A more careful look showed that not everything was the same. I was creating a big giant bitmap and I knew that had to take up more memory somewhere. My process (PockeTwit.exe) was using the same amount of memory, but another process had quadrupled in size!

After some more research I discovered that all bitmaps in memory are assigned to their a seperate process -- gwes.exe (Graphics, Windowing, and Events Subsystem). It handles all device-specific graphics functions, and my bitmaps fell under it's umbrella.

But it still didn't explain the OutOfMemory error. The device itself still had plenty of memory. I had to dig even more to discover that each process in Windows CE 5 (which is what Windows Mobile is based on) can use up no more than 32M of memory. On VGA devices, combining my giant bitmap with the bitmaps of all the other applications on the device would sometimes put gwes.exe over that limit. I finally found the cause of my solution.

So, to sum up the problem:

1. I make a giant bitmap.
2. Because of the architecture of the OS, that bitmap is assigned to gwes.exe instead of my process.
3. gwes.exe may expand past the 32M of memory per process limit.

Now that I understand the cause of the problem, it's time to work on a solution that will let me keep my smooth scrolling without using up so much bitmap memory.

The quest for a smooth-scrolling interface

PockeTwit's still crashing, and I blame it all on those clickable links.

Early on in developing PockeTwit, I decided I wanted to include "hyperlinks" in the status text, so the user only has to click on an @reply name and be able to see that user's timeline. This was an important usability feature for me.

I also wanted a smooth-scrolling modern interface. This meant I couldn't just rely on the PocketIE rendering engine for my UI. I had to do "owner-drawn" controls in GDI, where my code would specifically draw the seperating lines, the avatars, and even the lines of text.

So for each status on screen, PockeTwit would have to figure out how to word-wrap the text, find any clickable links, then draw it all. It would have to do this every time the screen was scrolled, even a single pixel. It worked, but all that computation slowed things down a good bit.

I tried a few things to speed it up. I "cached" the line breaks and clickable locations in memory. It helped a bit, but nothing was going to make it really smooth like I wanted.

With v.49, I tried something radical. Instead of looping through all the statuses every time and only rendering the ones on-screen, I created a giant bitmap and rendered ALL the statuses to it ONE time. Then as the user scrolled, I just copied the appropriate section of that bitmap to the screen.

This was WAY faster. I was amazed at how well it worked and how smoothly it scrolled. And I looked at the task manager and was suprised to see that PockeTwit.exe was still using almost the exact same amount of memory as before. Unfortunately, it really was too good to be true. . . (to be continued)