Root Android 4.3 Galaxy Nexus from OS X

My trusty Galaxy Nexus has just received its fresh Android 4.3 OTA, which, on the plus side, brought "improvements to performance and stability", but has also wiped out root along the way. When life gives you lemons... you write a blog post about delicious lemonade you've made, so I decided to update my old guide on how to root JB Galaxy Nexus. Check the link if you need more detailed explanation of each step.

First thing first, if you haven't done so yet, reveal Developer options in settings, enable USB debugging, unlock your bootloader and install or update adb, fastboot and other tools. I've written posts about that in the past, so follow the links if you need a walkthrough.

Download the latest ClockworkMod Recovery image for Galaxy Nexus (GSM) from ClockworkMod site, at the moment it's recovery-clockwork-touch-6.0.3.4-maguro.img and save it as cwm.img in your home directory.

Download the latest SuperSU CWM installable ZIP using the link in the XDA-Developers forum thread (UPDATE-SuperSU-v1.45.zip at the time of this writing) and save it in the same directory as CWM Recovery image.

Make sure adb is detecting your devices correctly and is displaying phone's serial number when you run:

adb devices

You should see similar output in the terminal:

List of devices attached 
0149AXXXXXXXXXXXX device

Push the SuperSU installer to your sdcard with command:

adb push UPDATE-SuperSU-v1.45.zip /sdcard/

Reboot into bootloader with command:

adb reboot-bootloader

Make sure fastboot is also detecting your device and is displaying phone's serial number when you run:

fastboot devices

Boot into CWM recovery using cwm.img file in your home directory:

fastboot boot cwm.img

You will see the following output in the terminal:

downloading 'boot.img'...
OKAY [  0.707s]
booting...
OKAY [  0.388s]
finished. total time: 1.096s

Device will boot in to CWM recovery and present you with various options. Choose install zip option, then chose zip from sdcard, then 0/ - this is the default path to internal sdcard, then find and select the UPDATE-SuperSU-v1.45.zip file we pushed earlier. Chose Yes - Install UPDATE-SuperSU-v1.45.zip when presented with confirmation dialog and SuperSU will be installed from sdcard.

Once installation is complete select +++++Go Back+++++, then reboot system now. I personally choose not to disable recovery flash when presented with the next prompt.

Once the phone has restarted test for presence of root prompt (#) using:

adb shell su

You might want to update the SuperSU app via GooglePlay if you're not using the latest version, and it's done.

Using Git with Dropbox

Recently I challenged myself to get much more familiar with Git. I've used it briefly in the past, but some of my recent work got me seriously craving for a better version control system (VCS) than Subversion. Something faster, something that didn't require connection to central server, yet still allowed me to access my work on both my home and work computers. Git alone met majority of these requirements, I just needed a remote repository to share and sync my work between computers.

GitHub was my first choice, but my some of my work is not supposed to be available to the public, and I didn't think premium account was worth it. Setting up my own server seemed like too much work. My search for something simple, free, yet fast and reliable pointed me to Dropbox.

Dropbox has always been my favorite way to sync files and passwords between my devices. Here's how to turn it into a perfect remote Git repository.

First, we need a local Git repository with our work in it. If you don't already have one create one:

git init /path/to/new-project

It's good to have a README file, so let's make one:

cd /path/to/new-project
touch README

Now, let's stage and commit our changes:

git add .
git commit -m "Initial Commit"

Next, we create an empty "remote" repository in our Dropbox folder:

git init --bare ~/Dropbox/git/new-project.git

Technically, it's still stored locally on the same machine, but because it's inside our Dropbox folder, it will be automagically synced to the cloud and other linked devices.

We still need to "add" our remote repository to the local one:

git remote add dropbox ~/Dropbox/git/new-project.git

I'm naming my remote repository "dropbox" instead of traditional "origin". This could be handy if you have multiple remote repositories.

Then, push local master branch to remote:

git push -u dropbox master

I am also telling git to track remote master branch as upstream with -u flag.

Now I can work in my local repository, make changes and commits, and periodically updating remote repository with:

git push

If I need to work on my home computer, I can simply clone my project from remote repository inside my Dropbox with:

git clone -o dropbox ~/Dropbox/git/new-project.git

If local repository already exists on my home computer, but doesn't have the latest changes, we can fetch and merge them from remote with:

git pull

Now we have a distributed VCS with a central server, which is fast, reliable, super easy to setup, doesn't require maintenance, private and is free of charge. It might not be suitable for collaboration between multiple developers, but it's ideal for the needs of one.

Grab Android screenshot to computer via ADB

There are many ways to take a screen shot on Android device. One simple way to capture the screen on Galaxy Nexus is to simultaneously press and hold Power and Volume Down buttons. The image will be saved in a "Screenshot" directory and accessible via Gallery.

Quite often, however, I need to copy the captured image over to my computer. Usually, I do this with adb pull command, but if I'm going to use CLI to retrieve the file, why don't I take the screen shot with it as well?

One method is to use screencap command via adb shell like so:

adb shell screencap -p /sdcard/screen.png
adb pull /sdcard/screen.png
adb shell rm /sdcard/screen.png

Not bad, but seems like there's some room for improvement.

Information provided by screencap -h indicates that screen shot can be sent to stdout, but running adb shell screencap -p > screen.png results in seemingly corrupt file.

Luckily, I wasn't the first person looking into this issue. Apparently, adb shell is performing an EOL (end-of-line) character conversion, from LF (line feed, '\n', 0x0A) to CR+LF (carriage return followed by line feed, '\r\n', 0x0D0A).

If that sounds familiar, that's because I've dealt with this before. Unfortunately, this time we're working with binary data, so using tr -d '\r' command to remove all 0x0D will likely corrupt it by also removing bytes that aren't part of the 0x0D0A sequence. We want to only remove carriage return characters when they are followed by a line feed.

The solution is to use sed search and replace as follows:

adb shell screencap -p | sed 's/\r$//' > screen.png

Unfortunately, I found that sed fix while works on Ubuntu doesn't work on OS X. This solution using perl handles binary search and replace better:

adb shell screencap -p | perl -pe 's/\x0D\x0A/\x0A/g' > screen.png

Now we have a one-liner that grabs a screen shot from the Android device and stores directly on the computer executing the command. Beautiful.

Access Android app data without root

Recently I recommended a certain data collecting Android app to my coworker. The application was great, but it was designed to upload collected data to the cloud and didn't offer any means to export or backup information locally.

This isn't an issue on my rooted Galaxy Nexus. Usually, I connect my phone to my computer with USB cable, open CLI and do the following, where app.package.name should be replaced with the actual package name of the application:

adb shell
su
cp /data/data/app.package.name/databases/application.sqlite /sdcard/
exit
exit
adb pull /sdcard/application.sqlite ~/

Update: after I've discovered this neat trick, the above can be reduced to one line:

adb shell su -c cat /data/data/app.package.name/databases/application.sqlite | sed 's/\r$//' > application.sqlite

On my coworker's phone, however, this wouldn't work since he was using a stock, non-rooted Android OS.

I Googled around for a solution and seemingly found one. If the application, whose data you wish to access, is debuggable, its read-protected folder can be accessed with the help of the run-as command. In essence, we pretend to be the application in question and copy file(s) stored inside the application's data folder into user readable folder like so:

adb shell
run-as app.package.name \
cp /data/data/package.name/databases/application.sqlite /sdcard/
exit
adb pull /sdcard/application.sqlite ~/

Unfortunately, in this case the application was not debuggable, so the method above did not work either. It appeared that reading Android application data folder with root access was impossible...

At this point I remembered that starting with Android v4.0 (Ice Cream Sandwich) Google has provided a way to backup data and applications from Android devices without root via adb. So all I had to do in order to pull that application's data from the device is to run:

adb backup -f ~/data.ab -noapk app.package.name

This will prompt you to "unlock your device and confirm the backup operation". To keep things simple do not provide a password, otherwise you will have to jump through the hoops to decrypt it later. Just click on "Back up my data" button. The screen will display the name of the package you're backing up, then close by itself upon successful completion.

The resulting ".ab" file contains application data in android backup format, which, thanks to +Nikolay Elenkov, is very well explained and documented in his excellent blog post. Basically, it's a tar archive that has been run through deflate and optionally encrypted (in a somewhat peculiar way) by AES-256-CRC cypher. Nikolay even went as far as to write a convenient Java program that can pack and unpack such Android backups with and without encryption.

To quickly extract a simple non-encrypted backup (you did omit the backup password as I suggested, didn't you?) run:

dd if=data.ab bs=1 skip=24 | openssl zlib -d | tar -xvf -

Update: It was brought to my attention that not all openssl installations are compiled with zlib support. Here's an alternative one-liner that makes use of python to achieve the same result:

dd if=data.ab bs=1 skip=24 | python -c "import zlib,sys;sys.stdout.write(zlib.decompress(sys.stdin.read()))" | tar -xvf -

The result is the apps/app.package.name/ folder containing application data, such as SQLite database I was particularly interested in and the application preferences.

Enjoy!

Update: A few people mentioned in comments below and on StackOverflow that this method doesn't work if application developer has explicitly disabled ability to backup his app by setting android:allowBackup="false" in the application manifest.

Backup your Android without root or custom recovery

Recently discovered a neat new way to back up apps on my Android without having to use Titanium Backup, having to unlock bootloader or root the device to take a NANDroid snapshot. The icing on the cake - I can do it from command line!

Warning: this method is only available to devices running Ice Cream Sandwich or higher and it has bugs, but it is an alternative for folks who want to make a backup before rooting their device.

You'll need to have the latest adb installed and in you path, USB debugging enabled and your phone connected to your Mac.

The command in question has the following format:

adb backup [-f <file>] [-apk|-noapk] [-shared|-noshared] [-all] [-system|-nosystem] [<packages...>]

Here are some common uses:

1. Backup all non-system apps, their data and your shared data (i.e. SD card contents):

adb backup -f /path/to/backup-file -apk -shared -all -nosystem

2. Backup all apps, their data and shared data (note: won't work on DRM protected apps):

adb backup -f /path/to/backup-file -apk -shared -all -system

3. Backup only app data (not the APKs themselves) and your shared data:

adb backup -f /path/to/backup-file -all

4. Backup only non-system apps:

adb backup -f /path/to/backup-file -apk -noshared -nosystem

5. Backup only your shared data:

adb backup -f /path/to/backup-file -noapk -shared -nosystem

6. Restore everything from your backup:

adb restore /path/to/backup-file

During backup your phone will display a backup confirmation screen giving you an option to enter an encryption password. In my case it would only work with non-empty password, otherwise it would generate a zero-sized file.

Install adb and fastboot on OS X

There are plenty of packages floating around the Internet with just the adb and fastboot utilities for OS X, but I prefer installing those (and few other android utilities) on OS X is via Android SDK. This way you know exactly what you're getting and it's easy to can keep them up-to-date with Android SDK Manager.

First download the latest Android SDK package from http://developer.android.com/sdk/index.html (android-sdk_r20.0.3-macosx.zip at the time of this writing).

Change into your download directory and unzip archive into your preferred location. I like my home directory so I run:

unzip android-sdk_r20.0.3-macosx.zip -d ~

This creates a android-sdk-macosx directory in our home directory. Now run android application inside the tools directory to open Android SDK Manager (ASM).

~/android-sdk-macosx/tools/android

ASM will connect to various repositories and present you with all possible Tools, Platforms and Add-Ons that could be downloaded and installed. Since we're only interested in few specific tools, deselect all except "Android SDK Tools" and "Android SDK Platform-tools" packages and then install / upgrade them.

Note: to accelerate checking for updates during future runs, go to "Tools"->"Manage Add-on Sites..." and click on "Disable All" button in "Official Add-on Sites" tab. From now on ASM will check only the default repository containing tools and APIs.

Now we'll add android tools to our PATH to make executing them more convenient. Create or edit .bash_profile file in your home directory to contain these lines, just don't forget to replace YOURUSERNAME with your actual user name:

ANDROID_TOOLS=/Users/YOURUSERNAME/android-sdk-macosx
export PATH=${PATH}:${ANDROID_TOOLS}/platform-tools:${ANDROID_TOOLS}/tools

Done.

Rooting Jelly Bean Galaxy Nexus from OS X

Now that we've unlocked bootloader on our Galaxy Nexus and updated OS to Jelly Bean 4.1.1, rooting it is extremely simple. In fact, to root GNex all you need is su (Super User) binary as one of the system apps. Since /system partition is mounted as Read Only, we need to boot into custom recovery mode and install su utility on our phone from there.

In this example we'll use ClockworkMod Recovery. You don't have to install the ClockworkMod Rom Manager application, or replace your stock recovery partition with CWM's. Having unlocked bootloader allows us to boot into CWM Recovery without overwriting anything on our device, as you would boot into Linux on PC from a LiveCD/USB.