HD2 running Darkstone Froyo 2.1

After spending the last few weeks on the HD2 Android Development forums at XDA I learned how to coerce my T-Mobile HTC HD2 into running Android. As of today I’ve got Android Froyo running with few issues and reasonable battery life. In this post I’ll explain how how you can run Android on your HD2 Leo.

Before starting this tutorial, you can try out Android on the HD2 by skipping ahead to install Android.

First, flash the boot loader. Doing so will allow you to replace windows mobile with custom roms better suited for running Android. I’ve flashed two HD2′s with Cotulla’s HSPL3 boot loader and haven’t had any issues.

With the new boot loader installed, flash the windows rom. Find a good replacement rom that will load Android quickly and run it stably. Roms I’ve found worked well with the HD2 Android builds are Miri and Chucky ROMs. Check XDA and htcpedia for custom roms.

Next reformat the SD card. Don’t forget to back-up your data first.

With a fresh file system on the SD card you’re ready to install Android. Download and install any of the recent HD2 Android builds from the forums on XDA and you should be all set. My favorite builds right now are the near-stock Froyo builds created by darkstone.

Once you’ve got Android installed, make running it easier. Exceller Multiple Build Loader provides a nice interface for booting into Windows or Android automatically after the phone powers on. There is also support for booting into Ubuntu, if you’re into that sort of thing.

Govern Android battery consumption. SetCPU can be used to manage clock speeds, making it easy to reduce power consumption. It can be downloaded from the Market at a cost, or here for free. (Note: Free version requires adb to install.) Try setting a “Screen Off” profile that limits the CPU usage to 245 max to prevent much of the battery drain associated with sleep mode.

If you experience issues in the Android OS, try upgrading the radio. Though something I overlooked at first, upgrading the radio seems to have helped me prevent SoD from occurring as often. Grab a new radio from the forums at XDA.

With enough tweaking your phone should be running Android issue-free all day with acceptable battery life. If not, keep messing with it. Try downloading a different Android rom or swapping in a newer Linux kernel image (zImage) to find the one that works best with your build. Learn from experience by checking out others’ reply signatures on the forums on xda and htcpedia.

Here’s my current setup:

Device: TMOUS HTC HD2
Boot Loader: CotullaHSPL (SPL-2.08-HSPL)
WinMo Rom: ChuckyROM-23134-Tabtastic-TMOUS.Aug.30
Radio: 2.12.50.02.2
Android build: darkstone HD2Froyo V2.1
Launcher: EBL2.0d_PlusUbuntu

Happy modding!

Identification

Look for IE-only display issues (outside the norm) when CSS files contain many style rules. Rules near the bottom of the stylesheet will not be applied. The issue can be confirmed with the IE dev toolbar. To do so use dev toolbar to save off the CSS file downloaded from the web server, and then scroll to the bottom of the CSS tab in the toolbar (assumes IE8 dev tools). Compare the last style rule displayed in the tab with the last rule in the actual file itself. If they do not match, and the CSS is valid, the limit has been reached.

Solution

Reduce the number of CSS selectors per file to a number less than 4096. Several approaches for doing so include:

• Refactoring existing CSS
• Splitting the CSS payload into multiple files
• Reevaluating current browser support strategy (wink, wink)

If the stylesheet under scrutiny contains IE-only hacks interspersed with standards compliant CSS, consider moving the hacks into a separate file and using Conditional Comments to pull them in. Otherwise, the best option may be to split the CSS payload into multiple files–resulting in an extra HTTP request for all browsers, not just IE. Reevaluating browser support is another option, but probably not feasible for large-scale applications.

Regardless of which approach is taken, something will need to be done for IE once the per-file limit is reached. Though 4095 selectors ought to be enough for anybody…

Fake virus scan running in Google Chrome

After a few seconds on the page the following occurred:

• Result page successfully hijacked and a faux Windows Explorer interface loaded.
• A fake virus scan ran in the look-alike Explorer window, conveniently uncovering trojans and other malware.
• Alerts, dialogs and phony windows displayed in an attempt to execute a little social engineering.

After attempting to interact with the document the following occurred:

• A potentially threatening program file download initiated if you clicked on anything in the page.
• Additional warning dialog and pop-up window displayed on attempting to unload the page (e.g. hit the back button).
• Intermittently, the Google Chrome 3.0 back-button and tab [x] icon stopped functioning as expected.

The last point was of particular interest at first, as it is not like modern browsers to allow websites to modify browser functionality. But after some additional research, it was clear the hijack was more complex than a page titled with search engine optimization in mind. This article will study the hijacking in some technical detail, discuss the black hat SEO likely used to get the rogue application listed on Google and how the app was able modify the expected functionality of browser controls in Chrome 3.0.

The bait and switch

How does activating a seemingly innocuous (and tasty) link from Google land someone on a potentially dangerous page? 302. HTTP 302 that is.

After locating the offending link on Google, I used a client-side proxy to trap the HTTP headers for a play-by-play review:

GET http://woodstockfolkmusic.com/bftwe/tiijy/carne.php HTTP/1.1
HTTP/1.1 302 Found
GET http://goodstats1.net/in.cgi?2 HTTP/1.1
HTTP/1.1 302 Found
GET http://sunstats1.net/in.cgi?default HTTP/1.1
HTTP/1.1 302 Found
GET http://sunstats1.net/redirect3/ HTTP/1.1
HTTP/1.1 302 Found
GET http://bookletantcars.cn/?pid=283s01&sid=2a15a0 HTTP/1.1
HTTP/1.1 302 Found
GET http://wwwantispyware10.com/scan1/?pid=283s1&engine=%3D3W59jDuNTIuMTUxLjE1MyZ0aW1lPTEyNTE2NYcMOAkN HTTP/1.1
HTTP/1.1 200 OK

Stepping through the sequence we can see the initial GET request followed by a 302 redirect response. Several redirects later and the browser successfully arrives at destination malware, beef stew long forgotten.

Black hat inside

The next thing I tried was to analyze the source of the PHP file cataloged by Google, which was not difficult for two reasons:

1. Directory listing was enabled on the web server, and
2. Navigating directly to the PHP file caused the page to load without redirect.

The PHP file was stowed away on woodstockfolkmusic.com (which appears to be a legitimate folk music site based out of Illinois) along with some 300 similar PHP files, covering a range of topics from Alba to Wisconsin. The files found contained mostly deprecated HTML markup (remember the <marquee> tag anyone?) and no PHP script or META tags. The files were stuffed with hundreds of keywords, a form of spamdexing I thought was no longer practiced. Nevertheless the result still appeared on Google, possibly with a little help from cloaking.

Engage the cloaking device

Curious as to why the PHP files (with no PHP script or META tags, mind you) would redirect links coming from Google, but not when loaded directly, I again pulled up a client-side proxy for closer investigation. Below are the results of several slightly modified HTTP requests for the file initially requested by Google. Each request contains a modified Referer request-header field.

First request
Hacked the Referer field to point to the Google domain.

GET http://woodstockfolkmusic.com/bftwe/tiijy/carne.php HTTP/1.1
Host: www.google.com
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0 Paros/3.2.13
Referer: http://www.google.com/
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Encoding: sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

First response
Hijack successful; browser redirected to bogus antivirus page.

HTTP/1.1 200 OK
Date: Sun, 18 Oct 2009 22:33:31 GMT
Server: Apache
X-Powered-By: PHP/5.2.8
Connection: close
Content-Type: text/html

Second request
Hacked the Referer field to a domain other than Google.

GET http://woodstockfolkmusic.com/bftwe/tiijy/carne.php HTTP/1.1
Host: woodstockfolkmusic.com
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0 Paros/3.2.13
Referer: http://www.habdas.org/
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Encoding: sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

Second response
No hijack; browser sent directly to indexed page.

HTTP/1.1 200 OK
Date: Sun, 18 Oct 2009 22:31:36 GMT
Server: Apache/2.0.63 (Unix) mod_ssl/2.0.63 OpenSSL/0.9.8e-fips-rhel5 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 PHP/5.2.6
X-Powered-By: PHP/5.2.6
Content-Type: text/html

Additional testing
Additional testing revealed page redirection would occur only when the Referer field was included in the HTTP request header, and only when the field value contained certain phrases. Two phrases found to trigger the hijack include “google” and “yahoo” (case insensitive) while other likely phrases such as “bing”, “msn”, “aol” and “ask” did not.

Note: I am I not currently aware if Yahoo is susceptible this particular brand of page hijacking. If you’ve seen it on Yahoo or know of any examples, please comment and let us know.

Testing for the presence of phrases “Googlebot”, “googlebot” or “google” and “bot” (separated) all resulted in the 302 redirects, which leaves some of the following possibilities:

• The 302 redirect (likely of the .htaccess kind found in the Antivirus 2009 approach) was turned on after the page was indexed.
• The web crawler that originally accessed the page did not pass the phrases “google” or “yahoo” in the Referer [sic] field in the HTTP request header.

Monitoring over a several day period landed the browser on some of the following domains, each with their own similar virus scan or some derivation:

• wwwantispyware10.com
• topantimalwarescan7.com
• top-antispyware-scan8.com
• computer-protection11.com
• webprosecurity.com
• guardpconline.com

The Chrome 3.0 Browser Button Issue

The Chrome button issues are happening on and off. Some of the changes in behavior I have witnessed using Chrome v3.0.195.27 (Win):

• Browser unable to navigate backwards in history;
• Navigation backwards in history only after several tries; and
• Tab hangs and cannot be closed, and Windows clocks, until the pop-up notification window is closed.

Example user agent string sent from a web browser during an HTTP request:
Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.89 Safari/532.5

The above example, for instance, provides information such as browser and browser version, user locale (language), OS, system architecture and the layout engine used. When authoring documents for the Web, information from the user agent string can be valuable in determining how best to mark-up documents.

Getting the information is easy.

Collecting user agent strings

Two methods for accessing the user agent string include:

1. From the HTTP request header’s User-Agent field; and
2. Using DOM and JavaScript.

Reading from the User-Agent field

A benefit of using the HTTP header to gather data is simplicity of design.

HTTP request header showing the User-Agent field (in bold):

GET / HTTP/1.1
Host: livehttpheaders.mozdev.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.89 Safari/532.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive

Using the HTTP header the user agent is transmitted directly to the HTTP server on page request, making it possible for servers to output the user agent string to a log file for later analysis. The user agent string alone provides enough information to implement on websites valuable browser support strategies such as graded browser support.

User agent retrieval using DOM and JavaScript

Using DOM and JavaScript, on the other hand, add additional development complexity, but provide more detailed and valuable analytic data, in addition to the user agent string alone. Tools like Urchin (now Google Analytics) utilize JavaScript and the DOM to gather analytic data about visitors.

Bookmark the following link to create a bookmarklet that will retrieve the user agent from a browser: javascript:alert(navigator.userAgent).

Regardless of the collection approach used, methods for extracting data from the string remain similar.

Data extraction methods

Once the user agent string(s) are collected, data extraction may take place. Two methods for reading and extracting information from the user agent string include brute force and pattern recognition:

• Under the brute force approach the user agent string is compared programmatically to a database of known strings. Though it offers a relatively simple implementation, the brute force approach can be difficult to maintain and becomes increasingly inefficient as comparison data sets grow larger.
• Thanks to RFC 2616 and preceding RFCs, and de facto standards for formatting user agent strings, another method known as pattern recognition is possible. Using pattern recognition the user agent string is broken into its component pieces and heuristics applied to gather information. Though more complex to implement than the brute force approach, pattern recognition does not suffer from the same problems in efficiency and maintainability in the long-run.

Due to its drawbacks in the application of extracting data form user agent strings, the brute force approach will not be discussed further in this article.

Pattern recognition on the user agent string

Check out Identify User Agent by string format recognition for an example of user agent pattern recognition. Though a little outdated, the article provides additional depth, in addition to some useful programming techniques and lax copyright restrictions.

User agent spoofing

Impersonating browsers and mobile devices is simple with Firefox. Just download User Agent Switcher plug-in and put it to the test at useragentstring.com. See Web Development and Debugging Tools for a list of tools useful for front end development.

Screenshot of a Lightview modal dialog

Though modal dialogs are not a new concept in UI design, the number of homegrown Lightbox clones appearing on the Web since major JavaScript libraries like Prototype and jQuery hit the scene has been staggering. Unfortunately, many of the clones developed leave some key usability considerations unaddressed, and struggle with common problems in accessibility. Some key usability features that should be considered during creation of a Web-based modal dialog include (1) manage focus and allow tab navigation (2) disable elements outside the modal dialog (3) give users an out and (4) provide graceful error recovery.

Manage focus and allow tab navigation

Guide the user experience by managing focusable page elements using JavaScript.

• Tabbing should allow the user to navigate back to the browser’s location bar and other tabbable toolbars.
• When the modal dialog is shown, the user should not be able to tab to document content outside the dialog content area.
• When the dialog is hidden, focus should be restored to the initial element used to activate the modal dialog, the original tab ordering should be restored and the user should no longer be able to tab to content inside the modal dialog.

Disable elements outside the dialog

Guide user interaction with page elements outside of the modal dialog and keep focus in the lightbox window.

• Display a translucent overlay above the page while the dialog is shown, giving the perception of modality while maintaining frame of reference to existing page content.
• Prevent interaction with elements outside the modal dialog using script to do the following:
  • Save and then temporarily set a new tab order for all applicable elements;
  • Save and then temporarily set the disabled flag on all applicable elements; and
  • Save and then temporarily update applicable hyperlink click handlers to return false.

Give users an easy way out

Give users the ability to exit unwanted modal dialogs without thinking much about it.

• Use a graphical [x] link the user can click to initiate the hide method. In addition, consider using the graphic as a background image for a text-based link (e.g. “Close”) to help improve comprehensibility slightly for both sighted and non-sighted users while maintaining application scalability with a single link implementation.
• While the dialog is displayed, listen for the [Esc] key. If it is pressed, exit the modal dialog and restore the previous display state.
• If the Lightbox was activated by the user, make the close link the next focusable element in the tab order.

Provide graceful error recovery

Script display methods (e.g. hide/show) should listen for script errors and, upon error, close the modal dialog and restore previous settings–including any previously focused element. Utilize the try/catch block to make it happen, but beware that suppressed errors increases debugging complexity.

Related articles

See also Roger Johansson’s Lightboxes and keyboard accessibility for additional considerations in improving Lightbox usability, with a focus on keyboard accessibility.

Recommended scripts

• ColorBox — A light-weight, customizable lightbox plugin for jQuery
• Lightview — Lightview was built to change the way you overlay content on a website

Tools for Firefox

Build for standards.

Firebug > *

• Firebug integrates with Firefox to put a wealth of development tools at your fingertips while you browse.
• HTML Validator is a Mozilla extension that adds HTML validation inside Firefox and Mozilla.
• Web Developer adds a menu and a toolbar with various web developer tools.
• IE Tab, an extension from Taiwan, embeds Internet Explorer in a Mozilla/Firefox tab.
• ColorZilla provides Advanced Eyedropper, Color Picker, Palette Viewer and other colorful goodies for your Firefox.
• YSlow analyzes web pages and tells you why they’re slow.
• HttpFox monitors and analyzes all incoming and outgoing HTTP traffic between the browser and the web servers.
• MultiFirefox is a small launcher utility that allows you to run multiple versions of Firefox side-by-side.
• User Agent Switcher to spoof user agent strings for browser support testing.
• Google Toolbar to visualize Page Rank.

Tools for Internet Explorer

Ensure application compatibility.

• Internet Explorer Developer Toolbar provides a variety of tools for quickly creating, understanding, and troubleshooting Web pages. It also allows for the inspection of the MSIE-specific CSS passed in through the use of Conditional Comments.
• A look-alike is just that, not the real thing. Here are the Internet Explorer Application Compatibility VPC Images, made available by Microsoft for use in testing browser compatibility with MSIE.

Other tools

Polish it to a bright shine.

• Selenium IDE for automated testing, helping unlock the potential for test-driven development in the UI layer.
• Paros is a simplistic proxy tool that allows you to trap raw HTTP request and response headers for analysis and testing.

PassKeeper is a Windows utility that allows you to keep a list of accounts with usernames, passwords, and notes. This list is stored encrypted.

The utility is freeware and has been available for public download since the mid-90’s. Data are encrypted using the 56-bit DES cipher and stored in an DAT file in the application’s root directory. The size of the application (189 kilobytes) and the data file (~400 bytes/entry) are lightweight and can easily be carried around on any USB flash drive.

Image of PassKeeper running under Windows Vista

The application’s user interface (pictured left) is straight-forward and easy to use, and the system-oriented UI design has become more visually appealing as Windows has evolved.

One thing that hasn’t evolved, however, is the utility’s application icon. The application icon (not pictured) has looked outdated since about Windows 98. But fixing the blemish is easy enough. Just create a Windows Shortcut and use a different icon. The imageres.dll located in %windir%\system32\ in Windows Vista contains a decent-looking padlock icon that can be used if desired.

With a little practice, the entire utility can be navigated using only the keyboard, and passwords can be quickly copied from PassKeeper and pasted into online forms and desktop applications without the use of a mouse. Coincidentally, the copy/paste behavior may help enhance security by masking password keystrokes from key loggers.

Over time, one noticeable drawback of using PassKeeper is that it does not provide a built-in password generator. Another is that passwords copied to the clipboard are not automatically cleared after a set amount of time, requiring the user to do so by some other means—if at all. There is also a bug with account names using certain special characters, though in my ten years using the utility I only saw it once. According to program readme.txt on www.passkeeper.com the utility is limited to 128 entries, but offers a simple workaround for the limitation.

Overall, PassKeeper is a straight-forward, easy-to-use utility for managing and securing personal passwords and account data. And though it’s starting to show its age, it continues run stably as Windows evolves. If you decide to use PassKeeper and carry around password data on a USB flash drive, the 56-bit encryption used should buy most users plenty of time to change any sensitive passwords should the device be lost.

Other password managers worth checking out

• KeePass Password Safe — A free open source password manager, which helps you to manage your passwords in a secure way.
• KeePassX — Platform-independent port of KeePass Password Safe that works on Windows, Mac and Linux to name a few. Compatible with existing KeePass password databases.
• KeePassDroid — A port of the KeePass Password Safe for the Android platform.

Password managers to pass up

RoboForm — Though it has a version specifically for use with USB flash drives, RoboForm is reliant on a web browser to function; it is not suitable for managing desktop application passwords and may not function in all browsers.

The song|movie “name” could not be used because the original file could not be found. Would you like to locate it?

Not so useful when a lot of files are moved at once. But if a directory containing hundreds of files needs to be moved, to a larger hard drive for example, locating files individually becomes too much work.

Of course, there’s always the “fresh start” approach. Just delete everything and start over… A good idea if iTunes ratings, playlists and play counts are not considered important keepsake. But to users who spend a good deal of time carefully tweaking song ratings and organizing playlists, scrapping their information and starting fresh may not be a desirable approach.

Thankfully there’s an easy workaround for the problem. A sane alternative to relocating files one at a time (like when you move hundreds or thousands of files). The workaround straightforward and reversible. And useful whenever moving many files together.

Applying it will allow users to move files around in bulk, while still hanging onto most of the iTunes metadata they have likely grown fond of.

Note:Some less significant meta information, such as Last Played time, is lost during the process.

Addressing the problem

Here are step-by-step instructions for updating the Library XML  and recreate the iTunes Library to enable file relocation when iTunes files are manually organized.

Backup iTunes Music Library

Before you move your files, do the following:

1. Close iTunes for now.
2. Create copies of the following:
   • (2x) Library XML (iTunes Music Library.xml)
   • (1x) Library ITL (iTunes Library.itl)
     Vista users can find the Library XML at C:\Users\<Username>\Music\iTunes
     Mac users can find the Library XML at /Users/<Username>/Music/iTunes

Relocate iTunes files as necessary

Before moving files note the following two necessary pieces of information:

1. The path where the files were previously stored (e.g. D:\Downloads\Music)
2. The path where the files will end up (e.g. E:\Music\Archive)

Hack the Library XML and delete the Library ITL

After your files are moved, complete the following steps to implement the fix.
Tip: Do not open iTunes during this process.

1. Open one of the Library XML copies created.
2. Perform a Find/Replace using the following inputs:
   Find: Path where files were moved from
   Replace: Path where files were moved to
3. Save and close the document, noting which file contains the modifications.
4. Delete the existing Library ITL.

Restart iTunes

Once the Library ITL has been deleted, restart iTunes. The application will open with a blank library; custom playlists, music and other items will be gone. To recover the data do the following:

1. Import the modified copy of the Library XML.
   In iTunes 8 and 9, choose File > Library > Import Playlist…
2. Navigate to and Open the modified Library XML.
3. iTunes will then import the file contents into the blank library.
4. Wait for the import to complete.

iTunes may display a dialog if any files cannot be located during the import process. The dialog can occur as a result of improper changes to the Library XML, or because the Library XML already contained references to files previously moved or renamed.

Wrapping up

Once the updated playlist is imported into iTunes lost songs and other media should be restored though there will be some duplicate playlists. Before deleting the extra playlists confirm the changes are working as expected. Once satisfied, delete the duplicates in the Library and enjoy your hard work. You’ve earned it.

If something goes wrong

If something goes wrong during the process, or if the results are not as expected, the original Library can be restored from the backup files created. To do so, close iTunes and copy the backup Library ITL and XML back to their original locations (overwrite existing files, if prompted) and restart iTunes to restore the previous Library.

Additional resources

Apple Support KB Article HT1451: How to re-create your iTunes library