Definitely needs looking into. I just ran a bunch of tests using Blaze.IO’s latest technology for measuring Mobile performance. I then compared that with WebPageTest.org (whose service they’re using) and then extracted all the HAR reports for viewing.
Ignore for the moment the timings for WebPageTest.org vs. Blaze.IO – the actual number of GET requests for the page should be identical (each are requesting the exact same page). But then it gets weirder – the HAR report shown on Blaze’s page shows 42 GET requests but if you download the actual one from the test it shows 53 GET request. Now start comparing all the times. Using WebPageTest.org through Chicago and a DSL connection it takes 3.419 seconds vs. Blaze at 3.78 seconds.
Think about it for a moment. Blaze gets 96kb down in 3.78 seconds (42GET requests) – WebPageTest gets 47kb down in 3.419 seconds. Throughput wise the Android device is twice as fast as a DSL connection. (I don’t think so).
Also one other HUGE issue. Androids browser is single threaded – which means the elements (GET Requests) load one after the other. All of the charts show data loaded concurrently – that’s not true.
Blaze.io Mobile test on www.jetsms.com
42 GET requests, 3.78 seconds, 96.1 Kb
The actual HAR report – 53 GET requests, 115.1KB and 3.78 seconds
WebPageTest.org report – 13 GET requests, 47KB and 3.419
The HAR report exported from the WebPageTest.org test
13 GET requests, 43.2KB and 3.76 seconds