Benchmark Comparison of 3 ROMs

[Ridgeland]

ROMs compared:
HCV9 - B6 from
http://www.androidtablets.net/forum...-tc21-hcv925-b7-kyros-7015-all-4-02-11-a.html
Watery-Kyros v1.2b from
http://www.androidtablets.net/forum...tery-kyros-v1-2b-custom-firmware-mid7015.html
And the Coby ROM update.zip
From the Coby ROM I tested three variations.
1. Base - almost nothing except a fresh install of Coby's update.zip with the addition of gscript, AndExplorer, SuperUser and AndTerm.
2. Market - Android Market plus Amazon Appstore
3. Performance - for lack of a better name, This is my everyday system, It does not have Market or Amazon Appstore. It has all the apps I have chosen as keepers.


Repeating that makes five system benchmarks:
HCV9-B6
WK1.2b
Coby {Base, Market, Performance}


For each system I used four benchmark tools:
CaffeineMark 3.0 - cm3.apk pulled from HCV9-B6 /system/app
System Benchmark (ABenchMark.apk) from Market
NBench from Market
Quadrant Standard Edition from Market


And the results:


[FONT=Courier New, monospace]CaffeineMark 3.0[/FONT]
[FONT=Courier New, monospace]- BenchMark - ...... HCV9 ..... WK ..... C-Base .. C-Mkt .. C-Perf[/FONT]
[FONT=Courier New, monospace]Sieve score = ....... 798 .... 2016 ..... 758 ..... 855 ..... 830[/FONT]
[FONT=Courier New, monospace]Loop score = ........ 946 .... 3272 ..... 934 .... 1022 .... 1021[/FONT]
[FONT=Courier New, monospace]Logic score = ....... 677 .... 3478 ..... 671 ..... 736 ..... 739[/FONT]
[FONT=Courier New, monospace]String score = ...... 846 .... 1164 ..... 784 ..... 807 ..... 828[/FONT]
[FONT=Courier New, monospace]Float scroe = ....... 676 .... 1656 ..... 610 ..... 733 ..... 463[/FONT]
[FONT=Courier New, monospace]Method score = ...... 708 .... 1811 ..... 655 ..... 763 ..... 764[/FONT]
[FONT=Courier New, monospace]Overall score = ..... 769 .... 2076 ..... 727 ..... 813 ..... 753[/FONT]


[FONT=Courier New, monospace]System Benchmark (aka ABenchMark.apk )[/FONT]
[FONT=Courier New, monospace]- BenchMark - ...... HCV9 ..... WK ..... C-Base .. C-Mkt .. C-Perf[/FONT]
[FONT=Courier New, monospace]Memory: ............. 101 ..... 108 ..... 106 ..... 103 ..... 105[/FONT]
[FONT=Courier New, monospace]CPU Integer: ........ 221 ..... 239 ..... 236 ..... 233 ..... 236[/FONT]
[FONT=Courier New, monospace]CPU Float: .......... 148 ..... 162 ..... 159 ..... 157 ..... 161[/FONT]
[FONT=Courier New, monospace]2D Graphics: ........ 200 ..... 207 ..... 197 ..... 202 ..... 213[/FONT]
[FONT=Courier New, monospace]3D Graphics: ......... 87 ...... 93 ...... 91 ...... 91 ...... 92[/FONT]
[FONT=Courier New, monospace]Database IO: ........ 100 ..... 100 ...... 85 ...... 85 ...... 75[/FONT]
[FONT=Courier New, monospace]SD Card Write: ....... 32 ...... 33 ...... 35 ...... 34 ...... 34[/FONT]
[FONT=Courier New, monospace]SD Card Read: ........ 74 ...... 73 ...... 74 ...... 70 ...... 73[/FONT]
[FONT=Courier New, monospace]Total scores: ....... 963 .... 1015 ..... 983 ..... 975 ..... 989[/FONT]


[FONT=Courier New, monospace]NBench - [/FONT]
[FONT=Courier New, monospace]- BenchMark - ...... HCV9 ..... WK ..... C-Base .. C-Mkt .. C-Perf[/FONT]
[FONT=Courier New, monospace]Memory: ........... 1.054 ... 1.133 ... 1.129 ... 1.111 ... 1.123[/FONT]
[FONT=Courier New, monospace]Integer: .......... 1.919 ... 2.084 ... 2.08 .... 2.045 ... 2.079[/FONT]
[FONT=Courier New, monospace]Float: ............ 0.546 ... 0.593 ... 0.589 ... 0.58 .... 0.589[/FONT]


[FONT=Courier New, monospace]Quadrant Standard Edition - Info only[/FONT]
[FONT=Courier New, monospace]- BenchMark - ...... HCV9 ..... WK ..... C-Base .. C-Mkt .. C-Perf[/FONT]
[FONT=Courier New, monospace]Current freq MHz .... 660 ..... 720 ..... 720 ..... 720 ..... 720[/FONT]
[FONT=Courier New, monospace]Memory: - Total always showed 157820 kB[/FONT]
[FONT=Courier New, monospace]Free: .............. 6316 .... 5044 ... 20720 ... 26092 ... 10456[/FONT]
[FONT=Courier New, monospace]Inactive: ......... 77052 ... 68408 ... 58548 ... 51508 ... 64712[/FONT]




First reaction: Run that CM3 test on WateryKyros again. I did, the second run the overall score was 2061 - No real difference. But compare the "Float" test between the test suites. There WateryKyros is more in line with the other tests.


Watery Kyros totally blows the others away in CM3. None of the others scored half as much. Why? Why did HCV9 take a hit in clock speed? I expected to prove that having Market hurts performance. That was not proven by the tests. That surprised me since I've seen too many side effects to keep Market on my everyday system. Also I expected to see BASE be noticeably better than BASE + apps (market, games etc), again data did not prove it so. Maybe the tests are subject to background activity. In Ubuntu when cron runs mlocate in the background I get a lot hard drive noise even though I'm not using the hard drive. Also except for WK CM3 I only ran one test. To be really fair I should run a dozen of each combination and average them, ain't gonna happen.


Having all these ROMs and booting each as needed is trival with gscript, using ClockWorkMod. It's also very simple without gscript. Just remember to rename your ClockWorkMod backup folder to a meaningful name like "Base", "WaterKyros.1.2b.2011.03.29" etc


These tests were all run on the same Kyros MID7015. From one Kyros to the next I expect variation in hardware from production dates, supplier changes, component lot/batch changes etc. YMMV.


Run your own tests comparing ROMs. Post them here. To format a table see Font in the FAQ. Remember than browsers reduce multiple spaces to one space, so I use "." rather than multiple spaces.
http://www.androidtablets.net/forum/misc.php?do=bbcode
All the ROMs are subject to updates so thing should change.

 

[xaueious]

JIT isn't stable on 1.2b yet. There's little reason to benchmark that sort of thing. We don't have a workable kernel source, so everyone has been making mainly minor or aesthetic changes. None of that stuff is going to show up in a synthetic bench.

 

[tipstir]

B6 would be slower try B7.. But I use 3 different tweak tools and my own. 830 out of B7 but it's more graphical than the other two. Again testing with GE wouldn't be good. How are you testing these ROM which model of the 7015 are you using? 7015 first gen or 7015A or 4G or 8G model?

I think the B1 max score was 857, B3 max score 833 was the fastest but B6 and B7 max score 830 are made for those who want to play Angry Birds and watch HD, and etc..

 

[Ridgeland]

MID 7015 - 4G
I bought it in mid December. I used the first Coby update and I ignored the second update as it sounded like a step backwards with ADB, rooting etc. My microSD is class 4 - 8GB.
I want to learn how to up performance and still keep the apps I want, the most demanding of which is Angry Birds
I don't use the Google suite. I restore my Market backup if I want to try a new app from Market. So next I'll test things like Airplane mode, turning off WiFi when not in use etc. Which such simple things bump up performance?

 

[WhiteWhiskers]

System Benchmark (ABenchMark.apk) from Market

Not sure of the Benchmark program you used. Looking on the Android Market there is something called System Benchmark by AnTuTu. Is that what you used?

Would be interesting to see the 3D graphics results when the test is run with the graphics patch. If you can tell me what benchmark program you used, I'll run it and post the results for CyanogenMod which is running Froyo.

 

[Ridgeland]

AnTuTu is it.
I did have the graphics patch installed on WK, C-Market and C-Perf. Gotta have Angry Birds you know.

 

[tipstir]

Well on B7 Angry Birds plays much better gone are those white boxes. I even loaded up old AB from 2010 and that looks so different on B7 I was expecting to see white boxes. Everything is all smooth and better now.. Rage Of Thunder II should be even greater now. Still I have to have my music videos to watch. Coby they works fine..Rotate them if I want or leave them as is.. Benchmark just tells you what you got for your money. The CMP738A with all my tweaks and CPU Boost I get 1,667 but it can't play AB Rio or any AB because it doesn't have 3D support is weaker than 7015. Trade off and such Coby is the winner.. Now go and free up those cage birds in Rio.. LOL!

 

[WhiteWhiskers]

Here are my benchmark results using System Benchmark by AnTuTu:

..................C-Market.....CyanogenMod 6.1.0.......Motorola Droid
Memory:..........108.................109..........................200
CPU Integer......237.................239..........................362
CPU Float:........160.................163...........................88
2D Graphics:.....208.................268..........................161
3D Graphics:.....95....................83..........................159
Database I/O:...85....................85..........................195
SD Card Write:..31....................33...........................44
SD Card Read:...78....................81...........................47
Total:..............1002...............1061........................1256

The CyanogenMod is only slightly faster, seems like the total score increase was mostly due to an increase in 2D performance. Included the Droid results just to see how a different device would compare.

Also, in doing the benchmark test on the Coby, my misbehaving SD Card finally went over the other side and died. After I backed up my current ROM, tried to restore CyanogenMod and received an md5 checksum failure. Then when I tried to restore the backup I just made, it wasn't listed! I had an older backup on the card, but I still lost some stuff.

Then no matter what I tried to do to format the SD card, it just wouldn't format. I could delete the files but they would always come back. Finally went to the store and bought a new card, snapped the old card in half!

Moral of the story, be careful of benchmark programs. They sometimes push hardware over the edge towards failure.

 

[Ridgeland]

WhiteWhiskers,
Were do we get CyanogenMod for Coby Kyros? I'll try it too.

 

[WhiteWhiskers]

Were do we get CyanogenMod for Coby Kyros?

Here: Coby Kyros 7015 - How to install CyanogenMod (Froyo based) - xda-developers

It's a true Froyo Android 2.2. Also very tricky to install. You need to be comfortable with ADB, pulling files off the Coby from the Eclair build, and then pushing them back once Froyo is installed. The instructions are very clear.

 

[kawal]

Can someone backup their file in Clockwork recovery mode from the CyanogenMod - and post here - that should make it easy for others to copy the backup.

Here: Coby Kyros 7015 - How to install CyanogenMod (Froyo based) - xda-developers

It's a true Froyo Android 2.2. Also very tricky to install. You need to be comfortable with ADB, pulling files off the Coby from the Eclair build, and then pushing them back once Froyo is installed. The instructions are very clear.

 

[Ridgeland]

Thanks WhiteWhiskers.
I installed CyanogenMod soon after your post but it was very difficult. There were several issues to get around. Like manual mount to get around read-write permissions issues etc. I got it working and like it, but I did not get WiFi working so for me that was a show stopper. I also expected flac files to play with the default music player but that did not happen. Speed tests were like Watery with the CM3 blasting high. The look and feel are nice.

kawal,
I use Linux and started a script to use for the installation. Do you have ADB working? Linux? If no one posts a backup we can help you install it. The instructions may be clear but it ain't easy.

 

[tipstir]

If you rip out the GE and extra code you would have one super duper Rooted ROM. But that's going to the extreme. Not like were are testing HDD/CPU it's more like RAM/CPU. The 7015A is faster than the 7015.

 

[Ridgeland]

My approach is to rip out the GE (Google Experience, Google Market extras) by having multiple restores in ClockWorkMod.
I have a nearly original ROM with just SU and gscript and term. I've expanded and forked that ROM to have Google Market on one restore and another without it. I download apps using GoogleMarket and if I like them I use ADB to pull the app to the PC for backup. I can later boot to the restore that does not have Market and use ADB to push the new app to it. Only those apps linked to GE-maps etc fail. I also thinned out the apps in /system/app. I moved the following apps to /sdcard/Android/StockApps as a backup then sync and reboot:
AlarmClock.apk, Browser.apk, Calculator.apk, Email.apk, HTMLViewer.apk, LiveWallpapers.apk, LiveWallpapersPicker.apk, MagicSmokeWallpapers.apk, Phone.apk, PicoTts.apk, SoundRecorder.apk, TelephonyProvider.apk, Term.apk, TtsService.apk, VisualizationWallpapers.apk
I have a much better Browser, DolphinHD (thanks to tipstir) and use RealCalc. LiveWallpapers caused more FCs than it was worth.

 

[WhiteWhiskers]

I did not get WiFi working so for me that was a show stopper

Wi-Fi should work. The trick is to disable ethernet in one of the settings. Not sure of its purpose. Once you do that Wi-Fi should work. With ethernet enabled the Wi-Fi won't connect unless you toggle the Wi-Fi off and then on.