I’ve been in software a LONG time. I’ve been in mobile devices and mobile computing even longer… What I’m about to say may draw a great deal of criticism and some harsh debate (and at least a great deal of, “well what did you expect, Chris..? iOS 8 IS in beta after all…). But to tell you the truth, I’ve been a registered Apple developer for a while now, (since just before iOS 6 was in Beta) and as a QA Guy, I’m very good at identifying patterns and trends… it’s what I do. So, here goes…
While iOS 8 Beta 2 improves some things over Beta 1, Beta 2 seems a worse train wreck than Beta 1.
In other words, the latest development milestone release (beta) is worse than the last.
In past releases of iOS 6 and iOS 7, by the time Beta 2 was released, the OS was usable. While the official stance is always to put beta software on non-mission critical devices, as a software tester, if I don’t have it on a primary computer or device, I’m not going to give it a real good burn in test. There’s a difference between working with a device and living in one; and in my opinion, you’re going to find more bugs, buried deeper in the code by living in a device than by simply using it and running test cases. Don’t get me wrong. I know that formal structured testing is a MUST. However, living with a device and using it outside of a structured test can provide more information on the overall performance, look, feel, etc. of a system than can be found in a formal test setting (though, in all honesty, this ad-hoc information is USELESS without the formal feedback provided by structured testing, so you can’t cut corners…)
This is the major reason why I run new iOS betas when they become available. Yes, yes… I have the beta software itch. I gotta have the new stuff; but I file bug reports as I find issues. This week, I think I’ll be filing a boat load of them. I’ve bumped into the following and have a great deal of feedback to provide Apple on iOS 8 Beta 2. The following issues are listed in no particular order of severity or priority:
- Personal Hotspot – Personal hotspot fails to provide internet service to devices connected to it for over 15 minutes. Even though the iPhone indicates that a guest device is connected, and the device itself has internet connectivity, the guest won’t have access to the internet through the host after a 15 or so minute period has elapsed. I’m not certain why yet. It’s not a matter of the iPhone going to sleep and cutting internet access off. I’ve got my iPhone set to sleep after a few minutes. I’m good for the first 15 minutes or so, and then internet access just disappears. This issue needs more exploration. I will report more on it as I find out more.
- Bluetooth and Bluetooth LE – This is the biggest train wreck of them all, I think. The BT stack is a hot mess, failing to communicate with any number and types of devices. BT-LE service is nearly unusable at the moment. My iPhone 5 won’t (officially) pair with any LE device I’ve got, no matter how hard I try, though LE notifications can travel across the link at times. My Pebble Steel is little more than a hockey puck on my wrist right now. I have Casio calculator watches that are smarter than my Pebble right now as a result…Bluetooth connectivity, especially BT audio, is spotty at best and doesn’t always work. There are times when I try to listen to music while at the office, or watch videos after work. Audio comes across the link inconsistently with both audio and video media. Sometimes it doesn’t come at all. Sometimes, it drops during brief periods of silence in the audio track and may or may not pick back up after the silence ends. During music playback, this happens in between songs, and can happen during podcast playback when there’s a brief silence among the show hosts. The only reason I can attribute to this, is that the signal activity in the audio track (of either media type) drops and the BT device and stack on the iPhone are trying to conserve power by cutting off use of the radio and the device when it senses inactivity. The problem is, it’s WAY too sensitive.ALL of my Bluetooth devices – from different headsets, keyboards, Pebble, etc. – also drop connections on a random basis. I have not been able to put any kind of a pattern to the losses of connectivity; and it doesn’t seem to be limited to any one kind of device or during or after any specific kind of activity or with any specific media. Connectivity just drops, and reestablishing it is VERY difficult. Currently, it may require turning either device on or off, tuning the Bluetooth radio on either device on or off, or forgetting devices on either end of the pairing chain, and then repairing. This often has to be repeated, as it doesn’t always work. Part of this was a problem with BT-LE in iOS 7.x and it seems to be amplified in iOS 8 Betas 1 and 2. Which brings me to the next big issue…
- Battery Life – Oh, it sucks. Anything processor or radio intensive – like playing a game or long data downloads – really sucks the life out of my iPhone 5′s battery. I wouldn’t make a point of this if it were the same thing in iOS 7; but it’s not. It is clearly more noticeable in iOS 8. When things are (seemingly) working right, the same battery that may last the entire day with moderate game play in iOS 7 may only last 1/2 that time in iOS 8. A fix is needed here for certain.
- Performance – The train derails and comes off the tracks, here too. There are still a number of issues with core apps. Most, if not all of them – Mail, Calendar, Podcasts, Music, Clock, Siri, Maps, etc. – just plain don’t work right. Previously working functionality just doesn’t work (deleting messages is still broken, calendar info doesn’t sync or display right, audio doesn’t play correctly, tracks are often skipped and don’t play, even if they are local to the device, Siri is being belligerent and won’t listen, etc…); and the device clearly seems to try to compensate for it.For example, there are performance stutters throughout the ENTIRE system. Any and ALL apps appear to freeze but then release and catch up to where they need to be based on physical or data input. Scrolling through posts on Facebook or messages in Mail seems to be a big hic-cough right now. My phone can freeze at any particular moment and may or may not come back, requiring a hard reset (home button-power button until the screen blacks out and the Apple logo appears, then release both buttons) before it will come back; and then it takes about twice as long for that process to complete as it did in iOS 7.x… I’ve also found that my phone will just spontaneously reboot, usually at an inconvenient time. It’s happened three or four times since I installed Beta 2, just the other day.I’ve also had the screen go completely black out of nowhere with only a white spinner appearing on the screen. This usually happens during navigation (with Apple Maps, but has also happened with Google Maps). The last time it was right near the end of a route (and of course, I didn’t know where the destination was located at, so I nearly missed it…). The OS, just simply isn’t reliable by any stretch at this time.
- Storage and Logging – I have a 16GB iPhone 5. I know exactly what it will hold and what it won’t. I know which apps I can plug and chug off the device when something interesting is released, how much music I can have on the device, etc. With iOS 8, I’m at a total loss. The device is CLEARLY in debug mode by default at this point, as it seems to be going through a large amount of event logging.Storage on the device gets eaten up very, very quickly. I’ve had Beta 2 installed for less than a week, and I’ve already had to blow it and restore it twice because I’ve strangely run out of storage space. When this happens, the battery life tanks, the device gets VERY warm and the device becomes very unreliable.
To say that I am disappointed with the overall stability and performance of iOS 8 Beta 2 is an understatement. I was really looking forward to it after working with Beta 1 for about 15 days. Historically, the reliability of Apple software goes up as the asset moves through its development lifecycle. Unfortunately, that’s NOT the case here with iOS 8. Beta 2 clearly feels less finished than Beta 1.
At this point, I have no idea what to expect from Beta 3 and beyond. However, I would HOPE that Apple is aware of the issues with Beta 2 and will push to get a replacement for it – i.e. Beta 3 – out sooner rather than later. While I will be filing these issues as bugs, I would hope that they all end up being duplicates of bugs found by other developers (meaning, they are already aware of the issues…).
I wish I could comment of some of iOS 8′s newer features at this point. However, I’m loath to do so, not because of any NDA that comes with my developer’s account (Apple changed their NDA so I can speak freely about any and all items in either Yosemite or iOS 8. I cannot, however, post screen shots…yet); but because I’m having trouble with regression issues and with legacy functionality, let alone the new and shiny stuff.
Suffice it to say that there are issues throughout the ENTIRE system at this point. Apple has a long row to hoe with iOS 8; and if they wish to make a July/ August iPhone announcement and a September/ October release, then they better get their butts in gear and start pumping out the testable code. With what I’m seeing, it could be quite a long time before iOS 8 is ready for GM or RTM status. With iPhone 6 highly anticipated to ship with iOS 8 AND with new screen sized and form factors, this is gearing up to be one of – if not the MOST – highly anticipated iPhone releases yet. It could likely surpass that of the original iPhone or the iPhone 3G as well.
What questions do you have about iOS 8? Is there anything in particular that you’re curious about? I’d love to have your input and questions on the new iDevice mobile OS. Do you find it compelling? Are you an existing or potentially new iDevice user? Are you someone who left the iPhone behind and went with either an Android device or Windows Phone when iOS 7 was released? Does the look and feel of iOS 8 interest you? Are you interested in switching back to iPhone with iPhone 6 when it’s released later this year? Why don’t you meet with me in the discussion area below, and ask a question or two? If possible, I will answer your question(s) and/ or address them in a separate column as soon as possible.