Journal tags:iphone

8

sparkline

Web Audio API update on iOS

I documented aweird bug with web audio on iOSa while back:

On some pages ofThe Session,as well as the audio player for tunes (using the Web Audio API) there are also embedded YouTube videos (using thevideoelement). Press play on the audio player; no sound. Press play on the YouTube video; you get sound. Now go back to the audio player and suddenly you do get sound!

It’s almost like playing avideooraudioelement “kicks” the browser into realising it should be playing the sound from the Web Audio API too.

This was happening on iOS devices set to mute, but I was also getting reports of it happening on devices with the sound on. But it’s that annoyingly intermittent kind of bug that’s really hard to reproduce consistently. Sometimes the sound doesn’t play. Sometimes it does.

I found a workaround but it was really hacky. By playing a one-second long silent mp3 file usingaudio,you could “kick” the sound into behaving. Then you can use the Web Audio API and it would play consistently.

Well, that’s all changed with the latest release of Mobile Safari. Now what happens is that the Web Audio stuff plays…for one second. And then stops.

I removed the hacky workaround and the Web Audio API started behaving itself again…butyour device can’t be set to silent.

The good news is that the Web Audio behaviour seems to be consistent now. It only plays if the device isn’t muted. This restriction doesn’t apply tovideoandaudioelements; they will still play even if your device is set to silent.

This descrepancy between the two different ways of playing audio is kind of odd, but at least now the Web Audio behaviour is predictable.

You can hear the Web Audio API in action by going toany tune on The Sessionand pressing the “play audio” button.

Web Audio API weirdness on iOS

I told you about howI’m using the Web Audio API on The Sessionto generate synthesised audio of each tune setting. I also said:

Except for some weirdness on iOS that I had to fix.

Here’s that weirdness…

Let me start by saying that this isn’t anything to do with requiring a user interaction (the Web Audio API insists on some kind of user interaction to prevent developers from having auto-playing sound on websites). All of my code related to the Web Audio API is inside aclickevent handler. This is a different kind of weirdness.

First of all, I noticed that if you pressed play on the audio player when your iOS device is on mute, then you don’t hear any audio. Seems logical, right? Except if using the same device, still set to mute, you press play on avideooraudioelement,the sound plays just fine. You can confirm this by going toHuffdufferand pressing play on any of theaudioelements there, even when your iOS device is set on mute.

So it seems that iOS has different criteria for the Web Audio API than it does foraudioorvideo.Except it isn’t quite that straightforward.

On some pages ofThe Session,as well as the audio player for tunes (using the Web Audio API) there are also embedded YouTube videos (using thevideoelement). Press play on the audio player; no sound. Press play on the YouTube video; you get sound. Now go back to the audio player and suddenly youdoget sound!

It’s almost like playing avideooraudioelement “kicks” the browser into realising it should be playing the sound from the Web Audio API too.

This was happening on iOS devices set to mute, but I was also getting reports of it happening on devices with the sound on. But it’s that annoyingly intermittent kind of bug that’s really hard to reproduce consistently. Sometimes the sound doesn’t play. Sometimes it does.

Following my theory that the browser needs a “kick” to get into the right frame of mind for the Web Audio API, I resorted to a messy little hack.

In the event handler for the audio player, I generate the “kick” by playing a second of silence using the JavaScript equivalent of theaudioelement:

var audio = new Audio('1-second-of-silence.mp3');
audio.play();

I’m not proud of that. It’s so hacky that I’ve even wrapped the code in some user-agent sniffing on the server, and Ineverdo user-agent sniffing!

Still, if you ever find yourself getting weird but inconsistent behaviour on iOS using the Web Audio API, this nasty little hack could help.

Update:Time to remove this workaround.Mobile Safarihas been updated.

The magical and the mundane

The iPhone—and by extension, the smartphone—is a decade old. Ian Bogost has writtenan interesting piece in The Atlanticcharting our changing relationship with the technology.

First, it was like a toy dog:

A device that could be cared for, and conspicuously so.

Then, it was like a cigarette:

A nervous tic, facilitated by a handheld apparatus that releases relief when operated.

Later, it was like a rosary:

Its toy-dog quirks having been tamed, its compulsive nature having been accepted, the iPhone became the magic wand by which all worldly actions could be performed, all possible information acquired.

Finally, it simply becomes…a rectangle.

Abstract, as a shape. Flat, as a surface. But suggestive of so much. A table for community. A door for entry, or for exit. A window for looking out of, or a picture for looking into. A movie screen for distraction, or a cradle for comfort, or a bed for seduction.

Design dissolves in behaviour.This is something that Ben wrote about recently in his excellentSlapdasheryseries:“Everything’s amazing and nobody’s happy.”

Technology tweaks our desire for novelty; but as soon as we get it we’re usually bored. There are no technologies that I can think of that haven’t become mundane.

This is something I touched on in my talk last year at An Event Apart. There’s a thread throughout the talk about Arthur C. Clarke, and of course I quotehis third law:

Any sufficiently advanced technology is indistinguishable from magic.

I propose an addendum to that:

Any sufficiently advanced technology is indistinguishable from magicat first.

The magical quickly becomes the mundane. That’s exactlythe point that Louis CK is makingin the piece that Ben references.

Seven years ago Frank wrote his wonderful essayThere Is A Horse In The Apple Store:

I have a term called a “tiny pony.” It is a thing that is exceptional that no one, for whatever reason, notices. Or, conversely, it is an exceptional thing that everyone notices, but quickly grows acclimated to despite the brilliance of it all.

We are surrounded by magical tiny ponies. I mean, just think: right now you are reading some words at a URL on the World Wide Web. Even more magically, I just published some words at my own URL on the World Wide Web. That still blows my mind! I hope I never lose that feeling.

Someday

Inthe latest issueof Justin’s excellentResponsive Web Designweekly newsletter, he includes a segment called “The Snippet Show”:

This is what tells all our browsers on all our devices to set the viewport to be the same width of the current device, and to also set the initial scale to 1 (not scaled at all). This essentially allows us to have responsive design consistently.

<meta name= "viewport" content= "width=device-width, initial-scale=1" >

Theviewportvalue for themetaelement was invented by Apple when the iPhone was released. Back then, it was a safe bet that most websites were wider than the iPhone’s 320 pixel wide display—most of them were 960 pixels wide…because reasons. So mobile Safari would automatically shrink those sites down to fit within the display. If you wanted to over-ride that behaviour, you had to use themeta viewportgubbins that they made up.

That was nine years ago. These days, if you’re building a responsive website, youstillneed to include thatmetaelement.

That seems like a shame to me. I’m not suggesting that the default behaviour should switch to assuming a fluid layout, but maybe the browser could just figure it out. After all, the CSS will already be parsed by the time the HTML is rendering. Perhaps a quick test for the presence of acrawlbarcould be used to trigger the shrinking behaviour. No crawlbar, no shrinking.

Maybe someday the assumption behind the current behaviourcouldbe flipped—assume a website is responsive unless the author explicitly requests the shrinking behaviour. I’d like to think that could happen soon, but I suspect that a depressingly large number of sites are still fixed-width (I don’t even want to know—don’t tell me).

There are other browser default behaviours that might someday change. Right now, if I typeexampleinto a browser, it will first attempt to contacthttp://examplerather thanhttps://example.That means theexampleserver has to do a redirect, costing the user valuable time.

You can mitigate this by putting your site onthe HSTS preload listbut wouldn’t it be nice if browsers first checked forHTTPSinstead ofHTTP?I don’t think that will happen anytime soon, but someday…someday.

Orientation and scale

Paul Irish,Divya ManianandShi ChuanlaunchedMobile Boilerplaterecently—a mobile companion site toHTML5 Boilerplate.

There’s some good stuff in there but I was a little surprised to see that themeta viewportelement included values forminimum-scale=1.0, maximum-scale=1.0, user-scalable=no:

<meta name= "viewport" content= "width=device-width, target-densitydpi=160dpi, minimum-scale=1.0, maximum-scale=1.0, user-scalable=no" >

Settinguser-scalable=nois pretty much the same as settingminimum-scale=1.0, maximum-scale=1.0.In any case,I’m not keen on it.Like Roger,I don’t think we should take away the user’s right to pinch and zoom to make content larger. That’s why my usual viewport declaration is:

<meta name= "viewport" content= "width=device-width, initial-scale=1" >

Yes, I know that most native apps don’t allow you to zoom but I see no reason to replicate that failing on the web.

But there’s a problem. Allowing users to scale content for comfort would be fine if it weren’t fora bug in Mobile Safari:

When the meta viewport tag is set tocontent= "width=device-width,initial-scale=1",or any value that allows user-scaling, changing the device to landscape orientation causes the page to scale larger than 1.0. As a result, a portion of the page is cropped off the right, and the user must double-tap (sometimes more than once) to get the page to zoom properly into view.

This is really annoying soShi Chuan set about fi xing the problem.

His initial solution was to keepminimum-scale=1.0, maximum-scale=1.0in themeta viewportelement but then to change it using JavaScript once the user initiates a gesture (thegesturestartevent is triggered as soon as two fingers are on the screen). At the point, thecontentattribute of themeta viewportelement gets updated to readminimum-scale=0.25, maximum-scale=1.6,the default values:

var metas = document.getElementsByTagName('meta');
var i;
if (navigator.userAgent.match(/iPhone/i)) {
document.addEventListener( "gesturestart", gestureStart, false);
function gestureStart() {
for (i=0; i<metas.length; i++) {
if (metas[i].name == "viewport" ) {
metas[i].content = "width=device-width, minimum-scale=0.25, maximum-scale=1.6";
}
}
}
}

That works nicely but I wasn’t too keen on the dependency between the markup and the script. If, for whatever reason, the script doesn’t get executed, users are stuck with an unzoomable page.

I suggestedthat the script should alsosetthe initial value tominimum-scale=1.0, maximum-scale=1.0:

var metas = document.getElementsByTagName('meta');
var i;
if (navigator.userAgent.match(/iPhone/i)) {
for (i=0; i<metas.length; i++) {
if (metas[i].name == "viewport" ) {
metas[i].content = "width=device-width, minimum-scale=1.0, maximum-scale=1.0";
}
}
document.addEventListener( "gesturestart", gestureStart, false);
}
function gestureStart() {
for (i=0; i<metas.length; i++) {
if (metas[i].name == "viewport" ) {
metas[i].content = "width=device-width, minimum-scale=0.25, maximum-scale=1.6";
}
}
}

Now the markup still contains the robust accessible default:

<meta name= "viewport" content= "width=device-width, initial-scale=1" >

…while the script takes care of initially setting the scale values and also updating them when a gesture is detected. Here’s what’s happening:

  1. By default, the page is scaleable because the initialmeta viewportdeclaration doesn’t set aminimum-scaleormaximum-scale.
  2. Once the script loads, the page is no longer scalable because bothminimum-scaleandmaximum-scalehave been set to1.0.If the device is switched from portrait to landscape, the resizing bug won’t be triggered because scaling is disabled.
  3. When thegesturestartevent is detected—indicating that the user might be trying to scale the page—theminimum-scaleandmaximum-scalevalues are updated to allow scaling. At this point, if the device is switched from portrait to landscape, the resizing bugwilloccur because the page is now scaleable.

Jason Weaverpoints out thatyou should probably detect for iPad too.That’s a pretty straightforward update:

if (navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPad/i))

Mathias Bynensupdated the codeto usequerySelectorAllwhich is supported in Mobile Safari. Here’s the code I’m currently using:

if (navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPad/i)) {
var viewportmeta = document.querySelector('meta[name= "viewport" ]');
if (viewportmeta) {
viewportmeta.content = 'width=device-width, minimum-scale=1.0, maximum-scale=1.0';
document.body.addEventListener('gesturestart', function() {
viewportmeta.content = 'width=device-width, minimum-scale=0.25, maximum-scale=1.6';
}, false);
}
}

You can try it out onHuffduffer,Salter Cane,Principia Gastronomicaand right here onAdactio.

Right now there’s still a little sluggishness between the initial pinch-zoom gesture and the scaling; the scale values (0.25 - 1.6) don’t seem to take effect immediately. A second pinch-zoom gesture is often required. If you have any ideas for improving the event capturing and propagation, dive in there.

Update:the bug has beenfixed in iOS 6.

Accessimobility

When the originalcame out, it was pretty impressive. Every subsequent iteration has featured improvements of varying levels of impressiveness. The latest version, though, has bowled me over.

I’m not talking about faster speeds, bigger storage, or any new fangled gizmos or geegaws. I’m talking aboutVoiceOver in the iPhone.

Watch the videoto get the low-down. Then read this first-hand account ofusing an accessible touch-screen device.

That’s quite a design challenge: anaccessibletouch-screendevice! I doff my hat in the general direction of the Apple engineers who rose to this challenge.

Speaking of exciting developments in the world of accessibility…

The secondAccessibility 2.0 conferencewill be taking place inLondononthe 22nd of this month.It was a cracking eventlast yearand, judging bythe line-up,this year is going to be a winner too.Grab a ticket now.

iPhone, uPhone, we all scream for iPhone

Like everyone else in the geekosphere, I was vicariously attending MacWorld through all those Mac rumour sites as well asTwitter.Unsurprisingly, theiPhonehas a lot of people excited.

Jason Kottke got so excited, he immediately whipped upa cardboard scale model of the iPhone.Jeff Croft is really excited thatthe iPhone is running Leopard.

A lot of people are talking about how to get their heads around this thing. Is it a phone or is it a PDA? John Allsopp wouldn’t mindlosing the phone functionality altogether.

Mike Davidson thinks that the iPhone is worthy of the monikerSteve’s Amazing New Device.He must be pretty pleased that his prediction thatApple would no longer be just a computer companybecame reality with the official change of the company’s name. Khoi Vinh, on the other hand, will be disappointed to hear thatthe iPhone does indeed use iTunes to do its syncing.

I spent part of the keynote chantingGet to the web browsing! Get to the web browsing!Then Steve Jobs got to the web browsing… with expando-Safari.

Dave Hyatt may be slightly biased but he thinks that this may spellthe beginning of the end for a separate mobile web.Dan Cederholm ispretty impressedtoo. Cameron Moll, on the other hand, believes that the iPhonewon’t revolutionise the mobile web landscapefor most people. Brian Fling disagrees. He thinksthe impact of the iPhonewill be huge.

Bursting Apple’s reality distortion field with Photoshop, Jon Hicks demonstratesthe problem with the iPhone’s screen.Roger Johansson also throws a cold bucket of reality on proceedings when he asks where the tactile feedback is supposed to come from whenthere’s no keyboard.That’s a valid concern according toDavid Pogue’s hands-on experience.

The biggest downer probably won’t be anything to do with the device itself butthe lock-in with some crappy provider,as Dave Shea explains. That’s still not enough to dissuade Jason Santa Maria fromwanting an Apple mobile device.

I met up with Brighton’s own mobile guru, Tom Hume, for lunch today. He’s taking a pragmatic and somewhat pessimistic approach with histhoughts on the iPhone.

Only time will tell how Apple’s baby will fare once its released into the wild. For some historical perspective, I invite you to cast your mind back toa Slashdot article from 2001 announcing the iPod.

New Year’s Resolution

Ina commentonRoger’s post about fixed and liquid layouts,Cameronwrote:

This issue seems to generate a heated debate every time it’s mentioned. I imagine one could pen an article with the headline “Fluid or fixed?” and nothing else, and yet dozens of comments would inevitably appear.

But rather than use that title, I couldn’t resist borrowing a pun fromAndy,prompted by a post from Scrivs calledWhat Resolution Will You Design for in 2007?(a classic example ofthe fallacy of many questions).

Now, firstly, we need to draw a distinction between monitor size and browser size. In other words, the difference between screen resolution and the viewport size:

There’s a real danger in thinking that “the numbers speak for themselves.” Numbers don’t speak for themselves;numbers need to be interpreted.

The numbers clearly show that monitor sizes and resolutions are getting bigger. The most common interpretation of that ismore and more people have bigger displays.But an equally valid interpretation of the numbers isthe range of displays is bigger than ever.It’s a subtle but important distinction. One interpretation focuses solely on thesizeof the highest numbers; the other interpretation focuses on therangeof all the numbers.

The way I see it, the range is growing at both ends of the spectrum. Yes, desktop monitors are getting wider (though that doesn’t mean that viewports get any wider above a certain size) but handheld and gaming devices are likely to remain at the lower end of the scale. The Wii, for example, hasa resolution of 640 x 480.

Mind you,the iPhoneturns the whole question on its head with its scalable browsing. At MacWorld, Steve Jobs demonstrated this by visitingthe New York Times,an unashamedly wide fixed-width website.On the Apple site,Wikipedia—a liquid layout— is shown fitting nicely on the display. The iPhone deals with both. Still, rather than letting my liquid layouts scale down to the iPhone’s width, I should probably start putting amin-widthvalue on thebodyelement.

Speaking of which…

A common argument against using liquid layouts is the issue of line lengths. On the face of it, this seems like a valid argument. Readability is supremely important and nobody likes over-long line lengths. But it’s not quite as simple as that when it comes to readability on screen compared to print, asRichard noted:

Surprisingly, I find short line lengths tiresome on screen; I don’t really subscribe to the empirical prescription of 7–10 words per line for comfortable reading. Most novels have 10–15 words per line and I think the upper region of that range is more appropriate for screen.

In any case, the idea that liquid layouts automatically means long line lengths on large screens is, I feel, a misconception. The problem is that a lot of the examples of liquid layouts aren’t very good and line lengthsdoexpand without limit. But it doesn’t have to be that way.

In my opinion, the most important addition to Internet Explorer 7 is themax-widthproperty. It means that we can now really start to look at creating fluid layouts within defined parameters, asdemonstrated by CameroninAndy’s book.In fact, I think we’re just scratching the surface of what’s possible in creating seamless adaptive layouts (and, more importantly, seamless adaptive page elements) using the dual power ofmax-widthandmin-width.

That still leaves Internet Explorer 6 and below. Should they get unbounded fluid layouts or should they get a fixed width fallback? The second is certainly an option usingconditional comments,which is the Microsoft-approved way of dealing with rendering inconsistencies. I think that the lack of support formax-widthcertainly falls into that category. Call ittranscending CSSif you will; I call it routing around damage on the designer’s network.

I want to hear what you have to say… if you’ve got something new to say. Let’s not just rehash the same old arguments that would inevitably appear had I simply asked “Fluid or fixed?”