Google I/O 2012 – The Next Evolution of Chrome Apps

>>Erik Kay: Hi, everyone. Welcome. I think we will have a few more people sprinkling in here, but my name is Erik Kay. >>Mihai Parparita: And hi, I’m Mihai Parparita, and together we work on the Chrome apps team. >>Erik Kay: So just a quick show of hands so I can get a feel for what the audience is here, how many people are as familiar with Chrome apps as they are today? All right, great. That makes things easier. So why don’t we dive in a little bit and just sort of do a little recap about how we got to where we’re at today? So I/O last year we talked about a few things. First off, as you know, we talked about Chrome getting to, you know, 160 million active users. We also launched for Chrome apps in-app payments through Google Wallet. We announced that the Web store was now localized in 42 languages. And we had the Angry Birds exclusive promotion that really showed how that it was really possible to make great quality apps just using HTML5.

And we’ve kept busy over the last year since then. The first thing, you know, to highlight, is Native Client. We launched that this fall. And Native Client really brought to the table a great set of games. You know, some showed that you could do high performance computing, that you could leverage native code in your browser. Things like Bastion, Mini-Ninjas, From Dust, great titles that show off the capabilities of the platform. You saw Gaikai in the keynote today that really was another great example. We redesigned the Web store. We have a completely new look and feel, and that really expanded the usage that we saw out from our users. Users were much more engaged. They come back more frequently. They install more applications. We’ve announced today that we’re up to 750 million app installs. Overall things are going great for the Web store. And of course, you know that Chrome is now at 310 million active users, almost doubling last year’s number.

So the reach potential for apps has been going through the roof. However, you know, that’s never enough. And we want — we’ve been worrying about the platform itself and how to keep pushing that forward. And so we’ve been thinking about, well, what do Web developers need? What are the capabilities that are missing from the platform and what can we do to move things forward? So we’ve been listening to your feedback and we’ve actually been bringing people on the team who have come from Web development backgrounds. People who have experience building some these Web apps, like Mihai, who is our tech lead for Google Reader, which many of you may know. >>Mihai Parparita: Yeah. So I was on Google Reader for quite a few years. As you may know, Google Reader is an advanced HDIC application that has pushed the edge of what browsers can do. On the other hand, we feel like the platform could do more, could provide more flexibility, more power for Reader and similar apps.

Just to give you a few examples, Reader is one of the first apps to be offline enabled, and that was a technical challenge and we fought with manifests and caches like everyone else, but even with that it wasn’t the best user experience. Users had to enable offline mode. Users had to launch a browser to use the app which sort of feels like a tautology. Why would I use a browser when I’m offline? Another issue that we faced in Reader was dealing with third-party content where feeds can have Flash, they can have iFrames, they can have arbitrary JavaScript, and we want to have the experience to be as rich as possible, but we don’t want a random feed to be able to steal your Google cookies or otherwise take advantage of the fact that it’s being displayed in a privileged environment.

We have to jump through a lot of hoops to sanitize the content, have whitelists, blacklists, and we always lived in fear of well, what if we missed something? What if tomorrow anyone can access your Reader account and your account and all that? And finally, users wanted a lot more flexibility about how they consume feeds. Do they want to open up the item in a new tab, in a new window, in a pop-up, they want a mini browser inside of Reader? We couldn’t provide any of that. We could provide a or target equals blank, and sort of hope for the best, hope that the pop-up blocker didn’t get us, but that was about it. So Reader, you know, is in some ways a well-suited app for the Web environment and I’m sure others of you have run into even more issues.

>>Erik Kay: Right. So we took feedback like that and we really started to push forward in terms of what we wanted to do. We’re evolving in four main areas. The main one that we’ll start with is we’re pulling apps outside of the browser. We’ll dive into each of these in more detail, but just the quick overview is pulling them out of the browser. We’re enhancing the user interface, giving you more control over the windows and we’re making apps be offline by default, making it so it’s easy to build them in a way that they behave reasonably with poor network connectivity. And finally, we’re adding a wealth of new APIs to give you access to powerful system and device capabilities.

So let’s dive in a little more deeply. So what do we mean by breaking out of the browser? Well, the first thing we mean is that you launch apps from outside the browser. So you’ll see in a minute here that basically you’ll no longer need to launch the browser, go into Chrome, go to the new tab page to find your apps. The apps will instead come from the operating system, the traditional place that you’re used to finding them and launching them. Second, the apps will act as peers to their other native applications, top level windows. They interact alt tab to switch between applications.

Everything you would expect a native app to behave like in a user interface perspective, OS integration perspective. It will no longer be a tab in a browser. So let’s give them a demo and show them what this looks like, Mihai. >>Mihai Parparita: Sure. So this presentation is actually in itself an app. The demo just breaks out of the app. And if I move the window out of the way you can see that I have my top level window.

It has, you know, minimal — no tab strip, no Omnibox, and there’s a Chrome icon at the bottom that seems to be getting clipped. And where I can make tabs and the application itself has its own icon in the launcher. This is the experience on Chrome OS where it is a full application launcher. On Windows there will be shortcuts on a desktop and the appropriate behavior on other platforms. So let’s switch back to the app and let’s talk about UI. >>Erik Kay: Right. So enhanced user interface. So you already saw that it has a native level — top level window frame. There’s no browser Chrome, there’s no tabs, no address bar, no back button. And further, you have full control over the look and feel of that frame. Next you have — you can actually manage multi-window applications.

Actually, I did this the wrong order. But multi-window applications. So that means you really will be able to cooperate between multiple windows and have them interoperate. Today in tabs it’s very hard to sort of have multiple tabs that are not cooperating with each other and interacting with each other and it’s hard to do things like tool bars and, you know, panels that cooperate and interact with your host window.

And you know, one of the things by not having browser Chrome is, well, what do links do? Well, links behave just like they would in a normal native app. When you click on the link it opens the browser window. Let’s show them. There’s a link to >>Mihai Parparita: I’ll tab back to the application. >>Erik Kay: Right. So now you’re back inside the application. So we’ve got another demo here that shows a little bit more about what windowing control can do. >>Mihai Parparita: This is a little toy demo showing that we have an app window. You can see the minimal Chrome. Bizarro World one is actually a full custom window where the app controls every single on it. In this case I just made it all black.

We have full control over behavior so I know I can have it resize itself and certainly drag around and follows. You imagine this could be a little dog toolbar and I can control it or have the window in a minimized state. >>Erik Kay: Right. So offline by default. Before we dive into the specifics of this, it’s probably worth explaining what do we mean by offline. So A lot of people think offline and they think, okay, I’m on an airplane, I have no connectivity whatsoever.

That’s not the only time that offline matters for applications. Offline can mean I have got slow connectivity. I’ve got spotty connectivity. For anyone who uses a cellular network, maybe in this building, the Wi-Fi, you know, you’re used to not have reliable connectivity. The speed might change as you go. Maybe you’re in a hotel with a captive proxy. If your application is trying to talk over the network, it needs to be able to be robust in the face of these different network connections. So how do we do this? Well, we’re starting from a base of Chrome packaged apps. Basically for those of you who aren’t familiar with Chrome apps in general, packaged apps are a zip file, a collection of HTML, JavaScript and other resources for your Web application that are downloaded, installed on the user’s computer. So all the code and logic is always there ready to go. We take some extra steps and — to make sure that we’re enforcing a strict separation between data and user interface. By being packaged, by making sure that all the code and logic are always being — are always running from client side and not being generated server side, we mean that the app doesn’t get into a broken state and it’s up to you to download your data, store it locally, render your UI appropriately. We further enforce this with — support this with building APIs that are themselves resilient to being in these poor network connectivity conditions. And finally, this may seem like a small thing, but the fact that we’ve moved apps from launching outside the browser actually addresses one of the key usability issues with offline. Today some of you may have actually gone through the effort to make your hosted Web app work offline and found that even after you went through all the effort to work with AppCache and make that do what you want it had to do that users are still not finding that functionality and they are still frustrated with using it.

For the reasons that Mihai was pointing out before, they had to know first to launch the browser, then they had to somehow navigate to your app. Well, if they know how to bookmark it, that’s about 10% of users, then they can get back to it. What about for the rest of the users? Well, for them they have to either know what the URL is ahead of time or for many of them actually what they do is they go to their favorite search engine and they try to navigate to it.

And guess what? When you’re offline search engines don’t really work. So people in a poor connected world are continuously reinforced that the browser doesn’t work because all their tabs are broken. Even when they have an app that they know works, they don’t know how to get to it. So pulling app outside the browser and making it a first class peer to their other native applications gets past all that anxiety and all the logistics of getting there. >>Mihai Parparita: So we’re gonna demo this very quickly and we’ll get the presentation out of the way. I’m going to yank the ethernet cable out of my Chromebook. As you can see there’s a little icon in the corner that says I’m offline. I can refresh Google and show that it doesn’t seem to work. And we’re going to install an app.

In this case we’re going to sort of do a kind of stress test where we are offline. We’re going to install the app right now in developer mode, but it should process the same. And this app has never had a chance to run before. It’s never had a chance to talk to its server, download data, but the experience should still be acceptable. The specific app we’re doing is a diff tool that lets you compare files. So when it launched it has an offline indicator that says you’re currently offline, so all it can do is it can diff local files, but dash will keep working. So let me pick a couple of files. And as you can see — I lost focus. As you can see, the application still works. If I manage to find the ethernet cable and plug it back in — >>Erik Kay: While Mihai is doing that, I’ll point out that we’re doing this demo on Chromebook and we’re doing it with the latest Canary build of Chrome OS. So we’re taunting the demo gods — >>Mihai Parparita: Fine in this case.

We are back on line. The app has noticed that we are online. The choose URL feature has appeared, so now we can diff not just local files, but ones that live on a server. So I think I have some cuts. And as you can see — >>Erik Kay: This is really a tortured case for an application. It launched for the first time never having seen the network. So it was able to run robustly. Without quitting and relaunching it, it recovered. So it was able to see, you know, the fact that the network was back. You didn’t have to do anything, you didn’t have to refresh the page, you didn’t have to take that extra step.

The app was just able to proceed. >>Mihai Parparita: Now that the projector has been fixed you guys can see that the app launcher at the bottom is what I was referring where all these apps can be launched offline. >>Erik Kay: Right. All right. So new APIs. Now, this is one of the more — the area that I think most of you are going to dive into first when you start playing with Chrome packaged apps. And it’s all the new APIs. We think of them in three groups. The first area are what we call system APIs. The system APIs interact with the operating system at a low level or with hardware on your device, so maybe USB or bluetooth, or as you saw in this morning’s keynote, raw UDP or TCP. These will enable a wide variety of apps that you’ve never been able to build with Web technology before. Next we’ve got shared data APIs, and shared data APIs are — provide ways of interoperating with other applications on your system.

Things like photos or contacts, calendar entries, things that you want to be able to access safely from multiple applications. And finally then a group of service APIs. The service APIs are your — the Web services that you all know and love, but built into Chrome, into the Chrome runtime in such a way that it’s going to be resilient to poor network conditions. So things like analytics, in-app payments, things like that, will be coming to the APIs. Why don’t we give a demo of one of these APIs in action? >>Mihai Parparita: Sure. So this demo is going to display me because I’m going to hold a physical device here. This is a little science fair kind of project that one of our engineers Mike made. It has an arduino board on the back. It has — it talks to the Chromebook via a serial port, a serial-over-bluetooth a serial-over-USB in this case, and it has a little stepper motor here that is controlled by the arduino.

And the slider at the bottom of the screen controls the motor. It just sends a one-byte command saying “please rotate to my current position,” and that goes over the serial wire, the arduino interprets it. The key thing here is that the Chromebook knows nothing about this particular device or about arduino in general, it just knows that there’s a serial port and that in turn is exposed to the application. So all of the logic about the specific device is in the application that can be installed by any user. >>Erik Kay: The other key bit is that all this code is just JavaScript and HTML, right? It is — no special, no device drivers, no, like, native code that needed to be there. That said, we are going to be providing bindings in addition to JavaScript, also to Dart and to Native Client, so you will be able to access these APIs from whatever programming environment that you’d like. Speaking of the programming model, why don’t we dive in and take a look at some code. Before we do that, though, just a refresher of what is behind the core bit of the programming model.

So as I said, these are packaged apps, and that means they’re a directory full of resources. They have a background page that acts as a hub, so what that means is this is what starts up. When you launch the app, a background page launches and it triggers an event, and that’s what — then it’s up to you to decide what to do.

In previous incarnations of Chrome apps we had a launch URL, basically a page that would open up in a tab. Now you’ve got full control over how your app behaves, how many windows it opens up, if it even opens up a window at all, or does something in the background. And that last step is sort of an important use case that we’re really interested in is being able to handle system level events and have your app come and go based on signals from the runtime. So let’s take a look at some of the code that shows this in action. >>Mihai Parparita: Sure.

So I’m going to call up a Hello World demo. And incidentally, to actually code the app I’m going to use an app. This is a text editor that’s been whipped up by one of our other engineers. So like Erik said, first we start with the manifest. So if you went to the extension talk yesterday, you know there’s a new manifest called Manifest Version 2. >>Erik Kay: Right. And Manifest 2 basically is the trigger that we use to enforce some of the new policies with apps and extensions, and fork turns on CSP by default which we’ll talk about in a little bit, but it also is one of the keys for showing off the new app behavior. It also has a — the basics that you’re used to, a name, a version, and another field that is — you’ve seen before, the app field, but rather than a launch URL like we said before, instead it has a designation of the background page, as I said, the main, the hub of the application, it uses the same event page technology that we’ve just announced with Chrome extensions which means that it basically can come and go on demand, as the system needs it.

So we specified the app. The other thing actually, Mihai, you might want to add permissions to it. So permissions are the way of specifying in the manifest what capabilities your app needs up front. This allows us to, you know, control, you know, it’s a security feature that basically allows us to limit the damage that an application can do if it has a bug in it. So then next, we’re diving into the actual background page itself. This is main. As I mentioned before, what it does is it loads, and then it fires off a launch event.

So the first thing Mihai does in his background page is set up an event listener in the background page, and in that, first thing he does is to create a window, using our new window app API. And basically this is the Hello World, launch the app, get a launch event, create a window, show just an obvious snippet of HTML, so here is our Hello World. The next step is to load into Chrome, now again, since this is just a packaged app directory, it’s built in the same technology as Chrome extensions and other Chrome Apps have been built in the past, so just going to developer mode, load and unpack extension, and then we can just launch the app from the app launcher directly, there you go, and we have our Hello World, so really trivial bit of code, as you can imagine you can do whatever you want from that launch event and do more advanced things, so before we do the more advanced things, let’s talk a little bit more about some of the other features of the programming model.

So one of the things that’s interesting is that all of the windows are what we call single page applications. There’s no navigation. So we think this leads to an overall better experience and simplifies a few things, again, the fact that we don’t have a back button, that we don’t have an address bar, we don’t want to have people building apps that wind up getting users trapped and stuck, we don’t — we want to get rid of that flash of white repaint. And we’ve also actually removed a few Web features, things that are — that we think lead overall to bad Web performance or have security problems with them. It turns out that on the Web at large, it’s very hard to deprecate features. Very hard to (inaudible) We have strict compatibility needs. As people who are going to be building these Chrome Apps are going to be starting from scratch, we have the opportunity to take a few and just eliminate them from the beginning, and they will be things like synchronous requests that will block the UI thread or again things that might call security problems.

Speaking of security, let’s talk a little bit about the security model. So to start with, you get all the things that you’re used to with Chrome, you get process isolation, Chrome is famous for being a multi-process browser, and apps are no different. Each app runs in its own process, completely separate from each other. We also Sandbox each app, so each app has its own storage area, its own Sandbox execution runtime Sandbox which makes it so that even if it had a bug, the limit, the damage to what it could do is tightly controlled by the runtime. And part of that is the permissions model I was talking about before. The app has to declare up front what capabilities it wants to use, so in this case we have our little app that used the serial port API and the Web cam, so if that happened to have a bug in it, that could be exploited by an attacker somehow, the worst that could happen is it could do exactly what that app could do and talk to the serial part, the Web cam.

This is very different from what you’re used to with native executables, where once you have a security bug in the native executable, you have access to the whole machine, everything the user could do, attacker could do. So I mentioned content security policy. Now, what content security policy is is a way to protect your app against cross-side scripting bugs. For those of you who aren’t familiar with cross-side scripting, it’s basically a way that an attacker can somehow get code injected into your page and run in the context of your page.

Now, in a Web page, this is bad enough, because it means like — in something like Gmail, it could — you know, an attacker could maybe read all of your e-mail and post it to another site. In the context of an app it could do even more potentially with the new more powerful APIs we’re adding, like, you know, I don’t know access your Web cam, leave persistent files somewhere on your hard drive, talk to some hardware you didn’t want them to talk to. So we really try to lock this down content security policy makes it so your apps will be resistant to these sorts of attacks. So let’s show that in action. >>Mihai Parparita: Sure, so I’m going to take my Hello World demo from earlier and make it actually do something besides play Hello World. I’m going to have it — have a little form that asks users for their name and then displays it in the output, so that is the markup side, and then on the scripting side, I have sort of the obvious implementation I just described where I — where I get the submit event and display the name and the output.

>>Erik Kay: For those of you who are, you know, the more observant in the crowd, you’ll know that inner HTML is a pretty dangerous feature of the web to use. Basically is taking — he’s taking unescaped content and just injecting it as code straight into his page. >>Mihai Parparita: Let’s load that same page because it doesn’t use any app APIs just in a regular Chrome tab the way a Web app would run in. So, sure, I can type in my name, but because of inner HTML, as Erik mentioned, markup gets interpreted, H1s aren’t so bad, well, what if I had an unclick handler here that — >>Erik Kay: So you can see that this is script. He’s actually typing in JavaScript, now it’s going to get injected into his page. Now, obviously this is a very simplified form of an access because there’s a form field, user had to type it in. Usually they might come in as a parameter in URL somehow got data stored into your system, but you can see that if they could run code in the context of your app, they would be able to do the dangerous things your app could do.

Let’s try that same code in an app. >>Mihai Parparita: So app still works. HTML still gets interpreted because, you know, Chrome does what it’s asked and gives us inner HTML, but if I type in that same script snippet, right, and try to run it, nothing happens. If I pull up the developer tools, these are the same developer tools that you’re used to, you can see a content script clause here that appears. >>Erik Kay: Right. This is basically showing you that we are locking down the app, so if you have a bug that might enable somebody to inject inline script into your page, it can’t — it can’t exploit anything in your app, it’s just stopped right up front. As Mihai pointed out, this is all tightly integrated again with all the developer tools that you’re used to, you know, so you can leverage all the power that’s there. They’ve been evolving really rapidly, and so there’s a lot of great features if you haven’t been checking them out.

So let’s talk about a few other bits of a security model. We take a further step and isolate storage from one app to the other. So what this means is that what gets written in one app, what happens in one app, stays in that app, it can’t affect what happens in the other. Cookies in your browser don’t affect cookies in your app and vice versa. And this is true not only for the origin, but if your app is doing — interacting with Web content from other origins, that too is isolated, so if you’re logged in to Twitter, you know, in your browser with one account, and your app interacts with Twitter, it isn’t affected by that.

It would have to log in separately. And I mentioned explicit shared data APIs before when I was talking about new APIs. This is also a security feature, because it means that we can give apps access to more powerful functionality without exposing your whole hard drive to them, and without giving them the ability to write arbitrary files into arbitrary locations, so they can’t leave an executable on your hard drive and they can’t, you know — they can’t just read your Quicken file, for example, but they can read your photos perhaps, if they’ve got the photos permission. And one further distinction that we’ve made for apps is that we no longer have access to extension APIs within packaged apps. So we made an explicit split to say that an app is outside the browser, and that means that interacting with the browser through these extension APIs no longer makes sense. It’s also true vice versa, extension APIs can’t affect apps.

These apps are truly isolated from the browser and operate on their own. There’s one last security feature that we added here, which actually has some very useful functionality as well, so you’ve heard us just talk about how we lock down this, lock down that, you can’t get this web content or that content. Well, sometimes it’s actually useful to get a host of Web content in your app. Well, in a similar way that you’re used to with native desktop and mobile applications, you can embed essentially a view, a full browser inside of your app using something we call the browser tag, so this is different from an iFrame in a couple of different ways. The first is that it’s a top level frame. It doesn’t think of itself as an iFrame, it thinks of itself as if it were in a tab or in a window by itself.

Second is that it runs out of process. So if there’s a bug, if there’s some issue where by going to remote website, somebody found a security hole in Chrome, it would only affect the browser tag instance. It wouldn’t be able to affect your app. And finally, once again storage is isolated, so again, what happens in this browser tag doesn’t affect what’s in your app and vice versa. So we’ve got a little more time, so why don’t we dive back into the — that serial port demo and show what it’s like to actually use some of the more powerful APIs and do a little bit more stuff with it. >>Mihai Parparita: Okay. So I’m going to close some more windows and load up a standalone version of that serial port demo that we had earlier, and let’s launch it.

So same sort of setup. Camera using the Web RTC API to show me, the same slider. This one is a bit fancier. It has a serial port ticker listing all serial ports that are local. So the goal is we have a Chrome logo, physical Chrome logo here, and we have the on-screen logo here, this board actually also has a potentiometer, this little knob I want to turn, and I want to make it so that turning this knob turns the logo on the screen, so let’s pull up the code for this. So it’s in — >>Erik Kay: Right, so the existing code was basically writing to the board, sending it data. Now this one, we’re going to change it to add read function, read back. >>Mihai Parparita: So walking through the existing code briefly, we have the Web RTC side of things where we get access to the user’s camera, and display the result. Since the app has the video permission, this did not trigger info bar, the access happened immediately. We have the slider, the HTML5 range element here, we set up an on-change listener for it to set the position, and we built a serial port picker by listing all of the ports that are available.

Once we pick a port, we have the open API call to actually start opening up a connection to it. And once we open, we can start setting the position. We have the connection ID that we get from the API callback. This is — all these functions are async and we can start setting the position. To actually set the position, we just send a single byte as — actually as an ASCII character from 0 to 9. We’re using the buffers so that the serial port API can work with any kind of binary data, too. We’re using ASCII here for convenience. So as Erik mentioned, the next up is to actually start reading back from the board. So given that there’s a write API call here, because you can imagine there’s also a read API call we can use — >>Erik Kay: Right. As Mihai mentioned all of the Chrome App APIs are asynchronous, so you’ll notice they all take a callback, and so he has in his function there, and on read function for the callback which he’s pasted in to save some typing time.

>>Mihai Parparita: Yep. So using array buffers we read back the data from the serial port, we do the opposite conversion from ASCII to a degree value, and then we use CSS 3 transforms to set the position of the logo and finally we want to read the next byte, so this isn’t a blocking call, it’s just whenever the next byte happens to appear, the call back will be invoked again. Let’s save this. We’re actually going to save it in the same place. >>Erik Kay: And that — by the way, that programming model did show some of the flexibility and power of having an asynchronous API, but we’re able to sort of cue these things up and have multiple async things going on at once, and the callbacks happen, you know, when they happen. We don’t have to worry about inner leaving them. >>Mihai Parparita: The App, it’s the same sort of edit reload cycle that you might be used to from web development, and so let’s make sure it still works, that we can still control it, so that part still works, and now if we have the knob up here, and we turn it, the Chrome logo appears to spin, so we’re reading data.

>>Erik Kay: Obviously, this is a really simple demo and not all of you are going to be sitting in your garage hacking on arduino boards, but we’re hoping that this sort of conveys the kind of thing you can do. Web technology, and interact with hardware and system level data. >>Mihai Parparita: For example, people on our team have built our C clients, you know, telling their clients to use the raw socket API, someone else built an interface to a heart rate monitor that uses Bluetooth, and all of this networks on a Chrome build, for example, that couldn’t talk to these devices before.

>>Erik Kay: So let’s sort of wrap up about and recap a little bit of what we’ve gone over. So we’re evolving apps to break out of the browser, to have enhanced user interface, more control over the windowing. To be offline by default, to have access to new, more powerful APIs, now, we’re also — this is also available for you to play with right now, basically as we said, this is canary build of Chrome. For those of you who don’t know, the Canary is our nightly update schedule.

It’s available for windows and Mac. If you’re a Linux or Chrome OS user, if you’re brave enough, you can do your own build, or you can wait for your next dev channel to come out and you can start playing with it then. We’ve also been asked in the past about why are these apps, you know, specific to Chrome? Why are they tied to Chrome? And part of the reason is, because, well, there hasn’t been a lot of appetite in the browser community for doing things, doing more dangerous capabilities, but in the last six months or so, we’ve been, you know, talking to more and more folks from the community and discovered that momentum had been building, that people were all starting to think along these same lines; that building — using Web technologies to build more powerful applications, its time has come, and so we’re working with the W3C and with other browser vendors including Mozilla, and Adobe, and Intel and Samsung, people in the Web community to form the new working group to help standardize this going forward, so we’re excited to help take the technology that we’ve been building for Chrome and try to contribute it and make it a standard across multiple operating systems and platforms.

And finally you heard Sundar mention this in the keynote, but our intent is to make these apps work on mobile as well. Everywhere the Chrome runtime exists, we think that we can build — bring apps — bring this technology along with it. Now, we only showed you this very simple serial demo, and — but there’s a lot more that you can check out. Some of it right here at I/O. If you go over to the Chrome Sandbox area for example, you can see Sencha has a neat media player demo that actually is implementing UP&P to stream from a local media server, which is kind of cool. Kendo has a photo booth app that uses a lot of the Web RTC stuff. There’s a text editor that was done by the annular JST at Google, and there’s a demo that we’ll call “Johnny” that is a — >>Mihai Parparita: Johnny HTML5. >>Erik Kay: Oh, Johnny HTML5.

Sorry. Johnny HTML5 which is a Lego mindstorms robot which is being controlled over Bluetooth from your desktop computer. Now, you can find all of these samples and a bunch more, actually, in our new GitHub repo at, Google Chrome, there’s an apps demo section there, so I encourage you to go fork and play and contribute. And — and finally, the way to learn more, you can read our docs up at Now just a quick warning. Since all of this new on the Canary channel by default, our apps documentation points you at our current stable channel stuff. But there will be a helpful link there that shows you how to get to the latest stuff.

You can also join our mailing list,, and if you want to talk to people in person, people on the engineering team, we also hang out on RC at Chromium apps@freenotes. So feel free to join in and ask away, we are looking forward to hearing from you. We’re really excited to see the kind of things that you guys can build with what we’re doing, but it is early on. Expect some bugs. Expect some growing pains. We would love feedback. So now we will take your questions. >>Erik Kay: So if you have any questions, please come to the mic. By the way, when if — if we end this and you still have questions that you would rather not ask at the mic, we will stick around afterwards for a few minutes, but also you can find us over at the Chrome booth. We will be having office hours, so feel free to, you know, not only check out the demos, but come by and ask us questions.

All right? >>> Can — can an app launch another app? >>Erik Kay: Not right now. That’s a good question. >>Mihai Parparita: Can you describe more specifically what you would like to do here? Is it a matter of like I would like to send an email so I could launch the Gmail app sort of thing? >>> Well, sort of like the launcher, but without having to bother the user with them going to find the app, saying this is the app that you are looking for to complete this request. >>Erik Kay: So you can do some of it with Web intense but not exactly the way that you’re talking about for making a full launch replacement. But that’s the kind of thing actually we would probably more likely expose through extensions in some way, but remember that each operating system is going to have it’s own sort of natural way of launching apps. So even if we did provide it, that might be specific to Chrome OS or something like that. >>Mihai Parparita: Go ahead.

>>> Yeah. You mentioned early on in the presentation about offline usage and — and the APIs into the system that would degrade gracefully. Offline. Can you give some examples of that? >>Erik Kay: Sure. So take, for example, something like that Google analytics right now. Everyone is used to using Google analytics in their app, they want to see how their app is being used. The way to traditionally include it would be to say script source equals whatever your path to your analytics JavaScript file. But what if you are offline? So first off, if you are offline when you launch the app, it might just fail to load the analytics code altogether or it might load part of it. Then let’s say that you load the code successfully but then go offline. Well, the traditional way that it does its thing is to ping a server to send its data. So in that case, we’re modifying — both the API to be loaded, you know, safely, always on the client side and to know when it’s offline and to store and cache some of its statistics locally so it that it can save them up and report them up at a later time.

>>> Okay. Thanks. >>Erik Kay: Okay. >>> So you said there’s going to be a split between the app extensions — I mean the app API and the extension API. Can you tell us a bit about more about these, which go to which category? >>Erik Kay: Sure. In general, extension APIs are going to be limited to the things that affect the browser. That affect Web pages, tabs, that sort of thing. And apps is going to be, you know, everything else. Now there’s going to be a few that are going to exist in both. In general, what we have done is for now limited the ones that are more dangerous system-level things and those are not going to be available in extensions to begin with. Something we’re evaluating, we might change our minds on some of that over time. But to start with, we’re going to try to keep a clean break between those sorts of things and apps and the things for extensions, keep those tied to things that affect the browser. >>Mihai Parparita: Some APIs might behave slightly differently, so the context of your API, for example, can affect any Web page.

But when used within your app it would affect only your own apps pages. The docs I think should cover — will start covering this. >>> You mentioned the — the applicant launcher can use a browser tag to launch another website. So can any JavaScript in that website interact with the app? >>Erik Kay: Sure. The way it will interact — you can interact with the browser tag, but it will be through a very specific post message type of interface. So it will be possible to have some hosted Web content that you want that can message out and for you as the better of the browser tag to inject bindings into the page as well.

So you will have full control over that page. We actually expect eventually the API to be rich enough that people will be able to build a browser using a browser tag. So maybe you don’t like Chrome’s UI and you want to build a browser with tabs on the side or something, I don’t know, we think that eventually you will be able to do that with the browser tag. >>> What is the level of integration with the file system? So I saw in the text editor you click, open a file, browser, you pick a file. But say you wanted to build an editor where you have, you know, a directory tree on the side like a normal ID, click through it. >>Erik Kay: Sure. >>> Is there some — I guess what kind of APIs are available for accessing the file system generally? >>Erik Kay: Right. Actually, good point.

We didn’t mention that. Probably most of you noticed that the text editor that we were using is itself an app and was interacting with the file system and that text editor was using — was, you know, all JavaScript and HTML as well and it was using some of the file system APIs that we’re working on. In general, we’re — we’re trying to limit it, we can think of file system access in kind of two ways. One way are ways where the user specifically indicates a specific file that they want to give you as the app access to, through a standard open save dialogue type of thing. For that, you can access files anywhere on the system, read, write access to them with the right sort of permission and that sort of a thing.

But in terms of arbitrary non-user initiated things, where things you might say oh, because of file A I want to access file B, C, and D, we don’t allow you to do that. So in those type of cases, take an IDE, as an example,we would say that you should bill something that interacts with files in your own Sandbox in that world. So in that case maybe you might use native client to build the git implementation that actually, you know, can directly, you know, pull down code from your repo right into your Sandbox and edit things locally there. In your Sandbox you have got full access to all of the directories, all of the files, unrestricted. It’s the rest of the hard drive that’s off limits, unless the user does a specific gesture to give — grant it — >>Mihai Parparita: The third way for getting access to other files on the hard drive is by the shared data API. So if your app is interested in not any arbitrary file but photos, then it can, yes, ask for any photos without a user gesture.

>>> So does that mean — so on Windows or Mac system, there’s a system photo directories, where supposedly the user puts their photos, you can ask for permission to access the files in that directory; is that the idea? >>Mihai Parparita: Yeah. >>Erik Kay: It’s not exactly that. It’s a little more. So we will actually try to discover the types of places that you store your photos and so a lot of people don’t — don’t keep their photos leveraged to that one thing; they have an extra hard drive, whatever. The ability for users to configure what defines their photos area. But even then it’s not just directory access, limited to the types of files that you can access to. There have to be known types that we expect to be photos in that case. >>> Okay. >>> Are there any particular patterns or practices that we should employ when designing apps in terms of UI state so when a user exits a UI, when we want that UI to come back exact the same as when the user exited.

>>Erik Kay: Yeah. That’s a good point. So there’s a lot of things that we think are going to change in terms of how you build these apps that are different from what you might expect with the Web page. Because you might be dealing with multiple Windows and because your app can get shut down and pulled back at any time, we definitely want you to be resilient to that and to be able to restore and pick up from exactly where you left off in those cases. You know, the example is remember — think about this, these apps also need to be able to run in mobile environments where mobile apps don’t have guarantees of a lifetime. So that means that you have to write your apps in a way where they might be shut down due to system resource issues, right? But the user then switches back to them and it should look like nothing happened and it should look like you never exited. >>Mihai Parparita: Sorry.

The other part there is that we have storage API. So for storage state, you can use the storage API which looks like local storage but is asynchronous and more importantly is synced so that can have the same state across all chrome devices that the users assigned them to. >>Erik Kay: Yeah, absolutely. So we actually think it’s a really interesting use case, that you have your app in one state, you know, on your machine at work, you go home and you launch it again from exactly where you left off at work. >>> Any ability for external devices to wake up apps? >>Erik Kay: Well, basically the whole gist of these — of this notification based model that we’ve got for — for the app, it should enable that sort of a thing.

We don’t have any specific APIs to talk about on that front yet. But — but absolutely, things like — you can imagine things like server push, synchronization, that sort of thing will all be — >>Mihai Parparita: I think the one thing we have is for the media gallery API, if the SD card is inserted, it will fire an event, for example, so your app can decide to do something in response to photos that just appeared even if it’s not running. >>> With the serial device that you were — that you were working there, can — when can we expect to — like support for USB devices in Chrome? And will it cover the whole range? Can I like have a device that emulates an RS 232 system, could I access these kind of things with Chrome, through Chrome? >>Erik Kay: So this serial device here was actually a over USB, so it’s serial over USB, and actually we do have a USB API as well. >>Mihai Parparita: Low level USB API. >>Erik Kay: Right, exactly. So basically if you know everything about a device, one of the demos that we have if you look in our repro — >>Mihai Parparita: It’s a Griffin PowerMate.

Sends its rotation position over the USB, and we can read a byte off of it. >>Erik Kay: So there’s a little bit more work we need to do with the USB to really flesh it out. For example, we don’t have support for HID devices yet, you know, and I think it would be nice for us to build in a little more knowledge about some of the specific devices.

Right now it’s low level, you have to know a bit more about the device that you are talking to. >>Mihai Parparita: For example, one of our developer relations people has got his connect hooked up to an app, but that seems to work. >>> Time schedule, approximately? Months, years? >>Erik Kay: Well, as we said, you can play with this today. And you know that in general our approach is to take a very incremental release early and often type approach.

So our intent is to get it out this year, you know, into the stable channel. But you should be able to start playing with it now and building apps now. Just depends on your tolerance for early code. >>> Right, thank you very much. >>Erik Kay: All right. Well, we might be out of questions. Any others? All right. Well, I think that’s about it then. If anybody wants to find us afterwards, like I said, we’ll be over at the Chrome booth answering questions and come check out Johnny HTML and the other fun demos over there and we hope to hear more from you.

Thanks. .

Erik Kay, Mihai Parparita

For all I/O 2012 sessions, go to