Missing a rationale for WebExtensions

Mozilla’s announcement to deprecate XUL/XPCOM-based add-ons raises many questions. Seeing the reactions, it seems that most people are very confused now. I mean, I see where this is coming from. XUL and XPCOM have become a burden, they come at a huge memory/performance/maintenance cost, impose significant limitations on browser development and create the danger that a badly written extension breaks everything. Whatever comes to replace them certainly won’t give add-on developers the same flexibility however, especially when it comes to extending the user interface. This is sad but I guess that it has to be done.

What confuses me immensely however is WebExtensions which are touted as the big new thing. My experience with Chrome APIs isn’t all too positive, the possibilities here are very limited and there is a fair number of inconsistencies. The documentation isn’t great either, there are often undocumented details that you only hit when trying things out. This isn’t very surprising of course: the API has grown along with Chrome itself, and many of the newer concepts simply didn’t exist back when the first interfaces were defined. Even worse: Opera, despite using the same engine, occasionally implements the same APIs differently.

So my main question is: does Mozilla really plan to keep bug-for-bug compatibility with Chrome APIs all for the goal of cross-browser extensions? That’s an endless catch-up game that benefits Chrome way more than it helps Firefox. And what is this cross-browser story still worth once Mozilla starts adding their own interfaces which are incompatible to Chrome? Don’t forget that Chrome can add new APIs as well, maybe even for the same purpose as Mozilla but with a different interface.

Further, I don’t see any advantages of WebExtensions over the Add-on SDK. I wasn’t a fan of the SDK back when it was introduced but I must say that it really matured over the years. It took much time for the SDK to become a modern, consistent and flexible framework that it is right now. Mozilla invested significant effort into it and it paid off. What’s even more important, there is now sufficient documentation and examples for it on the web. Why throw all this away for a completely different framework? Note that the announcement says that most SDK-based extension will continue to work in the new world but according to the comments below it won’t be a development focus any more — from my experience, that’s an euphemism for “deprecated.”

As mentioned above, I don’t see how theoretical cross-browser compatibility is going to benefit Firefox. Maybe the advantage lies in the permission model? But Chrome’s permission model is broken — most extensions need APIs that could potentially allow them to access user’s browsing history. While most extensions won’t actually do anything like that, privacy violations are a very common issue with Chrome extensions. The privilege prompt doesn’t help users recognize whether there is a problem, it merely trains users to ignore privacy-related warnings. Oh, and it shifts the blame from Google to the user — the user has been warned, right?

In my opinion, this permission model cannot be seen as a substitute for reviews. Nor will it make reviews easier than for SDK-based extensions: it’s pretty easy to extract the list of SDK modules uses by an extension. That gives one pretty much the same information as a list of permissions, albeit without requiring an explicit list.

There must be a reason why Mozilla chose to develop WebExtension rather than focus that energy on improving the Add-on SDK. Sadly, I haven’t seen it mentioned anywhere so far.

Comments

  • jr

    Apparently the previous plan (i.e. 3 months ago) was to create Firefox extensions from Chrome extensions: https://github.com/jetpack-labs/chrome-tailor

    (BTW Robert O’Callahan from Mozilla says WebExtensions “is what Jetpack should have been”, without saying what the differences are)

  • J. McNair

    My best guess for why the Mozilla leadership and some engineering talent are pushing “Webextensions” is a desire to tap the existing Chrome extension market, because that is what Microsoft is doing with Edge. It appears from the outside that there is a real fear that Gecko products will lose future, innovative Add-Ons because Gecko lacks a “somewhat-like-Chrome” extension system. What really frustrates me is their appeal to standardization. There is no evidence that Google, Opera, or Microsoft intend to freeze their APIs and submit them to W3C or WHATWG, and it seems naive to assume that they will because Mozilla asked nicely and wrote good code. A plain admission: “we want to attract Chrome developers to Gecko,” seems much less naive and far more forward thinking. Yes, it’s an admission of defeat, but it’s far less confusing.

    I want to think that Mozilla can invent something for Gecko that is BETTER than Chrome’s APIs, reusing some of the work that went into Jetpack, Firefox OS and Apps. Call it something catchy — “Rocket Boots Ignite LightBeam Spark Mandrill” or something and then release some community-owned Blink compatibility JS shims on github. Unfortunately, that’s much, much harder. A man can dream, though.

    Ah well.

  • Hugh Isaacs II

    I see WebExtensions as a move towards portability (since Mozilla wants to get Servo out there) and towards possibly standardizing browser extensions.

  • Ben Basson

    I also don’t see the rationale for this.

    1. Chrome add-ons really can’t do that much compared to Firefox add-ons.
    2. I don’t see how there can possibly be more Chrome add-ons (especially of any value), so why bother tapping into that market?
    3. Mozilla need to focus on not losing the one key benefit of Firefox, which is its current add-on ecosystem, and being able to port worse extensions doesn’t help.
    4. If people were really that bothered about writing cross-browser add-ons, they’ve probably done it already. It’s a bit late to try and standardise things now (especially if everyone’s going to have their own proprietary hooks anyway).

  • David Rajchenbach-Teller

    I’m sometimes involved in add-on stuff in Firefox, but I’m not in any way involved to WebExtensions, so keep in mind that what follows is purely an interpretation. Also, I’m writing as a community member (and add-on dev, in a now distant past), not as a Mozilla employee.

    As you mention, there were two possible choices forward, and both of them involved giving up on the current XUL/XPCOM model.

    The first choice is Jetpack, the second is some new API. I have not followed this closely, so I may be entirely wrong, but it is my understanding that there were fundamental design errors in Jetpack – I don’t know which ones, but apparently, the main symptom is that every single Jetpack add-on happens to `require(“chrome”)`, i.e. entirely short-circuits Jetpack itself, which is a pretty bad sign. Also, I remember a public announcement one or two years ago that Jetpack was effectively deprecated for everything except devtools. While it may have been possible to put Jetpack back on tracks, it is my understanding that making Jetpack as good as it was needed would have killed all existing add-ons, which would have made the current situation even uglier. Additionally, to answer your point about extracting the list of modules used by an add-on, it is pretty easy to write code that obfuscates this very convincingly and make any analysis untractable.

    The second choice is to go for a new API. Rather than inventing a new API from scratch, I believe that it makes sense to start with an API that is widely successful (in term of numbers of maintained & downloaded add-ons), and work on improving it. I know that Mozilla hopes that this will make the life of add-on developers much simpler. It is also my understanding that the WebExtension team does not aim for bug-for-bug compatibility with Blink. WebExtensions will not magically make all Blink add-ons work with Firefox, but they should make it much simpler to share code between the Blink, the Firefox version and, apparently, the Edge version. It is pretty clear that Blink, Firefox and Edge will all implement supersets of WebExtension and support feature detection.

    As for the question of permissions, you raise very valid questions. It is my understanding that Firefox-style permissions will not be the same as Chrome-style permissions, but I do not know any specifics. I can only suggest you get in touch with the WebExtensions team and work with them on making Firefox-style permissions better than Chrome-style permissions.

    Wladimir Palant

    Interesting, I must have missed a whole lot of discussions. Many SDK add-ons using low-level APIs doesn’t really match my experience – at least the simple add-ons I’ve seen didn’t do that. For the more complicated add-ons the high-level APIs provided were insufficient, sure. But these add-ons wouldn’t have worked with Chrome APIs either. If one wants to extend Chrome APIs, extending high-level APIs in the SDK would work just the same (and have the advantage of using CommonJS instead of ugly namespacing).

    Sure, obfuscating the required modules is always possible. However, AMO already requires the unobfuscated form for any source code. And it’s not like the permissions model would make reviews obsolete, I sincerely hope that this is not the idea here.

  • Michael Kaply

    I would love to know how many Mozilla folks involved in making this decision have actually developed really useful Chrome extensions (as opposed to just looking at the API).

    I’m surprised people like you (that have developed for both Firefox and Chrome) weren’t consulted as to how useful the Chrome API really is.

    To me, this is really an acknowledgement that Chrome has somehow won the extension war. I find that an odd conclusion.

  • Joe

    As a developer who loves Mozilla, I want to remind them that Mozilla does not have enough resource to compete with Google. Even if WebExtensions become somehow an standarized think (which I really doubt that), still the only one who benefits from this is Google Chrome. People solely use Firefox because they can alter it the way they like to, not because it is fast, nor because it is as responsive as Chrome.

    I do understand that security is the concern here, but why instead of blocking something useful like require(‘chrome’), why not just warning users the extension can access to your sensitive data?

    Also note that you will need something like require(‘chrome’) even in your new ecosystem. Without this small thing, Firefox is yet another clone to Google Chrome from extension development point of view.

    Let me emphasis one more thing here, as Wladimir also noted, it is to Firefox benefit to not have same APIs as Chrome supports, although I do think Chrome extensions need to be compatible with Firefox which can be achieved just by writing a simple wrapper to the current SDK. This way Firefox could take advantage of all the Chrome’s extensions gallery as well as its own gallery.

    Let me remind one other important issue with abandoning current extensions as well. I am pretty sure at least 50% of current addons hosted on AMO are not being actively developed anymore. Most of the developers just maintain their extensions. You cannot definitely ask these developers to migrate their extensions to the new system. And guess how many users are in Firefox just because of these small not-really-maintained-anymore extensions.

  • LorenzoC

    Once upon a time everybody was “making the Web” meaning the Web was Internet Explorer. Less tech users actually believed the “e” icon on the desktop was “the Internet”.

    Few people considered “other browsers”, at the time Opera and whatever Netscape was doing. From the ashes of Netscape rose Phoenix, then named Firebird, then Firefox (If I recall it correctly).

    Was Firefox “compatible” with Internet Explorer? Was it trying to appeal developers by sort of “common api”? Did even Firefox mimic Internet Explorer design? On the contrary, it went in the opposite direction in almost any aspect.

    But above everything else, did people making Firefox believe their software was actually hopeless and inferior and they did not have the man power to do anything different/better than IE so all they could do was sort of “side projects” that are born to be irrelevant (like, sorry, FirefoxOS)?

    I don’t know what Mozilla stands for, is it a no-profit organization driven by ethics (and/or politics) that use software as excuse to exist or does Mozilla develop innovative software as mission?

    The thing here is you can even take away any kind of extension/addon from the browser and provide only hardcoded features but you must give something in return. Your browser must be BETTER than the others in a way o another. By saying you are adopting the same “model” as Chrome and/or you are becoming “compatible”, without giving any actual improvement over Chrome as a whole, you are saying you use Chrome at home.

  • Goel

    Add-ons using require(“chrome”) somewhere happens when the SDK APIs aren’t enough.

    I think it’s a symptom of a problem with the ecosystem. When something isn’t available in the SDK, it’s easier to poke into XPCOM and stop once the extension works than to make a reusable module. I’ve seen a few third-party JetPack modules, but I have no idea how someone would make them discoverable or get them reviewed and accepted into Firefox.

    The SDK has a good story for e10s compatibility: high-level APIs are known to be safe, low-level APIs may need to be in a content process or a background process, depending on usage.
    Given the very incomplete state of reimplementing the Chrome APIs (so far this preview is less featureful than even GreaseMonkey; you couldn’t port a script that has persistent settings without doing some funky async IPC), the SDK is the easiest way to get past the Electrolysis transition, for new or existing extensions.
    If there are real, systemic problems with the SDK, someone who knows them should chime in.

    Anyway, how useful and successful the WebExtensions model becomes will depend on Mozilla defining the perimeter of APIs it is willing to accept (is competing with or extending Chrome APIs allowed? if an API wouldn’t make sense on another browser, is it still allowed? is fixing or configuring firefox features allowed? do all APIs have to be supported long-term? What are the quality requirements? What are the security boundaries?). There also needs to be a process to graduate sensible proposals into supported API (though not all of it will cross over to Servo; this could be a mechanism for sunsetting). There is a tradeoff between maintainability and power, but this is an “extension” API, and Mozilla needs to give some guidance to the community here.

  • Quicksaver

    [ TL;DR : From reading your blog post I get the impression you, just as almost everyone else who read the announcement, myself included on my first read-through, took WebExtensions as the future one and absolute way of writing an extension. IMO, WE isn’t supposed to be the only way to code an add-on, but rather a framework to do it, replacing the SDK. ]

    I think the biggest thing to keep in mind is the distinction between all the subjects covered in the announcement:

    - as you said yourself, alternatives to XUL and XPCOM have become an increasing necessity, deprecating XUL-based add-ons and those that use XPCOM is the logical first step in getting there;
    - signing has also become a necessity for its own reasons;
    - so has e10s, I think we all want a faster and more stable browser;
    - WE was introduced as the future alternative to the SDK, it is meant to (supposedly?) achieve the same goals, both in end-product functionality and in ease-of-development, with a few added bonuses, the biggest of which seems to be browser portability.

    All of those points are valid, and they have nothing to do with each other. They just happened to be jumbled up in the same blog announcement because they’re happening at the same time (and logically there will be some overlap in achieving all these goals), so most of it was lost in the middle of the whole “holy hell” reaction. You’re right that the SDK has matured over the years, but if you look at it from the outside, it seems to me the SDK was a sort of pre-WE in itself: it has its own modules to interact with the browser in its own “arbitrary” ways.

    IMHO, if I look only to the WE over SDK part and forget everything else, WE apparently aims to bring the best points from other established models (Blink) into Firefox, some things that the SDK doesn’t have or can’t do as it is.

    Maybe those things could have been brought into the SDK itself as has been suggested all around, I guess the design decision there was more in favor of browser portability than continuance. Whether that’s actually good or bad, well I’m sure it’s good for those that developer for multiple browsers, and I’m also sure that it in itself shouldn’t really hurt those that develop only for Firefox.

    I also wasn’t a fan of the SDK at first (I’ve still never used it actually), precisely because when it was in its infancy it could barely do anything useful. That’s the stage WE is in right now. And remember the goal at the time the SDK was created was “let’s bring as many developers as we can into using it”. Now apparently the goal with WE is “we want even more developers using WE than that”, and if it the point is to try and fit into it as much as add-ons need, as far as signs go I can’t say that’s particularly a bad one.

    What is setting off this little wave of rage and despair is that unfortunately the announcement made it sound like WE will be the only way to create add-ons, which I believe is false. XUL-based and SDK-based add-ons aren’t the only formats, you can also create a blank bootstrap.js file and code a restartless add-on from scratch without using a single piece of the SDK. I did, many times over.

    (I’m an optimist, I haven’t seen anything about dropping support for regular restartless add-ons, so I presume those will stay, even if they need some kind of special permission in the new WE permissions model to allow them to work. I did voice that idea just to be safe: https://webextensions.uservoice.com/forums/315663-webextension-api-ideas/suggestions/9440937-deprecate-xul-does-not-mean-deprecate-all-add-ons, and I wasn’t the only one apparently.)

    I think that’s the key point, because while I agree there isn’t a rationale for WE to become the only way to create Firefox add-ons, that’s of course a little insane to say the least, there is definitely a rationale, and a good one IMO, for WE as a framework model to create Firefox add-ons. To that point, so far I’ve not seen any XUL-based add-on that couldn’t be converted into a restartless format, with more or less effort of course depending on the case:

    - XUL deprecation is irrelevant, whatever you modify now you can still do so in an HTML environment as long as you have access to the DOM tree, and any anonymous nodes or XBL binding methods that go with it can be changed as well (now and in whatever comes to replace them) with some clever JS’ing;

    - XPCOM also has alternatives apparently, although I won’t say much about them as I’ve never used them myself, I’m just basing that on the impression I get from what I gather here and there;

    - signing isn’t a deal breaker, pretty much everything on AMO is signed already, add-ons not hosted there can also be signed (it is extremely easy to do it even), everyone can sign their personal add-ons without having them listed or made public, anyone can pick up an abandoned add-on and submit it as-is for signing and republish it, developers will be able to use unsigned versions in NIghtly and Dev.Edition and even in release and beta Firefox (those complaining about this should do proper research, “unbranded versions” means “exactly the same Firefox version except for the name and logo”), ESR will apparently keep the preference to not enforce signing, and lots more should be coming in the future, what more could anyone want? :)

    - e10s, well in many cases that will take a little work I guess, but if I, an absolutely amateur coder with no academic background in any kind of computers sciences what-so-ever doing everything in his spare time, managed to rewrite FindBar Tweak, an add-on that was absolutely riddled with e10s-related problems because it touched web content in every step of the way, to an e10s-friendly format, then the same absolutely can be done for every other add-on out there, it only takes some effort, and you won’t believe the performance increase I tell’z ya’ :)

    To quote myself, I’ve always believed it is the job of an add-on developer to adapt with Firefox, and not the other way around; after all, add-ons are made for Firefox, Firefox isn’t made for add-ons. (I know this is a point that many don’t agree with, but as far as middle ground goes, I think it’s the most appropriate way of looking at things.) Obviously some add-ons will be left behind if no one works on them, but if Firefox is rapidly and deeply changing, so must its add-ons.

    With that in mind, we’re at a stage where every add-on needs to adapt, as Firefox itself is undergoing massive changes. WE comes as a (future) alternative, for those who decide to (re)write their add-ons from scratch, just as the SDK is/does now. But as long as it doesn’t become the only way to code add-ons as I hope it doesn’t, the world of Firefox add-ons will be fine, it just needs some evolving and work done in order to keep going.

    Wladimir Palant

    I didn’t just read the announcement, I also studied the comments and all other available information. There isn’t a whole lot of it.

    Yes, the announcement was formulated very awkwardly – it started with well-known upcoming changes (signing and e10s) only to drop the bomb towards the end of it. It’s not surprising that everybody is ignoring the first part of the announcement, these topics have been discussed many times before. However, the way it was presented, deprecating XUL/XPCOM and introducing WebExtensions is very much related, with the latter being the preferred option in the new XUL-free world.

    Why the cross-browser portability (or an illusion of such) is suddenly more important than continuance is exactly my question here. Experience shows that extension frameworks take years to become usable. This was the case for the SDK and it was the same with Chrome’s extension APIs. I might understand the decision to start over if Chrome’s APIs were an architectural masterpiece – but they are not, IMHO they are inferior to the Add-on SDK.

    Note that Adblock Plus exists for both Firefox and Chrome – and yet we are horrified at the prospect of using the Chrome version in Firefox. Supporting Chrome always required hacks and tradeoffs, much of the functionality simply couldn’t be ported at all. Adblock Plus for Firefox was always the extension where we could do things properly and provide the best user experience.

    Note that all Firefox add-ons I worked on are restartless, not based on the SDK however. That’s what you mean by “blank bootstrap.js file” apparently. My understanding is that these add-ons are deprecated as well now – add-ons are no longer supposed to access browser internals, only via high-level SDK modules or WebExtensions. Everything else would still mean that add-ons depend on browser’s implementation details, so interdependencies between browser and add-on development won’t be resolved.

    I don’t mind having WebExtensions around, there is certainly a use case for it. But – no, I think that the announcement definitely goes in the direction that WebExtensions are to become the preferred and in the long term the only framework for developing extensions.

  • Quicksaver

    (In reply to your reply…)

    To make this one shorter, I absolutely agree with everything you said. In particular, I would have also preferred to see a continuance of the SDK, although I also don’t know why this decision was made. I find it unfortunate that you may be right (no disrespect intended of course), in that while WE has its place and can even be the preferred method, it may also lead to deprecating the whole current restartless/all-internals-accessing add-ons format, like I’ve also seen suggested before, as have you I’m sure.

    For instance in your example with ABP, you’ll have to make lots of sacrifices similar to the Chrome version, and that’s a real shame considering not only the nature of the add-on but also how popular it is. And that’s not even mentioning quite a lot of add-ons that rely on the depths of Firefox (like mine) which won’t have the APIs they would need so easily, some of which I can’t even imagine how they would work.

    But then again, the announcement was made only 4 days ago, and none of this will happen over night. Lots of opinions are still being voiced all around, which was the point anyway. I guess I’m an optimist in presuming I’m right and you’re wrong (again, no disrespect intended) about making WE the only way to write add-ons. When the developer of the most popular Firefox add-on out there so eloquently suggests it might suffer with WE, at least as things stand now, I find it hard to believe that something won’t be done to help with that. :)

    Wladimir Palant

    We’ve been gradually improving Adblock Plus for Chrome version for the past four years and we still have a long road ahead of us. One year really isn’t much if the goal is creating an entirely new codebase with comparable functionality. So I’d rather not wait too long.

  • Danny Shintag

    Hi Wladimir,

    I’ve seen a number of add-on developers who have publicly declared that they’ve had enough of Mozilla’s shenanigans – and that they’d halt developing their add-ons. Greasemonkey’s developer also seemed quite annoyed at the prospect of rewriting pretty much everything, after having spent 9 months making it compatible with e10s. DownThemAll is pretty much counting down the seconds to the add-on’s demise.

    I guess many users are wondering whether you and the team would still be committed enough to the ABP project to jump through the many hoops now required, vis-a-vis WebExtensions… Although it’s still early days, this does not bode well for Mozilla’s reputation. I, for one, am deeply disappointed with Mozilla’s lack of clear direction, despite having hung on over the last couple of years.

    Shame about the large baggage of experience and code they’re about to chuck.

  • anonymous

    FAQ – Why WebExtensions and not Jetpack?

    https://wiki.mozilla.org/WebExtensions/FAQ#Why_WebExtensions_and_not_Jetpack.3F

  • Noitidart

    I strongly agree with your article.

    The review process by a human is critical. I know users of Gchrome and they refuse to download anything from the google extension store.

    I am for ADDING WebExtensions but not removing all that we have right now. I want to add it because: (1) it brings all of chromes addons to firefox (2) firefox addons will not go over to chrome (3) this webext api is a step simpler then addon-sdk, newbies can jump into this api more easily, and then grow into addon-sdk and then classic bootstrap

    I take pride in my addons not being compatible with anything else other then Firefox. I would hate to see my addons go to help other browsers who are absolutely cruel to the user. I like your example of “gchrome show a warnning on addon install, trains users to ignore them, and when they accept them its not googles fault for whatever privacy violations” thats just a simple example of how evil google is.

    Lets take their addons though and bring them to firefox. But that doesnt mean we have to remove our wonderful addons already existing and which years of work has gone into by addon developers.

    Us firefox addon devs that are firefox only really cared for it. When gchrome cam around and addon devs jumped to gchrome they spit on firefox, why should we cater to them? We should cater to the ones to stuck with firefox through the thick and thin, and did their best to help out through the thin.

  • Noitidart

    I read this in a reply by @Wladamir:

    “Note that Adblock Plus exists for both Firefox and Chrome – and yet we are horrified at the prospect of using the Chrome version in Firefox. “

    I think that speaks a thousand volumes!! Amen baby!

    Also I highly doubt security of Firefox addons was ever an issue, we have a great review process that has proved over and over again to be more secure then google chromes.

    Even with our awesome power of addons, the review process has been a fool proof way of ensuring security.

    And the new signing process ensures security on addons not on addons.mozilla.org (AMO) so I don’t believe this “WebExtensions API is needed as we had security issues”. We never had security issues due to the review process. And any ones that would come up would be squashed by the signing process.

    Wladimir Palant

    I wish I could share your confidence. While malicious add-ons and privacy violations within reviewed add-ons on addons.mozilla.org are rare, security issues do happen. There are still plenty of ways to mess up security, and reviewers don’t always have the experience to recognize issues.