Resources for beginners? (very interested)

[Noitidart]

Hi there,
I am classic Firefox addon developer who used FFI via js (js-ctypes) heavily to tap into platform APIs. I supported all platforms including Android.

With the move to WebExtensions all my work is not possible except as stand alone app which talks to the browser addon via native messaging. I will stop supporting Android as it's not possible with this model. I will use the browser for XHR requests using their cookies (oauth) and a HTML dashboard/options panel.

This time around I was hoping to not use FFI but use C/C++ (objective-c) and pass the information as binary string or array buffer to a "web view" for things like react-redux based canvas editor. Then pass it back to my addon. An example of this is my current addon - https://addons.mozilla.org/en-US/firefox/addon/nativeshot/ (it has a bug on mac, but a fix for that will be approved sometime this week).

I was recommended to use "Electron" but as moz.fanboi I wanted to use Positron. I see development is going on. I am just wondering if you will have resources for a beginner to writing C++ (like me) before Jun 2017 (after probably Oct 2017 my addons will be defunct).

[mykmelez]

Hi @Noitidart, thanks for describing your use case! I have a few questions to ensure I understand it well, and a few comments…

I am classic Firefox addon developer who used FFI via js (js-ctypes) heavily to tap into platform APIs. I supported all platforms including Android.

Do any of your addons contain native code, or are they pure JavaScript? I'm curious about whether their use of js-ctypes is restricted to accessing platform libraries, or whether they include shared libraries that they load and access via js-ctypes.

Also: do any of them access any Mozilla libraries (in order to use APIs that aren't exposed via XPCOM); and if so, which ones?

With the move to WebExtensions all my work is not possible except as stand alone app which talks to the browser addon via native messaging. I will stop supporting Android as it's not possible with this model. I will use the browser for XHR requests using their cookies (oauth) and a HTML dashboard/options panel.

This time around I was hoping to not use FFI but use C/C++ (objective-c)

Are you hoping to develop the standalone app in those languages because you personally prefer them, because they'll reduce engineering effort, or for some other reason?

and pass the information as binary string or array buffer to a "web view" for things like react-redux based canvas editor. Then pass it back to my addon. An example of this is my current addon - https://addons.mozilla.org/en-US/firefox/addon/nativeshot/ (it has a bug on mac, but a fix for that will be approved sometime this week).

Hmm, it sounds like the standalone app would then need to embed a rendering engine like Gecko. That's possible to do—it's what Firefox does, after all—but it currently isn't well-supported for apps other than Firefox, and the supported way is to build a Mozilla app as part of the mozilla-central repository, which is complex.

I was recommended to use "Electron" but as moz.fanboi I wanted to use Positron. I see development is going on. I am just wondering if you will have resources for a beginner to writing C++ (like me) before Jun 2017 (after probably Oct 2017 my addons will be defunct).

Development is indeed taking place, but Positron remains an experiment, just like Tofino, so I can't recommend depending on it at this point in time. Also, Positron, like Electron, is designed for the use case of an app developer who wants to develop a desktop app using web technologies—HTML, JS, and CSS—instead of native code.

And Positron, again like Electron, exposes system integration APIs as JavaScript APIs. It's possible to write native code as a Node module, but those modules then expose JavaScript APIs to the (web-like) application.

For your use case, I wonder if something like the native.js proposal would make more sense than writing a standalone app. Alternately, I've been thinking about the possibility of integrating SpiderNode into Firefox and exposing it to WebExtensions that want to access system APIs.

Both of those solutions involve writing JavaScript code rather than native code (although in the case of SpiderNode it might be possible to write native Node modules). But they avoid the need to create a standalone app.

[Noitidart]

I have a few questions to ensure I understand it well, and a few comments…

Thank you @mykmelez for the fast reply and asking me to get even more detailed :)

Do any of your addons contain native code, or are they pure JavaScript? I'm curious about whether their use of js-ctypes is restricted to accessing platform libraries, or whether they include shared libraries that they load and access via js-ctypes.

It's not native code per say. It's all javascript, but all the types and functions had to be redefined in javascript. Then after that I just wrote javascript. All my addons depend on ostypes submodule - https://github.com/Noitidart/ostypes - as we see in ostypes_*.jsm it declares native stuff with javascript.

Also: do any of them access any Mozilla libraries (in order to use APIs that aren't exposed via XPCOM); and if so, which ones?

I don't access any Moz libs. Only Windows (WinAPI, COM), Mac (Cocoa, CoreFoundation, Carbon, libc), and *nix (GTK+, X11, XCB), APIs. I used minimal XPCOM, only thing I used was window watcher service, framescripts, and about: page registration. Just Services.jsm stuff, us addon devs were hoping as XPCOM phased out, they would rewire Services.jsm to the Rust or whatever replaced it. Services.jsm functionality is core to any piece of software that wants to call itself a web browser.

Are you hoping to develop the standalone app in those languages because you personally prefer them, because they'll reduce engineering effort, or for some other reason?

I preferred js-ctypes, because it was very performant (of course slower then native code but no affect to the main thread GUI experience). All my js-ctypes, XHR, file system stuff (also js-ctypes) and other stuff I would do in ChromeWorkers, which is off of the mainthread. Then I would transfer ArrayBuffer to the main thread, and then in the screenshot addon, I would transfer it to a window with postMessage and run my react-redux canvas editor app. Once user finishes editing it sends the ArrayBuffer back. So I had really high performance, no copying of huge binary strings. Just structured clones and transferring. Extremely performant stuff!

However WebExtensions really change the field. None of this stuff is possible. I can use WebRTC to get screenshot, screencasts, but its lacking, and its just one use. I don't have fine control over simple things like showing/hiding the cursor in the screenshot. And some other major things, extremely funky bugs when getting screenshot of single window etc.

I also want my products for the benefit of Firefox and Firefox users only. With WebExtensiosn anyone can easily steal it and take it Google Chrome for instance. So after lots of asking, unfortunately the only way is to not depend on Moz technologies. It's also a little safety net, lots of us addon developers feel stranded and feel that Mozilla has no qualms about doing it to us again - we won't be surprise to see WebExtensions to be replaced in the future and all our work made useless again. I see their point though, Firefox addons are apparently a burden and it does not help Firefox win over its competition.

And Positron, again like Electron, exposes system integration APIs as JavaScript APIs. It's possible to write native code as a Node module, but those modules then expose JavaScript APIs to the (web-like) application.

I understand. I can work with that. I'll try to learn and use Electron and when Positron is ready I'll move to that. :)

I actually still have some scenarios where I need to run native code from the process. For instance for most key listening APIs, I cannot listen at the system level, as that is a key logger. I have to listen only in the target app. I have this feature because I allow users to change the "global hotkey" by recording.

For your use case, I wonder if something like the native.js proposal

The proposal was perfect! However the actual implementation, which landed a couple weeks ago is nothing like it. :( https://wiki.mozilla.org/WebExtensions/Experiments#Other_proposals

Examples of my code "native code" from js-ctypes

I'm not too good at explaining, my code and work really do a better job of talking:

• My screenshot addon, which would need to use Positron/Electron webview for the cavnas editor is seen here - https://addons.mozilla.org/en-US/firefox/addon/nativeshot/ - v1.11 has a mac bug, so if you want to test it out, please use v1.12 - it is pending approval - but you can grab the usngiend version from the versions page - https://addons.mozilla.org/en-US/firefox/addon/nativeshot/versions/
• This example is me using js-ctypes to send a file to the "system recycle bin" - https://github.com/Noitidart/NativeShot/blob/master/resources/scripts/MainWorker.js#L384-L507
• Here is a file watcher submodule is js-ctypes - https://github.com/Noitidart/jscFileWatcher/issues
• Here is a global hotkey submodule in js-ctype - https://github.com/Noitidart/jscSystemHotkey/commits/master
• Here is an addon that uses the global hotkey module to set windows always on top - https://addons.mozilla.org/en-US/firefox/addon/topick/

[mykmelez]

I also want my products for the benefit of Firefox and Firefox users only. With WebExtensiosn anyone can easily steal it and take it Google Chrome for instance. So after lots of asking, unfortunately the only way is to not depend on Moz technologies. It's also a little safety net, lots of us addon developers feel stranded and feel that Mozilla has no qualms about doing it to us again - we won't be surprise to see WebExtensions to be replaced in the future and all our work made useless again. I see their point though, Firefox addons are apparently a burden and it does not help Firefox win over its competition.

I certainly understand why addon developers would feel stranded. I also know that there are no guarantees in technology, and today's best solution is tomorrow's abandonware.

Nevertheless Electron is mature, actively developed, and has momentum in the market for desktop application frameworks, so it's as good a choice as any for developing a standalone app generally (although I'm unsure about how suitable it is for this particular use case).

I understand. I can work with that. I'll try to learn and use Electron and when Positron is ready I'll move to that. :)

Note that, because Positron is an experiment, there isn't yet a plan for it to ever be "ready" in the sense I think you mean. And currently, our only goal, as described on the Positron roadmap, is to support Tofino. So I don't recommend waiting for Positron. (I'd love to change that, but that's the current status, and I want to make sure you aren't misled!)

I actually still have some scenarios where I need to run native code from the process. For instance for most key listening APIs, I cannot listen at the system level, as that is a key logger. I have to listen only in the target app. I have this feature because I allow users to change the "global hotkey" by recording.

FWIW, Electron explicitly supports creating global hotkeys via its globalShortcut API.

The proposal was perfect! However the actual implementation, which landed a couple weeks ago is nothing like it. :( https://wiki.mozilla.org/WebExtensions/Experiments#Other_proposals

That would be useful to note in the native.js bug, where the most recent commenter has asked if the bug can be closed because of WebExtensions/Experiments.

What about giving you access to Node APIs (both core Node APIs, third-party modules from NPM, and your own modules) in a WebExtension? Would that address your use case?

[Noitidart]

I also know that there are no guarantees in technology, and today's best solution is tomorrow's abandonware.

Haha true. Me and the addon devs need to get that in our head.

I think we understand it to a certain extent, for instance we would maintain for deprecation and new featres (e10s etc) but hearing we should expect many of our current features to be permanently deprecated with no replacement possible really worried us.

What about giving you access to Node APIs (both core Node APIs, third-party modules from NPM, and your own modules) in a WebExtension? Would that address your use case?

That would be perfect! All worries would be gone!

I don't understand how to use Node APIs or 3rd party npm modules that are not js based, but if there was a guide on it I would love it!

I could bring my git submodules over with copy-paste! Basically if background pages of WebExtensions could spawn a ChromeWorker, I can run my js-ctypes and OS.File there and transfer the ArrayBuffer it to the background page. From there I would need to use WebExtensions structured clone and copy the ArrayBuffer to wherever I need it (like my canvas editor in an new window).

I would also need to get the native handle to the window, so I can use js-ctypes to controls it placement to each monitor, and size to cover each monitor (different from fullscreen). I currently get the handle like this - https://developer.mozilla.org/en-US/Add-ons/Code_snippets/Finding_Window_Handles#Yet_Another_way_to_find_a_window_handle_(parent_window_handle)

Talking to you really gives me hope that I had lost! I really appreciate it!

PS:
Talking to you gave me so much hope. One thing I needed also was more control on "opening window" in Mac - https://bugzilla.mozilla.org/show_bug.cgi?id=1264280 - just wanted to mention because I feel full of hope right now haha

Note that, because Positron is an experiment, there isn't yet a plan for it to ever be "ready" in the sense I think you mean.

I really appreciate your detailed explanation to a script kiddie like me. I understand now more about Positron and Electron! If I try to do something I'll use Electron, but I see now that it wont work for the addons. Thank you. :)

That would be useful to note in the native.js bug, where the most recent commenter has asked if the bug can be closed because of WebExtensions/Experiments.

I'll make a comment there :) I saw Andy make a note there, he's a big guy, so kind of intimidating though to post after him :P

[mykmelez]

I don't understand how to use Node APIs or 3rd party npm modules that are not js based, but if there was a guide on it I would love it!

Native modules for Node are called "addons", and there's extensive documentation on Node.js Addons, but I'm not sure how we would support them in an implementation of Node for WebExtensions, since they expect to be compiled at installation time for the machine on which a Node app is being installed.

Perhaps Firefox could provide something like js-ctypes to Node, so WebExtensions could use JS to access native libraries. There are some related projects, particularly node-ffi, but also node-ctypes. Note also nodejs/node-v0.x-archive#2204.

I could bring my git submodules over with copy-paste! Basically if background pages of WebExtensions could spawn a ChromeWorker, I can run my js-ctypes and OS.File there and transfer the ArrayBuffer it to the background page. From there I would need to use WebExtensions structured clone and copy the ArrayBuffer to wherever I need it (like my canvas editor in an new window).

I'm unsure exactly how this would translate to a Node feature for WebExtensions, but it might look something like this: a WebExtension that opted into Node access would be able to spawn a process that runs a JS script as a Node program. The WebExtension and the Node program would communicate via an IPC API (supporting JSON or structured clones), and the Node program would have access to something like js-ctypes along with the standard Node APIs for filesystem access (i.e. the equivalent of OS.File, except that they'd be the Node APIs), so it could do the things your addon currently does with js-ctypes and OS.File, then transfer the ArrayBuffer to the WebExtension via IPC.

I would also need to get the native handle to the window, so I can use js-ctypes to controls it placement to each monitor, and size to cover each monitor (different from fullscreen). I currently get the handle like this - https://developer.mozilla.org/en-US/Add-ons/Code_snippets/Finding_Window_Handles#Yet_Another_way_to_find_a_window_handle_(parent_window_handle)

Hmm, I'm unsure how to do this.

Talking to you really gives me hope that I had lost! I really appreciate it!

PS:
Talking to you gave me so much hope. One thing I needed also was more control on "opening window" in Mac - https://bugzilla.mozilla.org/show_bug.cgi?id=1264280 - just wanted to mention because I feel full of hope right now haha

Thanks for the kind words, just keep in mind that this is only an idea! Although it's possible (especially given the work we've done to integrate SpiderNode into Positron), I don't know if it's something anyone will actually implement.

I'll make a comment there :) I saw Andy make a note there, he's a big guy, so kind of intimidating though to post after him :P

I understand, but I'm sure that Andy wants WebExtensions APIs to be useful to addon developers and thus will want to hear about use cases that aren't satisfied by the current APIs (nor by Experiments).

[Noitidart]

Thanks for the kind words, just keep in mind that this is only an idea!

Totally totally understand. There's an equal or more chance of it not happening. Just that the conversation we had is not how any of the others have gone, it had a very positive feel. And actual alternatives were discussed. :) No ending in "go write up a proposal, do a cost analysis, time analysis, and we have good will", that stuff seemed heavily PR'ish.

[Noitidart]

What I'm working on right now is learning C++ and handling the stuff in command line style compiled stuff. It seems native messaging conencts to it without popping open the command line and it terminates it, which is cool! I hope to learn enough C++ to use Chromium Embeded Framework so I can drop in a webview when needed.

[Ryuno-Ki]

I've seen some Node scripts spawning a shell script and consume their output. This way you can use almost any CLI from within Node.

[Noitidart]

Thanks @Ryuno-Ki that's bascially what I'm doing, but not a shell script, it's a "console application" - that's what visual studio says (very new to this stuff - i guess console app is cli app).

[Ryuno-Ki]

I had the spawn API in mind.
We used baudio in last year's Node Hackathon, which used that.