mismatch between what is maybe possible today and what is going on in many of today's IDEs.

Transcription

1 What will happen if we do very, very small and lightweight tools instead of heavyweight, integrated big IDEs? Lecturer: Martin Lippert, VMware and Eclispe tooling expert LIPPERT: Welcome, everybody, to this talk about Embracing Eclipse Orion. My name is Martin Lippert. I work for VMware for the tooling team, so the team that is doing a lot of Eclipse-based tooling. Before I start, who knows, who's familiar with Eclipse Orion in terms of what is it and what is it for? Okay. That's awesome. Perfect. Who is developing JavaScript? Maybe not all the time, but sometimes? Okay. Good. So what I would like to talk about is how we are using Eclipse Orion and what we did with Eclipse Orion. It sounds maybe sometimes a little bit brutal with what we ripped out and what we put in, but we're happy members of the Eclipse Orion team, so some of my team is actually also contributing to Eclipse Orion and we are working with them, so it's not really... it's really a great collaboration. So, a little bit about the background. So we are implementing tooling for quite some time now. Worked a lot on Spring tools, Spring IDE, AspectJ tooling over the past maybe five, six, seven, eight years now. So it's a long, long history in implementing tooling based on Eclipse and based on this kind of idea of having a desktop IDE where everything is integrated and highly integrated into. Based on this background, we thought, we stepped a little bit back over the past one or two years to think about, how should the developer tooling of the future look like? So are we going to stay with this, let's say, Eclipse IDE or intelligent whatever, these kind of desktop IDEs that are really integrated or, will that change maybe? And in what direction would that change? So, personally I don't know at the moment, but I have some ideas. The driving question behind that to me, the thing that initiated my thoughts, was this observation like, hey, there's a lot of stuff going on around IDEs, or let's say, beyond IDEs for application development, right? People are saying, "oh, yes, you need to do application development, it's all web-based, all web-based apps" and, "oh, they built all of JavaScript, now like Gmail and things like that," so all these modern and popular things are happening outside and inside the IDE development. Like, "oh, yeah, yeah, we are conservative, we've done the same as for 10 years ago doing SWT and desktop, big desktop-based applications, need to download 300 megabytes, install" and things like that. That makes me... I don't say it's bad, I just think, [it] made me think about what should happen to the IDE landscape in the future. And the second thought I had is that my observation of, hey, Google can search the whole web of the whole world in 10 milliseconds; why does it take one second to open a type hierarchy in Eclipse? That's kind of strange, isn't it? It looks like, hey, there's a mismatch,

2 mismatch between what is maybe possible today and what is going on in many of today's IDEs. So we went a step forward and thought, okay, let's try to explore a little bit, let's try to be innovative. And what happens if we turn everything upside down? What happens if we would do everything differently for IDEs? What will happen if we do very, very small and lightweight tools instead of heavyweight, integrated big IDEs? What would happen if we reuse existing tools instead of implementing or integrating them all over again into existing IDEs like what we are oftentimes doing for Eclipse-based integrations? What about connecting existing tools, like what people do on the web, instead of integrating everything into one big tool? What would happen if we go browser based, everything is browser based? Every UI today for modern apps is browser based. What would happen if we do IDEs inside browsers, or maybe not IDEs but develop a tooling inside browsers? And what would happen if we do that in the cloud, like this cloud-hosted environment, everything is hosted and everything is running on five million machines and the Google data center simultaneously for me and I have these five million machines and computing power of five million machines at hand at the same time just for me. What would happen? And we thought a little bit about that. And while thinking about that, we came across the Eclipse Orion project. The Eclipse Orion project, this kind of browser-based, open tool integration platform, sounds like, hey, this is something we should take a look at. It seems like it's going into a similar direction or thinking about the same things that we thought of. It's an awesome platform, so we can... it's really reusable, which also made us think, hey, maybe it's a good piece to reuse and to think about how to build something based on Eclipse Orion or at least prototype. So that was what's led us to build our own tool to explore things, to do prototyping. We did several prototypes. We chatted with a lot of JavaScript developers about their needs, what do they really need, what do they like, what they don't like, and that led us to do us the project that we are working on at the moment, which is called Scriptd.

3 Scriptd is a Java-based JavaScript editor, which is lightweight and runs inside your browser. So very, very lightweight. Just pure editor for JavaScript that does a little bit more than just maybe TextMate. Before we started to do this kind of prototyping and thinking about this, turning everything upside down, it was clear that we have to make some decisions and choices. We need to pick something. We cannot go everything. Right? And our choice was to focus on the editor. The editor, we did this because most of the JavaScript developers out there, they are still using TextMate, VI, emacs, whatever, and surprisingly they're really happy about that. They really love to use those editors. They connect remotely to server type VI and they can do everything, right? And usually most of them don't really like something like Eclipse which takes 20 seconds to boot or to start up or has 500 megabytes of download or something like that. Usually they don't really like that. So that made us think about, oh, maybe we should just focus on the editor, nothing really else, it's just plain editor experience. The second thing that we decided is we would like to build something that is very, very lightweight and very fast. So, speed is really essential for those people there; at least what I think. And also at the same time build something that is very keystroke driven, so you can do everything on your keyboard. You don't need your mouse to do everything. It's very, very editor centric. It's basically something like what would happen if you delete all your views inside Eclipse and just have the editor? You can do everything, right? In-place outline, in-place whatever, everything... it could be done with keystrokes in place and things like that. Something like that experience transferred into this, let's say, new world. And on the other side we also talked to people about these kind of cloud-hosted thing and would you be happy to use a cloud-hosted IDE and a cloud-hosted tooling. And most of them said, "oh, yeah, sounds interesting, but, wait, but I cannot upload my stuff to the cloud. Or maybe I can, but I need to use my stuff on my local machine because I have thousands of other tools that I'm using on my local machine to do stuff with my files, like get command line, like grab like whatever" -- thousands of different things that people are using on their local machine. And they would like to reuse those existing tools. And we thought, oh, yeah, maybe we should not really start rebuild everything inside the cloud but just reuse them. So we decided to let the editor first run locally. So you have a browser-based editor, but that can edit stuff locally on your hard drive, can just access your whole file structure. But there was one thing that people were really missing from editor experiences for JavaScript using TextMate or VI, and that is good

4 code comprehension. The things that you have in Eclipse like good auto completion, good content assist, good navigation that the IDE knows what are you working on and can navigate to the right place for that function or that classes define the IDE knows a lot of stuff about the code and the semantics of the code. That was missing. So we thought about maybe we should add that, we should try to put those pieces into this editor experience at the same... while at the same time reusing Eclipse Orion. And we're doing that using an inferencing engine for JavaScript. JavaScript's dynamic, not really typed, so you need to do some more extra work to get those features in. And the other thing that people also mentioned is they're using now kind of module systems for JavaScript like CommonJS, like A and D modules and things like that. And it would be nice if the editor would be bearer of that and can help me a little bit with these modules with my dependencies on my project. So when we took a look at Eclipse Orion, this is very roughly what Eclipse Orion looks like. Maybe I forgot some bits and pieces in there. But there is a client side which is purely JavaScript and there is a big editor component in the middle that's at least my central piece there, which is really an awesome thing. And then a lot of additional UI tooling around that, like you can host sites, you have Git integration, you have a navigator, you have content assist, code folding, JSLint support. A lot of additional things. Even sign on, so it's really a kind of complete set of tooling. And on the server side, Orion is based on some kind of... or, you can run it in hosted mode. The server is running Java and OSGi, so it's based on asynchronous OSGi runtime. And but the server side is very, very heuristic, right? It just serves files and does some... maybe some additional work to do sign on and Git integration. But it's basically running on Java. So what do we do? First we thought, oh, this is an awesome platform and it's very modular. It's very put out of different pieces, bits and pieces. So we decided to rip things out. Right? First we thought, oh, yeah, yeah, this Java-based server, you can run that locally, but we would like to do things different, so let's take, first look at the client, JavaScript client. So what should we rip out? We ripped out the navigator. You don't need a navigator, right? You can do everything in place. You don't need Git integration because running locally you can use a Git command line. People love to use their existing Git tools. We don't need to rebuild that. We don't need site hosting. People deploy their apps anyway somehow and they're running everything locally and then using their node stuff for JavaScript, whatever, so we ripped that out. Taking away sign on, nothing was left, right? And code folding as well. So even pieces out of the editor, we just ripped out.

5 Then we thought, okay, now we have this very heuristic piece off Eclipse Orion; now we should add some stuff back again maybe and implement some additional stuff. So the first thing that we added is cross-file analysis. Cross-file analysis means content assist in Orion... is used today giving you some kind of basic content assist within your file that you have open in your editor in your web browser. But what happens if some functions are defined in another file in your project, libraries? So we added support for analyzing all those files and giving you... providing content assist across files. We added support for A&D modules and for CommonJS modules so that the tooling is aware of those modules and can give you content assist across those modules and can tell you, hey, this module is not defined or this thing is coming out of another module. We added back a very simple navigator, so people from our team, so we have [dock] footing, we are using Scriptd all the time to implement Scriptd. We're using nothing else for that. And we thought, found out that, oh, yeah, maybe just tiny, little navigator to click on a file, maybe that's... maybe it's not that bad, right? So we added back a very, very simple navigator. And an inplace outline so get it in place view of your source code and a very, very simple single page layout. So there are no different views, no different pages that you can open up. It's just one page and that's basically the editor. And you will see that, I will give you a very brief demo. That's the client side. That was really awesome because we were able to rip those other things out of Orion and put different other things back into and assemble it a little bit differently, which was really, really great because you can really reuse a lot of parts, and especially the editor component is really awesome. On the server side, we were a little bit more radical, meaning that, oh, yeah, this OSGi-based Java server implementation of Orion, let's forget about that and let's do a Node JavaScript-based implementation. So we reimplemented the server side based on JavaScript and implemented everything to JavaScript, which is not that complicated because it doesn't really do anything like except from server a few files or so. So that's easily doable in Node, just basically a few lines. So we implemented on the server files and we added some basic search capabilities for the server. Built a Node.js. And added additional functionality to do this cross-file analysis, right, so that the browser is not really able to [search] all your files on file service, so we added some stuff on the server side to allow you to search across different files in your project to find out what is your project and things like that.

6 So that is what the server side a looking like at the moment. It's a Node server, very lightweight, very fast. Starts up quickly. And a client JavaScript browser-based implementation. So, let me quickly show you what that looks like. Therefore, I go to my terminal and said, "oh, yeah, we would like to get the same experience like TextMate or whatever, so I can just do, here," let's... I have an example project. This is an example JavaScript project here. And I can just type "SCR app JS," for example. And that opens up or starts up Scriptd and you get the editor and your browser opened up - - very fast, very lightweight, very easy. And at this moment when I hit enter, it's really starting up everything. So that was everything. Started up. It's just Node really starts up really, really quickly. So what I have here is the editor, the editor has some kind of nice and this tiny little easy navigator, nothing really sophisticated. I can switch between files, things like that. The nice thing, for example, is here I have a module that is defining a person, for example, and I get some nice content assist, for example. So in this case, just press this content assist thing, and it knows, "oh, yeah, yeah, I figured out I understand, oh, you're, there's a construct, kind of constructor thing in this JavaScript file so I can provide you this information saying, oh, yeah, yeah, you need a new person," and I get this tapped editing thing for these things. And even if I'm behind that, I can say, "okay, let's see what does the person have," and if I can do content assist, it gives me, "oh, yeah, the person has this kind of members," the kind of things that are directly in there, functions from the prototype because I defined here full name on the prototype of person, those informations inferred from the type. And in addition to that, of course the functions that are on the object prototype. And then I have some kind of fallback because we all know that it's not really possible to infer everything exactly and accurately in JavaScript because it's so dynamic. We had some kind of fallback saying, okay, in case we don't infer the right thing, we just give you everything. Every possibility that occurs somewhere in your project has some kind of fallback. So when I of course [try now] something, get this kind of, "oh, yeah, maybe I want to do a board or apps or whatever," which doesn't make sense in this case, but let's go with the first thing. Just as an example. The other thing, some kind of breadcrumbs so I can go back in my history, let's go back, or let's go to main in this case, for example. Here I have two modules that I'm using because it is able to understand these A&D modules. I can, for example, it gives me some kind of hint, like, hey, this module is not defined when mistype the

7 name of the module, which is kind of nice because this [isn't] Java runtime saying, "hey, I'm not able to find anything." Directly gives me that information. And it's also able to tell me, "hey, what are those functions and those object types that are coming from other modules, what do they look like?". Like in this case, get user count, say, "ah, this returns a number, not just our arbitrary object," whatever, which is the usual case in JavaScript. But it knows, oh, yeah, it returns a number. And if I would like to navigate to that definition, I can, for example, directly press F8 and I jump directly to the definition of that function -- which is kind of trivial compared to Java, like of course I have all that in my Java IDE, but it's not that trivial for JavaScript and you do not find many tools that do that for JavaScript. The other nice thing is I can also press shift F8, and that opens up a second info pane, like some kind of additional context information, like, oh, I just want to quickly take a look at that, see, ah, you would count that's implemented that way, okay, that's fine, I close that again and I'm back here to my original thing. The other things that I thought is doing everything in place and just, for example, I can also configure that saying I would like to show the navigator. I can, for example, open files just with some kind of keystroke and then type in what should I do, users, then I go to the other file so it finds out, "oh, this is contents of my project, those files are my project, I can quickly navigate to those files." I can also do some kind of find in my file, so look in files. Let's take a look where the name of my colleague Andy is appearing. Oh, that's there. Good. Just very, very quick and easy find through everything in my whole project infrastructure. Since some libraries are in there, I can also navigate and infer types from libraries that I'm using here. For example, if I go back to AppJS and I would like to take this stat which is kind of coming from node static, I can get content assist for that as well because the tooling knows, oh, yeah, that library is in my workspace or in my project. And it can analyze that for me and the server is providing me some kind of summary information that's stored in the client to provide you really quick content assist, because you're seeing it's just one keystroke. It's really, really fast and it's responding quickly. And that to me it looks like, hey, it doesn't look too bad. It looks like it's possible to do a lot of crazy and a lot of interesting things in web IDEs or for web IDEs or inside the browser for developer tooling for implementing new kinds of IDEs. One thing I would like to show you is there is also kind of a support in these lightweight infrastructure. There is a file called.scriptd

8 that you can put into your project, into the root of your project. It's one file where the tooling finds out, "oh, that's the root of my project if there's a.scriptd file in there." The other thing is, "oh, there's a.git file in there. Oh, that's the root of my project." That's good. Because it can, of course, it cannot walk through your whole local disk and try to find everything. Just want to focus on your project. And with this configuration, you can configure, do some basic configuration using JSON file, like in this case I configured, oh, yeah, JSLint, it's kind of JSLint error indication, should be configured this way or that way and it knows, "oh, this is some kind of browser-based environment so it knows, oh, there's some kind of"... the window type and the document type is there and some kind of global definitions require, consult. Please don't put an error marker on that because you don't where that comes from because some kind of globally defined thing. And the other thing that we did is we implemented this kind of, you can define your own keystrokes easily and execute commands in case of this keystroke is executed. So, for example, in this case here, I said after save, so whenever I save something in the browser here, on my local machine on my command line, this command is executed. And this command in this case, for example, updates my file on some hosted Platform as a Service Cloud Foundry thing, like it's a Cloud Foundry command line tool. But you can put anything in there. Like if you want, oh, yeah, every time I press save I would like to do a Git commit to something, you could do that, right, if you want, which makes it very easy, just stay in the editor and define your own keystrokes and say every time I do this, that thing is executed on my local machine. Because it's not in hosted environment, you can basically do everything or use all the tools that you have installed on your local machine and reuse them. From my point of view, very lightweight, very easy way to reuse all the stuff that you have on your local machine. Good. Quickly going back to the presentation, some kind of quick summary. Oh, 25 minutes, really challenging. So our experience is that right from reusing Orion is it's really awesome. It's really an awesome platform. You can really pick... you don't need to take everything that Orion provides; you can also just pick separate components, reuse separate components. You can also of course go the full Orion way of reusing everything. It's a really awesome platform. I think it's very, one of the most promising projects on Eclipse for this kind of web-based tooling and I think it's going to be very, very interesting to see what's coming up there in the next versions. For us, what is next for the Scriptd project? We will work on continuously contribute back to Orion, so some of the content assist that we did is already contributed back to the Orion project. The

9 thing that we are not able to contribute back at the moment is this cross-file analysis because Orion does not have this workspace context at the moment and not the server side JavaScript implementation. Therefore, we cannot just put our JavaScript stuff back to Orion because no place to run it at the moment. We are thinking about more and more crazy things, and we are trying things out and continue to try to think a little bit beyond the borders. If you want to take a look at that or if you want to use that yourself or try that out, it's all EPL on GitHub under scriptdeditor/scriptd. It's all EPL. You can go ahead, give us feedback, contribute back. We are very, very happy about feedback coming from you, if it usable or what are you missing from that. Forum, issue tracker, of course. The sample projects used here is also on GitHub and the slides will be publically available, so I don't think you need to write everything down, or I can Twitter that. And I'm happy to answer maybe one, two, three questions before, I think before we go into lunch, right? Lunch is next? Yes, lunch is next. Good. Questions? We're very happy to answer questions. I think there was a first one there. QUESTION: Just a quick one. Does it run on Windows as well? LIPPERT: The question is, does it run on Windows? Yes, it runs on Windows. The only requirement that we have at the moment is it needs a Node installation on your machine, but Node is, I think all its versions compatible with Windows. So you can run Node on Windows. As long you have Node running, you can just install Scriptd. Very easy. It's just downloading the zip, extracting the zip and putting the bin thing in your path and that's it. As long as Node is there. QUESTION: Related question. How do you do you handle user preferences between different platforms? So one user sets his keystrokes for, I don't know, Linux and the other for Windows-related similar commands? LIPPERT: The question is how do we distinguish on different platforms different keystrokes, user-defined keystrokes. We don't have support for that at the moment. It's all in this.scriptd thing. And what we are working on at the moment is having some kind of global.scriptd thing. So you will be able to put a.scriptd file, for example, in your user home directory instead of into your project. And then Scriptd will use that, and that way you can say, "okay, every user has his own.scriptd file in his own directory, in his user directory, and Scriptd would use that instead of having a Scriptd thing inside every project." Or you can say, "okay, these kind of keystrokes you define in your user home directory are Scriptd and you have an additional.scriptd file, [additional] for your project that does not have keystrokes in there, but just the rest of the information, like the globals and

10 other things, that would be just merged together." It's a bit of support for that, but it's not really taking into at different platforms or so. Not yet. I think there was another question? QUESTION: Yes. How difficult is to, let's say, define grammar for other languages? LIPPERT: The question is how difficult it is to define grammars for other languages. I don't know. I think it is not that difficult. But at the moment we are very much focused on JavaScript. So we built, we're reusing the Esprima parser and that is the most central piece. So we are not generating parsers for different languages out of grammar but we are reusing the Esprima parser. And that parser needs to be recoverable, which is very important. So I think it should be possible to insert parsers for different languages, but how do you create those parsers is at the moment up to you. So we don't provide specific support for that. The only important... the really important thing is need to be fast because we're doing a lot of parsing. And it needs to be recoverable, because otherwise you cannot do content assist -- like if you type something and press dot, it usually is broken, right, your JavaScript is broken to some degree, so the parser needs to be recoverable to provide a good AST information for this dot and then and now should not break in that case. But we don't have any additional like parser generators or whatever. Not at the moment, because we are very much... we're trying to do JavaScript right and then focus on other languages maybe. But it's a very valid thing because people are already asking us just a few days after we published the project, it's just two weeks since we published the project the first on GitHub, people are already asking about type script and CoffeeScript, of course. So it does not surprise me. And it's, maybe we should think about supporting different languages early in the cycle and not too late. Other questions? Okay. Thank you very much. I'm here until the closing session on Thursday, so if you have questions or comments, feel free to grab me somewhere here and tell me. Awesome. Thank you very much. [APPLAUSE] [END OF SEGMENT]