Scaling Xtext. Lieven Lemiengre

Transcription

1 Scaling Xtext Lieven Lemiengre

2 Sigasi IDE for Hardware Description Languages VHDL, (System)Verilog Using Xtext for 4 years Large user base (commercial, free, students)

3 Our company goal Assist hardware designer High quality interactive front-end compiler Instant error reporting parsing, semantic, linting, style checking Visualisations, design exploration,... Integrate with ecosystem other compilers, simulators, synthesizers

4 The challenge Large projects > 250 KLOC is not uncommon design + external libraries big files some libraries are distributed as 1 file generated hardware cores

5 Adopting Xtext Started with the early Xtext 2.0 snapshots Initial performance analysis Clean build performance of a big project (330k LOC) > 20 minutes > 2 GB Editing big files (> 1 MB) unusable

6 Adopting Xtext Started with the early Xtext 2.0 snapshots Initial performance analysis Clean build performance of a big project (330k LOC) > 20 minutes < 1 min > 2 GB < 1 GB memory Editing big files (> 1 MB) unusable usable with reduced editor

7 Improving performance Xtext framework improvements Measure analyze improve or cheat faster build reduce memory usage UI responsiveness

8 Overview Analysing build performance Analyze the build Macro build measurements Key performance points Reduce workload Parallelize the build Analyzing UI issues Monitoring the UI thread Saveguards

9 Analyzing builds: builder overview resource changes Global indexing Linking Validation Custom Validation Builder Participants resource descriptions warnings errors Global index Eclipse resources?

10 Analyzing builds: metrics For each build # of files being build timing: Global index, Linking, Validation, Individual builder participants Instrument by overriding ClusteringBuilderState & XtextBuilder Example: Building 134 resources, timing: { } global index=1806, linking=378, validation=823, totallinkingandvalidation=1364

11 Analyzing builds: resource loads Observation: Most time spent in resource loads Certain files are loaded multiple times?! resources POTENTIAL RELOAD POTENTIAL RELOAD LOAD Global indexing Linking Validation Builder Participants

12 Analyzing builds: Memory footprint Global index Resource Set Global index always loaded limit size use hashing: name + hash of contents Reduce EMF size derived model -> Xcore model

13 Optimize loading What is resource load? Parse build EMF model build Node model Parallelise Cache parse multiple files simultaneously ~3 time faster loads on 4 core machine EMF synchronization issues serialize EMF and Node model in a cache 3-4 time faster loads

14 Linking Language specific VHDL vs Verilog Avoiding linking expressions that we couldn t handle (yet) library files, only when used in user-code Many iterations lazy linking vs eager linking From 40% of build time to 20% Global indexing Linking Validation Builder Participants

15 Linking Pseudo-algorithm In your IScopePovider Find a econtainer that can be linked eagerly Link everything unresolved links become UnlinkedDeclaration In your DefaultLinkingService getlinkedobjects(...) already resolved? skip it Disable notifications

16 Validation Combine validations to avoid model traversals Local analysis, do global validations moved into builder participant Avoid validation Monitor disabled validations libraries: errors & warnings are suppressed anyway Global indexing Linking Validation Builder Participants

17 Track performance Nightly build log build times

18 UI responsiveness Measuring: detect a blocked UI thread initially Svelto now our own method & logging Eclipse Mars Improvements UI is for drawing only! Make sure everything is cancellable Safeguards certain services should never be executed on the UI thread => check & log

19 Lightweight Editor (fallback) Syntax-highlighting + markers For files > 1 MB Based on ContentTypes extension point

20 Two ContentTypes (based on file size) <extension point="org.eclipse.core.contenttype.contenttypes"> <content-type... describer="com.sigasi...fullvhdlcontentdescriber" name="vhdl editor" <describer class="...fullvhdlcontentdescriber" /> </content-type> <content-type... describer="com.sigasi...lightweightvhdlcontentdescriber" name="lightweight VHDL editor" <describer class="...lightweightvhdlcontentdescriber" /> </content-type> </extension>

21 Future work Continuous process Cache global index info per resource? Linking without node model? Avoid project locking build (Xtext 2.8?)

22 Come talk to us about... Documentation generation Fancy linking algorithms / type systems Graphical views Cross-language support Testing Xtext-plugins Lexical macros Manage large amount of validations...

23 Evaluate the sessions Sign in: