Theia: From Vision to Architecture

Eclipse Theia is an extensible framework for developing fully-fledged multi-language Cloud & Desktop IDE-like products. In a previous post we discussed Theia’s vision1 and in this post we delve deeper into the architectural patterns that inform the design of Theia at every level of the system.

As mentioned, Theia is built to enable the development of personalised multi-language, multi-platform IDEs. To that end, at its core, Theia uses a component-based architecture that prioritises flexibility and extensibility.

In component-based architecture, the design is decomposed into individual functional or logical components that represent well-defined communication interfaces containing methods, events, and properties2. Theia follows this architectural pattern. It has been designed in a very modular and extensible way, and everything in Theia is an extension3. Furthermore, the system’s components are loosely coupled through dependency injection, allowing integrating the many components that already exist or may be developed in the future4. This kind of architecture style means that using Theia, it is easy to deploy and develop specific components without affecting the rest of the system.

The Container View

A container is essentially a boundary inside which some code is executed, or some data is stored, and each container is a separately deployable object or runtime environment5. Theia is mainly made up of five containers including, a frontend, backend, a language server, a debug server, and a file system, shown in the figure below.

Figure: Server Interface Example

The frontend application runs as a single page application, which can be hosted in browsers and in an Electron Browser Window (Chromium)4. The frontend part is responsible for the client’s UI rendering and provides an interface for users to use the IDE built with Theia on a desktop application or browser. The backend runs in Node.js and can be deployed locally or remotely, communicating with the frontend application.

The language server decouples language-related features and functions from the IDE, runs as an independent program, and provides specific implementations of functions such as finding all references. Different language servers provide the backend with specific language support and can also configure according to users' preferences. Debugger servers can be integrated with Theia to extend debug UI and support debug function. Theia can access an external file system to obtain information, such as displaying folders and files in a tree structure.

Developers can change and add extensions to the frontend and backend application to implement specific services and functions, such as adding logic to decide when to launch a specific Language Server. The internal components inside these containers and their connection will be discussed in detail in the following sections.

The Components View

When we zoom into the container’s view, we can see the components in the containers. The component view shows how the containers are made up of a number of components5. In Theia, the frontend, backend, language server, debug server, and file system containers contain various components, as the figure shown below4.

Figure: Component Diagram

There are mainly four types of components:

  • Extension Component: developed by extension developer to extend the IDE’s functions
  • Platform Component: main components for the IDE platform
  • Runtime Component: supporting Theia’s runtime environment
  • External Resource Component: resource from external that Theia needs to use, such as CMake

Since there are so many components in each container, we only take two components in the frontend container as an example. In the frontend, the menu component is a platform component. It allows users to configure, register, and contribute to the main menu and the different context menus4. The Git-Support component is an extension component. It helps the users stage and prepare commits and review the history of changes4.

Theia uses Typescript to build most of the components, and there are some connections between the components. The platform components depend on the runtime components, and the extension components depend on the platform components. Besides, some shared components are deployed in both frontend containers and backend containers, such as the debug component4. The frontend container hosts the Debug UI and frontend Debug API parts, while the backend container hosts the backend Debug API part. The frontend Debug API and backend Debug API use JSON-RPC protocol to communicate.

Figure: Shared Component

The Connectors View

As the component diagram shows, some connectors are used to connect the containers and components.

On the container level, the connector types contain JSON-RPC protocols, REST APIs, Language Server Protocol (LSP), and some debugger protocols.

  • JSON-RPC protocols / REST APIs: JSON-RPC is a remote procedure call protocol encoded in JSON6. It can be built upon the WebSocket7. REST APIs are application programming interfaces (API) that comfort the constraint of REST architectural style8 and enable two parts to communicate over HTTP9. Theia can use JSON-RPC protocols or REST APIs to enable communication between the frontend and the backend.
  • Language Server Protocol (LSP): The LSP defines the protocol used between an editor or IDE and a language server that provides language features like auto-complete, go to definition, find all reference etc10. Theia uses it to connect the language server and the backend.
  • Debugger Protocols: There are a lot of debugger protocols such as Chrome Remote Debugging Protocol 11. Theia uses them to communicate between the debug server and the backend.

On the component level, the components in Theia are all loosely coupled. This is because the developers “believe that a loosely coupled architecture is key to allow integrating the many components that already exist or may be developed in the future. “4 To realize it, Dependency Injection (DI) framework Inversify.js12 is used by Theia to wire up the different components13. Using the DI technique, the components no longer need to create their dependencies by themselves. Instead, Theia uses the DI container to inject the dependencies for each component.

To explain how it works, we borrow an example from Theia’s website directly: “For instance, the Navigator widget needs access to a FileSystem to present folders and files in a tree. With DI the concretion of that FileSystem interface is not important to the Navigator widget. It can safely assume that an object consistent with the FileSystem interface is ready to use. In Theia, the used FileSystem concretion is just a proxy sending JSON-RPC messages to the backend, so it needs a particular configuration and treatment. The navigator doesn’t need to care as it will get injected a fully working FileSystem instance13.”

Development view

Developers are able to build their own extensions or contribute in enhancing the main Theia functionality. Extensive guidelines are given for coding14 and for creating and reviewing a pull request15. Further, Theia’s Developing document16 explains how developers can get started building IDE’s themselves, including example applications that they can run.

Theia’s roots can be found in the theia/package.json file17. The package.json file contains the relevant metadata, dependencies and their versions, scripts, packages and more. It makes the build reproducible, making it easier for other developers to build the project. The root package.json file refers to the Node.js packages of Theia, which are the Theia applications and extensions. These extensions provide a set of widgets, commands, handlers, etc., for a specific functionality18. Each extension is located in its own npm (Node Package Manager), a package manager for Node.js19.

As mentioned before, Theia makes use of the dependency injection framework inversify.js to implement extensions. The main two use cases of the dependency injections are to use a service, or a class, provided by Theia, or to contribute an extension to the platform20.

Figure: Inversify.js

An extension lists one or more dependency injection modules, which binds its contribution implementations to the respective contribution interface18. Each package has its own package.json file, in which the modules are listed. Modules can have one of the following types: frontend or backend for the browser environment, and frontendElectron or backendElectron for the electron environment. Theia uses Electron21 to build their desktop app; by default, the application runs in the browser.

The Runtime View

Since Theia is a framework to build IDEs, you are not running Theia itself, but the applications that are included in its repository. The Theia framework provides many kinds of functionalities, thus the interaction of the components depends on the extensions that the developer is using. However, for any configuration we know that the “dependencies” listed in the package.json file are the required runtime dependencies, whereas the “devDependencies” are dependencies that are only required for developing and testing, but not to run the code.

API Principles

Theia’s API principles are very well-defined; this is because APIs are not only created by core developers but also by all contributors who create Theia Plugins and so consistency is vital. Theia’s developers highlight the following principles as their key guides22:

  • Information Hiding: API should provide the application object model hiding DOM/CSS implementation details behind.
  • Completeness: stateful service should provide accessor functions and events.
  • Extensibility: API should be broken down to minimal interfaces with simple functions since minimal interfaces are easy to implement and new functionality can be developed by composing simple functions.
  • Convenience: provide convenient functions for typical complex tasks, such functions although it should not be complex, but broken down to follow the extensibility principle.
  • Robustness: provide reliable timing.

As for internal communication APIs that connect frontend components and backend components, these are done through well-defined JSON-RPC protocols and REST APIs, as mentioned earlier, and thus follow the principles of RPC and REST; uniform resource identifiers, statelessness, caching, etc.2324.

A note on API versioning and stability: though version management is usually built around API visibility so that when a public API is broken a new major release is required, this is unrealistic for Theia. Since all Theia APIs are more or less public, this approach would only slow down API innovation, accumulate technical debt or require many major releases. Instead, major releases are only required if a stable API is broken, while broken experimental APIs remain allowed in a minor release.

API stability is indicated explicitly by adding @experimental or @stable js-doc tags. Experimental APIs can be graduated to stable via the finalization cycle, which implies a review of the adoption, stability, and documentation of the API25.

/**
 * One does not need any annotations while working on experimental APIs.
 */
export interface ExperimentalInterface {
}
/**
 * @since 0.1.0
 * @stable since 1.0.0
 */
export interface StableInterface {
    /**
     * The same as `StableInterface`.
     */
    stableMethod(): void;
    /**
     * Adding new API to stable API should be explicit.
     *
     * @since 1.1.0
     * @experimental
     */
    experimentalMethod(): void;
}

Conclusions

Overall, we can see clearly how the choice of architectural design is influenced by the attributes of Theia it most champions; flexibility, extensibility.

Flexibility: To support both native desktop and browser solutions with a single source, Theia runs in two separate processes; the frontend and the backend, which are discussed earlier.26

Extensibility: This feature is supported mainly by the Theia API, which is designed to embrace openness and customisablity as much as possible. It accomplishes this by25:

  • allowing the API to default to public visibility for clients,
  • allowing the API to default to protected for extenders,
  • and never using language constructs that prohibit runtime access to internals.

Further, it is supported by the component-based, loosely-coupled design that allows frontend and backend applications to expose one or more extensions, that make contribution easier, including26:

  • Service hooks for other extensions
  • Service implementations of other extension’s service hooks
  • Singleton Services
  • Shared Resources (e.g., CSS)

Of course, no architectural choice is without downsides. The component architecture used at the core of the system, though it supports the aims of flexibility and extensibility, also risks long design phases and delayed project release. This delay can occur since designing for reuse creates a need to carefully balance generalisation and efficacy; while more general components are easier to reuse, they are also less accurate to the environment they are deployed in and can become less efficient27. We cannot accurately judge to what extent these delays have affected the development of Theia, though we can see from the architectural documentations that design choices were very deliberate.

  1. https://2021.desosa.nl/projects/theia/posts/essayone-productvision/ ↩︎

  2. https://www.tutorialspoint.com/software_architecture_design/component_based_architecture.htm ↩︎

  3. https://eclipsesource.com/blogs/2019/10/10/eclipse-theia-extensions-vs-plugins-vs-che-theia-plugins/ ↩︎

  4. https://docs.google.com/document/d/1aodR1LJEF_zu7xBis2MjpHRyv7JKJzW7EWI9XRYCt48/edit#heading=h.be67yp67be5g ↩︎

  5. https://c4model.com/ ↩︎

  6. https://en.wikipedia.org/wiki/JSON-RPC ↩︎

  7. https://en.wikipedia.org/wiki/WebSocket ↩︎

  8. https://www.redhat.com/en/topics/api/what-is-a-rest-api ↩︎

  9. https://www.sitepoint.com/developers-rest-api/ ↩︎

  10. https://microsoft.github.io/language-server-protocol/ ↩︎

  11. https://www.npmjs.com/package/chrome-remote-debug-protocol ↩︎

  12. http://inversify.io/ ↩︎

  13. https://theia-ide.org/docs/services_and_contributions ↩︎

  14. https://github.com/eclipse-theia/theia/wiki/Coding-Guidelines ↩︎

  15. https://github.com/eclipse-theia/theia/blob/master/doc/pull-requests.md ↩︎

  16. https://github.com/eclipse-theia/theia/blob/master/doc/Developing.md ↩︎

  17. https://github.com/eclipse-theia/theia/blob/master/package.json ↩︎

  18. https://theia-ide.org/docs/authoring_extensions/ ↩︎

  19. https://www.npmjs.com/ ↩︎

  20. https://eclipsesource.com/blogs/2018/11/28/how-to-inversify-in-eclipse-theia/ ↩︎

  21. https://www.electronjs.org/ ↩︎

  22. https://github.com/eclipse-theia/theia/blob/master/doc/api-testing.md ↩︎

  23. https://medium.com/future-vision/the-principles-of-rest-6b00deac91b3 ↩︎

  24. https://wiki.eclipse.org/Orion/Server_API ↩︎

  25. https://github.com/eclipse-theia/theia/blob/master/doc/api-management.md ↩︎

  26. https://theia-ide.org/docs/architecture ↩︎

  27. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.20.7079&rep=rep1&type=pdf ↩︎