Theia and Variability

In previous posts, we focused on Theia’s product vision, architecture, and quality and evolution¹²³. In this post, we would like to introduce another essential aspect of Theia: variability. Since Theia is designed to support users in creating customized IDEs and using them on various platforms, it follows that Theia is strongly variable in nature. In the following essay, we will first identify Theia’s variable features and their incompatibilities, and then build feature models. Subsequently, we will analyse the methods of variability management. Finally, we will discuss Theia’s variability implementation approach and potential future development.

Variability Modelling

We first identify the main variability features of Theia, which focus on two aspects: platform and software.

Platform

Theia can be deployed on multiple platforms; it can work as a native application as well as in the context of a browser. It also supports multiple browsers, including Chrome, Firefox, Edge, Internet Explorer, and Safari. Further, it can run on multiple operating systems, including Linux, macOS, and Windows. Therefore, Theia can adapt to different application scenarios and meet users' needs with different equipment and development environment preferences.

The following feature model shows the platforms supported by Theia.

Software

As an extensible framework supporting personalized building IDE, Theia has much variability at the software level.

Theia is composed of extensions, which provide a set of widgets, commands, and handlers for specific functionality. Theia ships a number of extensions, such as editors and terminals. Users can specify the extensions needed to create their own IDE by writing configuration files. Users can also add specific functions to their IDE by developing custom extensions. Plugins are another extensible model that can be loaded at runtime and allows users to customize the IDE without learning any framework.

Theia supports building multi-language IDE as it supports a series of programming languages, such as C, C++, Java, Python, TypeScript. Users can also add new grammars for specific languages⁴. Correspondingly, Theia offers an extendable framework for debugger UI and external debugger protocol integration to support all kinds of debuggers⁵.

Theia defines many commands to execute specific functions. Users can also add commands to Theia command registry and add key bindings and contexts to these commands so that they can only be called under certain conditions⁶.

Theia has a preference service that allows modules to get preference values, contribute default preferences and listen for preference changes⁷. Users can adjust the UI interface according to their own preferences, such as theme colour and font size. Theia’s layout is also variable, allowing for splitting views and laying them out using drag and drop⁵.

The following feature model represents Theia’s functionality variable features.

Incompatibility

Inevitably, Theia’s many variable features cause some incompatibilities.

We analysed the issues in Theia’s repository⁸ and found that incompatibility is most often related to the variability of the platform. Some functions are not available in specific operating systems or web browsers. For example, preference editing is generally broken for Windows users⁹ but works on other operating systems; the forward-slash “/” does not display in the terminal when running Theia using Edge¹⁰.

Another common cause of incompatibility is application version conflicts. For example, a specific function depends on a specific version of an application extension, but this may be inconsistent with Theia components' versions¹¹. Therefore, as a general rule, Theia’s development team considers updating application versions to always be an improvement to the system quality¹².

Lastly, some problems occur when the system is running a specific language server. For example, the back-end stops after running the JAVA language server¹³. Some modules react differently while the state of an unrelated module changes, for example, the open recent workspace command is broken if the editor focuses¹⁴.

Management

As described in our first blogpost¹, the end users of Theia are software developers, using Theia to build their own customized IDE. The main information source to get started is the developing guide ¹⁵, where developer instructions are given for the three supported operating systems, Windows, macOS, and Linux. The guide explains how developers can build and debug one of the example Theia projects in any of the compatible browsers or on their desktop.

As shown in the functionality feature model, users can adjust the layout of their IDE, preferences, the language that they want to work with etc., but the main changes are made through extensions and plugins. When users want to add extensions to their custom IDE, they need to add the corresponding dependency to the package.json file in the root folder of their project. Developers can either add existing extensions from the node package registry or develop their own extension and add it locally. To know which extensions are currently available, one of the core developers recommended looking at the example project to view the available extensions ¹⁶. An overview can also be found on the Theia Typedoc ¹⁷. However, clearly documenting the available extensions is still a matter that Theia is working on, as they do not all contain descriptions of their functionality. The core developers aim to have a central location for describing extensions, their commands, preferences, and their functionalities, and believe a good place for that would be the main website ¹⁶.

Another way developers are able to add functionality to their IDE is by adjusting the plugins they are using. The main difference between extensions and plugins is that extensions are more powerful and targeted at build time extensibility, while plugins are isolated, targeted at runtime extensibility ¹⁸. Sources that users can consult for the creation or addition of plugins are amongst others the Theia documentation ¹⁹ and the eclipse source blog posts ²⁰.

When a new release is launched and developers update to the newest version, the source of information they can rely on is the changelog, where all the (breaking) changes are listed with the corresponding issue number, if any.

When developers push code to the repository or want to merge their new or adjusted code, they are required to test the changes and explain how they have tested them. However, they do not explicitly test their new code for all available platforms and possible extension or plugin combinations. Instead, to ease variability management, there are automated tests covering each of the three operating systems to check whether the code is still functioning as it should. Besides, the different operating systems, different versions of Python and Node.js are covered as well. Unfortunately, these automated tests do not cover everything, and the developers still run into configuration problems, as we mentioned in the incompatibility section.

Implementation

Naturally, such large-scale variability requires specialised implementation mechanisms. We investigate this in relation to the two most variable areas of Theia discussed earlier: platform and software.

There are few techniques used throughout Theia that allow developers the breadth of products we see; largely, these are conditional compilation, configuration, and design patterns.

Conditional compilation, in which variation is introduced at compile time based on the desired end-product configuration²¹. It is often achieved using C pre-processors²², which we see employed in Theia to adapt to the product’s current operating system²³.

#ifndef LINUX_FFMPEG
#define LINUX_FFMPEG

#include "ffmpeg.h"

char *load_ffmpeg_library(struct FFMPEG_Library *library, char *library_path){
    // edited for space  
}

char *unload_ffmpeg_library(struct FFMPEG_Library *library){
    // edited for space 
}

#endif // LINUX_FFMPEG guard

As for configuration, it is used throughout the project, both to support platform variability and software variability. We consider here the example of Theia’s Task extension, which enables the execution of scripts or binaries in the application’s backend²⁴. We can see in the snippets below the application’s properties changing at load time based on the operating system Theia is running on²⁵. This setup allows the configuration of the Task extension even across different workplaces.²⁴

/**
     * Perform some adjustments to the task launch configuration, before sending
     * it to the backend to be executed. We can make sure that parameters that
     * are optional to the user but required by the server will be defined, with
     * sane default values. Also, resolve all known variables, e.g. `${workspaceFolder}`.
*/
const result: ProcessTaskConfiguration = {
     ...processTaskConfig,
     command: await this.variableResolverService.resolve(processTaskConfig.command, variableResolverOptions),
     args: processTaskConfig.args ? await this.variableResolverService.resolveArray(processTaskConfig.args, variableResolverOptions) : undefined,
     windows: processTaskConfig.windows ? {
           command: await this.variableResolverService.resolve(processTaskConfig.windows.command, variableResolverOptions),
           args: processTaskConfig.windows.args ? await this.variableResolverService.resolveArray(processTaskConfig.windows.args, variableResolverOptions) : undefined,
           options: processTaskConfig.windows.options
     } : undefined,
     osx: processTaskConfig.osx ? {
            command: await this.variableResolverService.resolve(processTaskConfig.osx.command, variableResolverOptions),
            args: processTaskConfig.osx.args ? await this.variableResolverService.resolveArray(processTaskConfig.osx.args, variableResolverOptions) : undefined,
            options: processTaskConfig.osx.options
     } : undefined,
     linux: processTaskConfig.linux ? {
            command: await this.variableResolverService.resolve(processTaskConfig.linux.command, variableResolverOptions),
            args: processTaskConfig.linux.args ? await this.variableResolverService.resolveArray(processTaskConfig.linux.args, variableResolverOptions) : undefined,
            options: processTaskConfig.linux.options
     } : undefined,
     options: {
            cwd: await this.variableResolverService.resolve(cwd, variableResolverOptions),
            env: processTaskConfig.options && processTaskConfig.options.env,
            shell: processTaskConfig.options && processTaskConfig.options.shell
     }
};

{
    "label": "Test task - list workspace files recursively",
    "type": "shell",
    "command": "ls",
    "args": [
        "-alR"
    ],
    "options": {
        "cwd": "${workspaceFolder}"
    },
    "windows": {
        "command": "cmd.exe",
        "args": [
            "/c",
            "dir",
            "/s"
        ]
    }
}

We also consider the example of package.json files used by yarn²⁶, where developers can control how Theia is built by including or excluding lines of code. For example, a set of plugins can be included in the Theia build²⁷.

"theiaPlugins": {
    "vscode-builtin-bat": "https://open-vsx.org/api/vscode/bat/1.45.1/file/vscode.bat-1.45.1.vsix",
    "vscode-builtin-clojure": "https://open-vsx.org/api/vscode/clojure/1.45.1/file/vscode.clojure-1.45.1.vsix",
    "vscode-builtin-coffeescript": "https://open-vsx.org/api/vscode/coffeescript/1.45.1/file/vscode.coffeescript-1.45.1.vsix",
    "vscode-builtin-configuration-editing": "https://open-vsx.org/api/vscode/configuration-editing/1.45.1/file/vscode.configuration-editing-1.45.1.vsix",
    "vscode-builtin-cpp": "https://open-vsx.org/api/vscode/cpp/1.45.1/file/vscode.cpp-1.45.1.vsix",
    "vscode-builtin-csharp": "https://open-vsx.org/api/vscode/csharp/1.45.1/file/vscode.csharp-1.45.1.vsix",
    "vscode-builtin-css": "https://open-vsx.org/api/vscode/css/1.45.1/file/vscode.css-1.45.1.vsix",
    "vscode-builtin-css-language-features": "https://open-vsx.org/api/vscode/css-language-features/1.45.1/file/vscode.css-language-features-1.45.1.vsix",
    "vscode-builtin-debug-auto-launch": "https://open-vsx.org/api/vscode/debug-auto-launch/1.45.1/file/vscode.debug-auto-launch-1.45.1.vsix",
    "vscode-builtin-docker": "https://open-vsx.org/api/vscode/docker/1.45.1/file/vscode.docker-1.45.1.vsix"
    }

Lastly, the design pattern at the core of Theia, component-based architecture, is specifically designed to support modularisation and reuse²⁸, and so works very well in this context, allowing contributors easy design-time binding of variability when contributing to the project. Design-time binding is an example of the ‘clone and own’ approach to contribution where new products are created by modifying existing ones²⁹. For more discussion of Theia’s architecture choices, see our previous post².

Future Development

Possible future developments of Theia have been discussed in our first post¹ and highlighted ongoing work to a) provide a Theia-as-development-tool product, and b) continue to implement parts of the VS Code API that are currently missing. In both cases, developers will have to manage an increased number of variants of Theia.

While the proposed development tool will most likely have a limited number of variants of itself in the initial stages, it is still, in itself, another configuration of Theia to be managed and thus increases the variability management workload.

As for the incoming VS Code plugins, this raises some concerns in terms of implementation scalability. Though the architecture choices of Theia make it easy to create and introduce new extensions, the use of configuration files to, at load time, control the possible combinations of said plugins is significantly more cumbersome. The root package.json file is already at 156 lines, and leaves too much space for human error²⁷. Increasing the number of plugins to be tracked in these files will very difficult.

One possible remedy is to move away from configuration files in favour of relying more on the conditional compilation, which is only rarely used. Conditional compilation places more of the responsibility for correctness on Theia’s core developers and less on casual contributors and possible future end users, as well as enabling easier automated testing of configuration correctness.

Conclusions

It is easy to see now that Theia is a highly adaptable, variable project with many possible end product combinations that will only expand with time, thanks to its encouragement of new contributions. However, based on our discussion, it is quite possible Theia developers will need to find new ways to manage and implement variability in their project in the near future if they continue to expand at their current rate. However, at the moment, their use of modular design patterns and sound variability management techniques places them on a good track for future development.