InteliJ - Quality and Evolution

IntelliJ IDEA is an open source IDE developed by JetBrains, it strives to be the best Java IDE.

Overview

In this blog we will discuss the various aspect of the quality and evolution of IntelliJ IDEA. First, we will describe the overall software quality process, then the CI/CD and tests. After that, we will discuss the code hotspots and their code quality, the culture surrounding code quality and finally, we will discuss the technical debt of the project.

Overall software quality process

Software quality can be split into two subsections: code quality (how it’s written) and code robustness (the logic). The overall code quality is managed through the PR’s. Here, any code changes are audited and feedback is provided where the code quality is not sufficient. Moreover, the code robustness is managed in the pipeline, where it’s tested to verify functionality.

However, the IntelliJ project does not strictly enforce any particular guidelines, aside from some basic formatting rules¹. They do recommend that the contributors follow the practices from “Effective Java” from Joshua Bloch. Moreover, there are no enforced rules regarding the implementation of tests¹.

CI/CD

Given that IntelliJ IDEA is a standalone application, their CI/CD does not include any deployment or automatic releasing. Instead, we suppose that IntelliJ releases a new version of their application manually which includes compiling the code and uploading it to the JetBrains repository. Nonetheless, the GitHub repository does include a continuous integration pipeline. This pipeline is run using Qodana, this is a JetBrains developed static analysis and quality management tool currently still in early access. Qodana can be used in different ways, including a cloud service, which is how it is used with IntelliJ IDEA². Qodana provides the IntelliJ IDEA engine used as a linter and static analysis tool together with an interactive and clean UI.

For IntelliJ IDEA, Qodana is configured with 180 inspections. This blog post will not be discussing every inspection, but they can be found here. It is worth noting that all the inspections which are run on Qodana are also run by the IntelliJ IDEA inspection tool locally whilst programming. This allows for easier and quicker feedback on code quality.

Tests

As described in the previous post, JetBrains has created their own testing platform. The current IntelliJ codebase contains mainly model level functional tests. These are tests which run in a headless environment which uses real production test cases. These tests usually test features as a whole rather then an individual function. IntelliJ has decided to only test their underlying model and not their UI³. The IntelliJ code guidelines suggest that tests must be run for any modified part of the code and these changes cannot introduce test failures. Furthermore, they also suggest to add new functional tests for any bug which is resolved or any new feature which is added⁴.

The IntelliJ coding guidelines do not mention any preferences or guidelines regarding test coverage. Furthermore, any opened Pull Request does not report any (changed) test coverage. This leads us to believe that JetBrains is not significantly worried about test coverage, but rather evaluate the need for it case by case.

Hotspots

Of the ten most edited files in the IntelliJ IDEA Community Edition GitHub repository, only three are java files. The remaining seven are xml and properties files. The three java files in question are:

AllIcons.java, this file is an auto-generated file responsible for loading all of the icons.
FileBasedIndexImpl.java, this class is the implementation of the file based indexing system.
EditorImpl.java, this class is the implementation of the editor system. Future hotspots will most likely be all over the repository as JetBrains will be focusing more on small quality of life changes rather than large functionality additions.

Code quality of hotspots

Generally speaking, classes in the IntelliJ IDEA CE conform to a reasonable code quality standard as contributors have to follow certain guidelines¹. However, documentation is often lacking. This is particularly noticeable in two of the three hotspots mentioned in the previous section. Both FileBasedIndexImpl.java and EditorImpl.java don’t seem particularly complex, but are extremely large and carry the responsibility of implementation two very extensive systems. Neither of the two have comments, making them difficult to understand without putting in considerable effort. The AllIcons.java file does explain that it is not supposed to be read, and only auto-generated. This is not a very elegant solution, but does allow for easy addition of more icons.

Code quality culture

The IntelliJ project is an open source project receiving dozens of commits on a daily basis⁵. With currently 133 open, 1447 total PR’s and 214,674 issues⁶, a proper code quality culture can go a long way in terms of maintainability and ease-of-use.

For this reason the project has an exhaustive document detailing the IntelliJ platform SDK⁷, with examples on how to implement plugins, tests and other features. Unfortunately, this document is not complete yet, with sections like ‘Component Model’ and ‘Diagrams’ still grayed out.

As mentioned previously, the code quality is managed in the PR’s. When browsing through closed PR’s⁸, one can see that some of them are never merged because the quality of the solution is not sufficient for the project⁹. Or there is some discussion about how to apply a certain fix before continuing¹⁰,¹¹,¹². One can find a number of great examples by searching for closed PR’s with requested changes, sorted by the number of comments¹³.

Generally, the code culture is positive, with swift feedback and polite discussion between the developers.

technical debt assessment

The IntelliJ platform is designed such that adding new functionality is easily done through plugins, using the APIs. However, non-API files often lack documentation. As a result, the main source of technical debt is the lack of documentation in files that are not part of the APIs. This makes it more difficult to edit existing files when necessary.