DESOSA 2021

openHAB - Quality and Evolution

openHAB: Quality and Evolution

Table of Contents

  1. Software Quality Processes
  2. Continuous Integration Processes
  3. Test Processes
  4. Hotspot Components
  5. Quality Culture
  6. Technical Debt

Software Quality Processes

The health care system is full of fixed processes that are in place to ensure patient safety. A patient is tested for a penicillin allergy before taking the antibiotic. A pre-anaesthesia checkup is in place also to test the sedation before an operation. Similarly, systems have software quality (assurance) processes to ensure that the system meets specific quality objectives before releasing the code into the world. Within openHAB, the following process is used to ensure software quality.

Any developer should adhere to the coding guidelines, which apply to all Java code inside the project and aim for easy readability and maintainability. These guidelines are used as a checklist for code reviews on a pull request. Some of the topics are the structure of a typical OSGi bundle project, code format for Java and XML files, JavaDoc, and logging usage patterns.

openHAB has devoted a repository1 to static code analysis using Maven plugins for FindBugs, Checkstyle and PMD (sometimes referred to as ‘Programming Mistake Detector’). This tool tweaked to openHAB’s needs. FindBugs detects and ranks possible bugs. Checkstyle creates a report regarding the code style. PMD finds frequent flaws such unused variables. This tool is available in a separate repository that generates an HTML report.

openHAB has a Jenkins server that is used for Continuous Integration. It is used to make sure all repositories are able to be built and pass all the necessary tests2. The next section goes more into detail.

openHAB uses jFrog to optimize processes within the CI pipeline and stores the binary output of the build data. In addition, jFrog provides automation and standardization across the repositories which can lead to a faster workflow.

The community highly appreciates pull requests, but also has a process in place to ensure quality. Changes can only be merged after an approval by one of the openHAB maintainers3. Maintainers focus on a specific code repositories and every repository has multiple maintainers. Apart from reviewing PRs they also define the code guidelines previously mentioned, decide the code’s architecture and features, and more. To become a maintainer, one should be heavily involved, be nominated by another maintainer, and receive at least three positive votes in the election4.

Continuous Integration Processes

Most of openHAB’s development and deployment processes are integrated into the GitHub organization, using GitHub Actions. As most openHAB’s containers have their own repository, they all have their own Continuous Integration Processes, even though most of their integrations are shared.

openHAB’s primary development integrations are based on Jenkins5, an open-source automation server. The Jenkins instance is publicly available 6. Each container has its own Jenkins project. Here the instance lints, builds, and tests the complete openHAB source on a schedule and whenever a commit or merge request has been pushed. On top of Jenkins, openHAB uses relativeCI for quality control of their web bundles.

Besides the primary CI, openHAB uses Crowdin7 for their internationalization process. Crowdin has automated this process and automatically creates merge requests using the openHAB-Bot, whenever language updates are available.

Test Processes

openHAB requires code to be well tested and has several test processes where they apply Unit tests and integration tests8. The Unit tests are applied to test individual components, while the integration tests test the combination of multiple software modules. For both the core system (openhab-core) and add-ons (openhab-addons), the tests are stored in a folder called itests. Writing tests for every line of code is not mandatory in openHAB, but is highly encouraged.

Figure: Test folders (screenshot from GitHub)

Mockito, an open source mocking framework, is used for the Unit tests9. openHAB’s documentation gives an example of how this can be applied8:

public class MyBindingHandlerTest {

    private ThingHandler handler;

    private @Mock ThingHandlerCallback callback;
    private @Mock Thing thing;

    @Before
    public void setUp() {
        initMocks(this);
        handler = new MyBindingHandler(thing);
        handler.setCallback(callback);
    }

    @After
    public void tearDown() {
        // Free any resources, like open database connections, files etc.
        handler.dispose();
    }

    @Test
    public void initializeShouldCallTheCallback() {
        // we expect the handler#initialize method to call the callback during execution and
        // pass it the thing and a ThingStatusInfo object containing the ThingStatus of the thing.
        handler.initialize();

        // verify the interaction with the callback.
        // Check that the ThingStatusInfo given as second parameter to the callback was build with the ONLINE status:
        verify(callback).statusUpdated(eq(thing), argThat(arg -> arg.getStatus().equals(ThingStatus.ONLINE)));
    }

}

Hamcrest assertions can be used to assist in writing Unit tests. It allows the tester to quickly create complex assertion matchers, instead of being stuck to assert(x == y); for instance. A few examples of Hamcrest assertions are stated below.

assertThat("myString", is(instanceOf(String.class)));
assertThat(Arrays.asList("one", "two"), hasItem("two"));
assertThat(new Object(), is(not(nullValue())));

Some components use the OSGi core services, which makes it hard to test them. Luckily, by using integration tests test classes which extend JavaOSGiTest can be run inside an OSGi runtime. Note that these test have a long run time, which should be taken into account.

Hotspot Components

Hotspot components are components that are often changed. These types of components are usually the most critical part of the system. Since openHAB does not have any known roadmap, we, unfortunately, cannot say much about the future’s hotspot components. We can, however, say something about which components have been the hotspot components in the past.

The two most active repositories for openHAB are openhab-core and openhab-addons. We used git effort to find out what are the most actively developed components. This command shows the number of commits and days worked on for all files in the repository. These are the topmost actively developed components for both repositories..

openhab-core

File Commits Active Days Description
FeatureInstaller.java 45 42 This services reads the addons.cfg and installs listed add-ons (Karaf features) and selected packages. It also allows for configuration of the base package through the UI.
RuleEngineImpl.java 32 31 This class is responsible for initializing and executing Rules when they are added in the rule engine.
ScriptFileWatcher.java 22 21 The ScriptFileWatcher watches the jsr223 directory for files. If a new or modified file is detected, the script is read and passed to the ScriptEngineManager.
RuleResource.java 20 20 This class acts as a REST resource for rules.
AbstractResourceBundleProvider.java 18 18 This is a baseclass for ModuleTypeProvider, TemplateProvider and RuleImporter.
ThingResource.java 18 17 This class acts as a REST resource for things and is registered with the Jersey servlet
ScriptEngineManagerImpl.java 17 17 This class provides the right script engine depending on the script type.
ItemChannelLinkResource.java 17 16 This class acts as a REST resource for links.
ItemUIRegistryImpl.java 17 16 This class allows a consumer to easily call a method for all available items across multiple different item providers without needing to itterate over them manually.
AbstractFileProvider.java 16 16 This is the baseclass for ModuleTypeProvider and TemplateProvider and is responsible for importing the automation objects from the local filesystem.

openhab-addons

For this repository we take a look at which bindings as a whole received the most commits, since that give a better representation of their importance than separate files give.

Binding Commits Active Days Description
org.openhab.binding.miio 40 33 This binding is used to control Xiaomi products using the Mi IO protocol.
org.openhab.binding.remoteopenhab 16 13 This binding allows bidirectional communication between two openHAB instances.
org.openhab.binding.amazonechocontrol 15 14 This binding allows control of Amazon Echo devices (Alexa).
org.openhab.binding.deconz 15 12 Specific binding which allows openHAB to used Dresden Elektronik ZigBee dongles.
org.openhab.binding.icalendar 12 11 ICalendar implementation for openHAB.
org.openhab.binding.sonos 11 10 This binding integrates the Sonos Multi-Room Audio system.
org.openhab.binding.http 10 9 This binding can do HTTP calls.
org.openhab.binding.linky 10 10 This binding uses the API provided by Enedis to retrieve your energy consumption data.
org.openhab.binding.ipcamera 9 9 This binding allows you to use most IP cameras in openHAB.

Quality Culture

Most changes that come to openHAB start out as an issue. Creators of such issues can choose to add labels to their issue to identify them easier. In the repositories openhab-addons and openhab-webui, the issues are clearly labeled. However, in the openhab-core and openhab-distro many do not go for this approach, and as a result the list of issues is a list of titles of the problem they would like to have fixed, see the issues page.

For a pull request (PR), all repositories use the same policies. A (future) contributor is expected to clearly explain the context of their contribution and sign off their work in their commit messages. Testing your code is encouraged, however we see that many pull requests do not test their pull requests (PR#2133, PR#1814).

A pull request can only be merged when maintainers of openHAB have accepted the pull request. They do this using LGTM (Looks Good To Me) in the comments of the merge request. Dependent on the repository the maintainers might differ, there needs to be an accept from the absolute majority of maintainers on both repositories.

Most issues identify a problem someone has experienced when using openHAB. These issues get explored by contributors and then either get resolved (issue#2216) or eventually converted to a pull request to fix the issue (issue#2200 to PR#2212). Some of the issues address test cases that fail on certain systems or are unstable (issue#1955, issue#1069). What stands out is that these issues are all posted by top contributors and resolved within a few weeks(PR#2093,PR#1148).

Very few PRs actually have in improvement in the code quality or optimization (PR#1068) and are mostly fixated on additions to the system , bug fixes or updating dependencies (PR#1216 PR#1242, PR#1245).

The main focus for most contributors is through the openhab-addons repository. This is the place where extra bindings to systems are implemented to allow for the widest range of connected components allowed with openHAB. The issues and PRs in this repository add new bindings, fix an issue with bindings or add functionality to a binding (bindings are the connection with the system and IoT devices) (PR #10277, #10355, #10302).

Technical Debt

Technical Debt is a popular concept which came into being when Ward Cunningham used it as a metaphor to explain motivations for refactoring in 199210. The term covers the costs of additional work in future when choosing a faster and cheaper short-term solution now. The image below perfectly illustrates this11. Although technical debt is hard to quantify, this section considers multiple aspects of it to at least get an idea of the current status.

Figure: much too busy

Bus Factor

Unlike the term technical debt which came from a financial metaphor, the term bus factor is a bit more to the point. It represents the number of people who would have to drop out, or more subtly ‘be run over by a bus’, before the project would be in hazard.

To analyze this factor, we take a look at core system and leave out the rest including applications and add-ons since these are less vital. The following graph shows the contributors who have had over 20 commits in total between 2016 and present day (March 22, 2021).

Figure: Contributions over the years (own figure)

As we can see, there are just seven contributions who meet this definition of active. Some of these contributors have only been active in a particular year. Especially between 2016 and 2018 openHAB’s founder Kai Kreuzer was the only active contributor on the openhab-core repository. After that it improved slightly, however, the number of active contributors is still low. Looking at 2019 and 2020, there are just four contributors with over 20 commits within that year. Therefore, the bus factor is four for the core system.

Code Quality Analysis

As mentioned before, openHAB also comes with a static code analysis repository1. This generates a report of possible bugs in the code of the openhab-core.

Category Tool Priority rule Count
bad practice
findbugs 3 Logger should be non-static field (SLF4J) 15
best practices
pmd 2 SystemPrintln 3
design
pmd 2 AvoidThrowingRawExceptionTypes 35
pmd 3 SimplifyBooleanExpressions 3
error prone
pmd 2 AvoidCatchingNPE 1
pmd 2 AvoidInstanceofChecksInCatchClause 2
pmd 2 DoNotThrowExceptionInFinally 1
pmd 3 CompareObjectsWithEquals 9
pmd 3 EmptyIfStmt 3
pmd 3 MoreThanOneLogger 2
pmd 3 EmptyWhileStmt 1
performance
pmd 3 AvoidArrayLoops 1
style
checkstyle 2 NullAnnotationsCheck 815
checkstyle 2 IllegalThrowsCheck 5
checkstyle 2 OhInfXmlUsageCheck 24
checkstyle 2 AvoidScheduleAtFixedRateCheck 2
checkstyle 2 AuthorContributionDescriptionCheck 2
checkstyle 2 ForbiddenPackageUsageCheck 1
checkstyle 3 TodoCommentCheck 6
checkstyle 3 ModifierOrderCheck 10
checkstyle 3 ConstantNameCheck 5
checkstyle 3 MethodNameCheck 8
checkstyle 3 NoEmptyLineSeparatorCheck 15
checkstyle 3 JvadocMethodStyleCheck 1
checkstyle 3 MemberNameCheck 37
checkstyle 3 StaticVariableNameCheck 9
total 1016

Reducing Technical Debt: 1.x Compatibility Layer

By releasing openHAB 3.0 much technical debt that originated from the second version was paid. openHAB 1 was just a clean and simple architecture. The second version become more complicated as comprises were introduced between ‘old’ first version and the new UI features from Eclipse SmartHome12. Bindings and rules worked quite differently in both approaches, but they did have to work together. As a result a 1.x compatibility layer was introduced13. openHAB 3.0 removed this unnecessary layer all at once14.

References


  1. Source: https://github.com/openhab/static-code-analysis ↩︎

  2. Source: https://community.openhab.org/t/new-openhab-build-server-jenkins/59824 ↩︎

  3. Source: https://www.openhab.org/docs/developer/contributing.html ↩︎

  4. Source: https://www.openhab.org/docs/developer/governance.html#maintainers ↩︎

  5. Source: https://www.jenkins.io/ ↩︎

  6. Source: https://ci.openhab.org/ ↩︎

  7. Source: https://crowdin.com ↩︎

  8. Source: https://www.openhab.org/docs/developer/tests.html ↩︎

  9. Source: https://github.com/mockito/mockito ↩︎

  10. Source: https://www.youtube.com/watch?v=pqeJFYwnkjE ↩︎

  11. Source: https://pbs.twimg.com/media/DTvOdk9VMAAwBQ5.png ↩︎

  12. Source: https://projects.eclipse.org/projects/iot.smarthome ↩︎

  13. Source: https://github.com/adarshlal/openhab2/blob/master/docs/sources/installation/compatibilitylayer.md ↩︎

  14. Source: https://www.openhab.org/blog/2020-12-21-openhab-3-0-release.html ↩︎

openHAB
Authors
Matthijs Wisboom
Sterre Noorthoek
Tom Hoefnagel
Nathan Prins