Stellar - From Vision to Architecture

Figure: Stellar Logo

Last week we examined Stellar and its vision on banking. The underlying architecture used to design the Stellar platform supports this vision. This post will look into this architecture and how it relates to Stellar’s product vision. More specifically, we will discuss the individual components and their relations and how design decisions impact operations.

Architectural Style

Above all, Stellar is a decentralised and distributed architecture. The Stellar-Core is a peer-to-peer network of nodes running independently, having independent data storage capabilities, connected and broadcasting transactions on the network. Each node validates blocks of transactions on the validity of consensus reached with the Stellar Consensus Protocol, further explained in the overview of main core concepts. This pattern is also known as blockchain architecture.

However, one can also see Stellar as a two-layered architecture, of which the bottom layer is the Stellar-Core, the peer-to-peer network of nodes that validate the transactions, and the top layer is Horizon. Application developers can use the Application Programmable Interface (API) in Horizon to interact with the Stellar-core.

Container View

The Stellar platform consists of three major containers. On the top level, there is the app itself with which users interact. The Stellar project does not create or maintain any of these applications, as these are related yet separated systems created by the community. For example, companies can develop these systems to offer a service. Other open-source projects by the community also help to create a way to interact with Stellar.

Figure: Container view

The app communicates with the next container, which is Horizon. Horizon is an API that sits between Stellar Core and the top-level apps. It acts as a bridge to let people interact with the system without knowing the Stellar internals in detail. For many, this is the way to interact with the Stellar Core. It is even possible to connect to Horizon directly through public horizon servers.

These Horizon servers, in turn, connect with the Stellar Core. These are the servers at the heart of Stellar. They maintain the ledgers and agree or disagree on transactions through the Stellar Consensus Protocol (SCP). Since Stellar is an open-source project, anyone can run Stellar Core. Nevertheless, due to the way SCP works, nodes in the network need to trust each other; thus, if one were to start one’s own core node, it likely would not be trusted by others immediately. In the Core, there are also order books, which contain outstanding orders between different assets. When a user requests a transfer to a different asset, the Stellar network will search for a path through this decentralised exchange (DEX) and navigate the best route.

Overview of Main Core Components and Their Interactions

The first component we will discuss is the Stellar Consensus Protocol (SCP) subsystem, an abstract implementation of SCP. Modules wishing to use the protocol for consensus should subclass its driver class. They should provide a specific interpretation for the ‘slot’ and ‘value’ used in the Byzantine-Agreement-based protocol 123.

In Stellar Core, the HerderSCPDriver, or Herder, provides the interpretation used for Stellar and connects to Overlay and Ledger subsystems 1. Herder is the component that receives and processes messages on agreements to make and operates in terms of ledger numbers and transaction sets, which reflect the ‘slot’ and ‘value’ for the Byzantine-Agreement 4. It implements the methods which SCP uses to exchange information with peers 5.

The aforementioned Ledger submodule applies transaction sets externalised by the HerderSCPDriver and handles transaction externalisation events. To do this, it communicates updated ledger entries to the BucketList, History, and Overlay subsystems. It also communicates with the History system the other way around, enabling information exchange from the History subsystem to update the node 5. The ledger is a state representation of the Stellar universe at any given time. The SCP mentioned before works in rounds, after each of which the system applies the decided-upon transaction to the last closed ledger, which defines the new last closed ledger 6. Another notable design choice for the ledgers is that each ledger always points back to the previous ledger in the sequence, which allows for further validation steps.

The Overlay subsystem manages connections with peer nodes via a virtual “broadcast network” of peer-to-peer TCP connections. It enables connecting to peers and floods messages and fetches. It is the exception to exclusive communications through the Ledger subsystem, allowing for direct communication between nodes. Its primary use is for the SCP-node consensus-reaching tasks. The subsystem implements various mechanisms for effective distribution and routing of messages between peers. It also allows for the propagation of API-injected transactions 57.

The BucketList communicates with the Ledger, has the main tasks of locally storing ledger entries, and hashing and deduplicating buckets. The buckets are a redundant store of information of ledger entries, storing entries based on age, This mechanism allows for storing recent entries in smaller, lower-level buckets, which aids efficient handling of changes in both the syncing process and the cryptographic hashing process when combined with other optimisations relying on the local hashed ledger and history 83.

The final of the significant components of the Stellar Core, History, handles off-site archiving of transaction and ledger entries for auditing and updating of other nodes. As mentioned in the Ledger section, this component can also interact with the Ledger to update this node if it has fallen behind the rest of the network 59.

Development and Runtime

Stellar Core

Stellar Core is the reference implementation for the peer-to-peer agent that manages the Stellar network written in C++14 and runs on Windows, OSX and Linux. It is a replicated state machine that maintains a local copy of a cryptographic ledger and processes transactions against it. It does this by creating consensus with a set of peers in accordance with the Stellar Consensus Protocol.10 There are many runtime dependencies such as c++, pkg-config, bison and more. Alternatively, one can install it with one command by using the pre-configured docker container 11.

Horizon

Horizon is the client-facing HTTP API server for the Stellar ecosystem, written in Golang. It acts as the interface between Stellar Core and applications that want to access the Stellar network. Actions that Horizon provide include submitting transactions to the network, checking accounts' status, and subscribing to event streams, amongst others. Horizon ingests and re-serves the data produced by the Stellar network in a form that is easier to consume than the performance-oriented data representations used in Stellar Core. 12 One can access Horizon through cURL, a browser, any HTTP request tool such as Postman 13 or Insomnia 14 and through one of the many Stellar SDKs. 15 Horizon’s only runtime dependency is Golang; other dependencies automatically install upon executing the first go command. Horizon has support for all major operating systems.

API Design Principles

The API design of Stellar Horizon builds on the RESTful API standard. 16 Its responses are formatted using the HAL format 17.

Combining these two standards means that the API has many features, such as easy navigation and exploration through linking in the responses, paginated responses, and connection between parent and child resources.

Furthermore, passing the “Accept: text/event-stream” header makes it possible to access the data in an event stream format, making it easier to follow and respond to new transactions coming in over the network. This streaming is compatible with the HTML Server-Sent Events API format, and access is possible through the EventSource interface of JavaScript 18 19

Realisation of Key Quality Attributes

Stellar makes use of blockchain architecture to realise the key quality attributes of safety and reliability. It uses a specifically designed consensus protocol, the Stellar Consensus protocol, to ensure low transaction costs and high transaction speed. On top of that, it uses a unique tokenisation system to increase usability and thus the adoption of the system. Finally, by choosing a layered structure with the Stellar-core at the bottom and an API on top of it, connecting an application to the Stellar project should be easy for developers.

Tradeoff 1

In a blockchain architecture, all nodes in the network individually verify all the broadcasted transactions. The system does not depend on a single authority or central software infrastructure, ensuring safety and reliability. The downside is that all nodes have to validate all the transactions. This contrasts with a conventional architecture where this only has to be done by a single node. Validating the transactions multiple times increases the upkeep cost of the system. Stellar uses the described architecture to ensure safety and reliability. These choices outweigh the increased upkeep costs because the system is still cheaper for stakeholders than alternative transaction systems.

Tradeoff 2

Stellar uses the unique SCP, a Byzantine-Agreement-based protocol, whereas other cryptocurrencies use a proof-of-work or proof-of-stake based protocol. In a proof-of-work based system, when all nodes are split into two groups that do not interact with each other, two different blockchains emerge out of one, a fork happens. It ensures that the blockchain always keeps processing transactions. In the SCP this can never happen since the system would stop processing transactions when no consensus is reached. Thus the SCP values security over liveability.

Tradeoff 3

Stellar makes use of a two-layer architecture where the Horizon API sits between the application of a user and the Stellar-core. The incentive to implement the Horizon layer on top of the Stellar-core is clear; it makes it easier for developers to connect their application to Stellar. The downside is performance: directly communicating with the Stellar-core instead of an API is always faster. However, since adoption is essential for Stellar, the ease of use outweighs the loss in performance.

References

  1. Stellar Consensus Protocol, Referenced 12 Mar, https://github.com/stellar/stellar-core/blob/master/src/scp/readme.md ↩︎

  2. Paper detailing the Stellar Consensus Protocol, Referenced 12 Mar, https://www.stellar.org/papers/stellar-consensus-protocol.pdf ↩︎

  3. Architecture: Referenced 12 Mar, https://github.com/stellar/stellar-core/blob/master/docs/architecture.md ↩︎

  4. Herder, Referenced 12 Mar, https://github.com/stellar/stellar-core/tree/master/src/herder ↩︎

  5. Stellar Docs, Referenced 12 Mar, https://github.com/stellar/stellar-core/tree/master/docs ↩︎

  6. Ledger, Referenced 12 Mar, https://github.com/stellar/stellar-core/tree/master/src/ledger ↩︎

  7. Overlay: Referenced 12 Mar, https://github.com/stellar/stellar-core/blob/master/src/overlay/readme.md ↩︎

  8. BucketList, Referenced 12 Mar, https://github.com/stellar/stellar-core/blob/master/src/bucket/readme.md ↩︎

  9. History: Referenced 12 Mar, https://github.com/stellar/stellar-core/blob/master/src/history/readme.md ↩︎

  10. Stellar Core, Referenced 9 Mar, https://github.com/stellar/stellar-core ↩︎

  11. Stellar Core Docker container, Referenced 12 Mar, https://github.com/stellar/docker-stellar-core ↩︎

  12. Stellar Horizon repository, Referenced 9 Mar, https://github.com/stellar/go ↩︎

  13. Postman, Referenced 9 Mar, https://www.postman.com/ ↩︎

  14. Insomnia, Referenced 9 Mar, https://insomnia.rest/ ↩︎

  15. Stellar SDKs, Referenced 9 Mar, https://developers.stellar.org/docs/software-and-sdks/ ↩︎

  16. Stellar Horizon Developer API Refernece, Referenced 9 Mar, https://developers.stellar.org/api ↩︎

  17. Hypertext Application Language, Referenced 9 Mar, https://en.wikipedia.org/wiki/Hypertext_Application_Language ↩︎

  18. HTML Server Sent Events API, Referenced 9 Mar, https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events ↩︎

  19. EventSource Javascript object documentation, Referenced 9 Mar, https://developer.mozilla.org/nl/docs/Web/API/EventSource ↩︎