Run an Orchestrate sandbox

Goal

development environment aims to run locally the full Orchestrate architecture of microservices, including kafka messaging, databases, APIs and workers.

Once started we could launch tests and reports to mock sending transactions and confirm that Orchestrate is working properly.

To integrate the development environment repository with our own service or application, we can use the SDK as described on Orchestrate’s documentation here.

Overview

Running Orchestrate development environment implies running a set of Orchestrate infrastructure services and Orchestrate APIs and workers using them.

Infrastructure services

  • Kafka: message broker service that enables communication between workers following a publish-subscribe pattern.
  • Zookeeper: used by Kafka for coordination purposes.
  • Redis: database service used as distributed cache to store nonce values.
  • Postgres: database service used by API envelope-store to store transaction execution contexts.
  • Geth: default Ethereum client to be used by workers unless otherwise specified.

APIs and workers

  • Tx-crafter: crafts transaction’s Payload, sets Gas Price and Gas Limit and request Faucet crediting.
  • Tx-nonce: sets transaction’s nonce using Redis as a distributed cache for nonce values.
  • Tx-signer: signs transaction with accounts loaded in memory or stored in an Hashicorp Vault.
  • Tx-sender: sends transaction to the Ethereum client and store the transaction execution context by sending it to api-envelope-store.
  • Api-envelope-store: stores transaction execution context while transaction is being mined.
  • Tx-listener: catches transaction receipts and loads and reconstitutes transaction’s envelope by interrogating api-envelope-store.
  • Tx-decoder: decodes raw events logs from transactions into a human readable mapping of strings.
  • Bridge: catches a message from a chain and broadcasts the message to another chain.

Note

Full documentation for each service is available on the section Configuring Orchestrate.

Quick Start

1. Prerequisites

  • Having docker installed, version 18.09 or upper;
  • Having docker-compose installed, version 1.23.2 or upper;
  • Having jq installed;
  • Having the right to pull images from the private gitlab registry.

Info

To launch Orchestrate development environment on Windows you will need to adapt the Makefile.

2. Configuration

Configure GitLab credentials

If we have not done this already, we will need to create a GITLAB_PERSONAL_ACCESS_TOKEN.

For doing so, on GitLab we go to User Settings > Access Token and create a token containing at least read_registry and read_repository rights.

One is needed for pulling images from a Docker image registry and the other is needed to fetch private go packages when building a project locally.

We need now to export our GitLab username and token to be able to pull private go packages when building locally. For doing so we can either run the two following lines or add them to our terminal’s config file for this to be persistent.

export GITLAB_USER=<GITLAB_USER>
export GITLAB_TOKEN=<GITLAB_PERSONAL_ACCESS_TOKEN>

Info

Please note that the config file name and location varies depending on the os and terminal we are using, e.g. .bashrc, .zshrc, .bash_profile.

Authenticate into the registry containing Orchestrate images

We now need to authenticate to GitLab with our GitLab login and our Personal Access Token as the password (see above). To login we run the following command:

echo $GITLAB_TOKEN | docker login registry.gitlab.com --password-stdin --username $GITLAB_USER

Note

You can find optional configurations for Ethereum endpoints and Signing accounts under the section How to.

3. Start Orchestrate development environment

3.1. Clone Orchestrate’s development environment repository

To clone the repository run the following lines.

git clone git@gitlab.com:ConsenSys/client/fr/core-stack/orchestrate.git orchestrate
cd orchestrate

3.2. Pulling images from docker registry

Start the infrastructure, Orchestrate APIs and workers

On this step all necessary images will be downloaded and automatically started.

We need to run the following command and wait till the execution is done.

make start

Now we are ready to start sending transactions by using Orchestrate’s SDK available in JS & TypeScript here.

Stop Orchestrate and Infra

If we want to delete infra’s and Orchestrate’s containers, docker network and volumes we run the following command:

make down

Important

    Be aware that if you do not run this command line, Orchestrate will continue to run on the back.