CircleCI Basics

What’s Circle CI?

What’s Circle CI? Before getting there, I think it’s important to know what’s a CI (Continuous Integration) process? We’re not going to deep dive into features of a CI process, but at least have a minimal grasp on what and why.

The main point of CI is to encourage developers to integrate their code into small buckets as often as possible. This approach contrast with a classic workflow where new features were developed entirely in a local machine until completion and merged into the master branch. CI promotes the split of new features in more handy modification lots, boosting the team workflow: more simple specs, smaller code reviews, faster integrations, easier error detections, and quicker rollbacks.

In short, making smaller features help us to improve team productivity, gaining efficiency and velocity. As a result, everyone is happier. Especially your boss. If she/he is happy, perhaps your salary will rise.

CI is not only a set of guidelines but also proposes the automation of testing, integration, and deployment of the codebase. And here is where the CircleCI enters. Like many other CI services, CircleCI offers the tools to automate all these operations.

Let’s see what pieces compose CircleCI.

How CircleCi works

What it takes to configure CircleCI is creating a configuration, config.yml file in a directory named .circleci. This folder must reside at the root of your project and being push into the remote repository. Once CircleCI detects the configuration file, it starts to run the CI operations on each new modification.

Of course, this setup won’t work unless authorizing CircleCI to access the software repository. Once configured, the CI is ready to listen for new commits.

A config.yml file describes the steps the CI performs. These steps are grouped in jobs that run in provided containers called executors. What jobs are executed is determined by a workflow, which orchestrates the process.

This whole process described in the previous paragraph is named a pipeline. Each time modifications are pushed into the repository, CircleCI triggers a new pipeline, executing what’s described in the configuration.

Anatomy of config.yml

Commands

A command is a set of steps sequentially executed. One command is composed of single or multiple actions. As from version 2.1, commands can be declared globally. Global commands accept parameters.

Right below, there is an example of a reusable command definition.

commands:
  saywhat: # command name
    description: "A say what command"
    parameters:
      what:
        type: string
        default: "Something"
    steps:
      # It can be more than one action per command
      - run: echo << parameters.what >> 

Parameters

Parameters can be used by commands, jobs, executors. Even pipelines. We’ll see in a bit what are jobs and executors and pipelines.

There are four types of supported parameters:

• string
• boolean
• integer
• enum

The former three are easy to understand. enum type is used to declare a set of values that can be provided as parameters.

parameters:
  log:
    default: false
    type: boolean
  version: # mandatory value
    type: number
  name:
    default: string
    default: swile
  type:
    default: "alpha"
    description: "alpha, beta, nightly".
    type: enum
    enum: ["alpha", "beta", "nightly"]

Jobs

A job is a collection of commands executed sequentially. E.g., running a test suite.

A test job can be comprised of the next actions to set up the environment: installing dependencies, create a database, seed with data, and finally, execute the tests.

To do so, a job must specify in which container the execution will take place. A container can be a docker, virtual machine, or the very same machine (only Linux). In the context of a job, a container is called an executor. Just like commands, jobs can be declared globally.

jobs:
  sayhello:
    parameters:
      to:
        description: "To whom we say hello?"
        default: "Anonymus"
        type: string
    machine:
      image: ubuntu-1604:201903-01
    steps:
      - saywhat: # invoking sayhello command
          what: Hello << parameters.to >>!

An interesting feature jobs present is the possibility to persist workspaces between jobs.

The workspace is the “physical” place where the build process takes place. Generated files will be available in subsequent jobs. This feature can save some time during the workflow execution.

It is important to mention this feature is only available in the current workflow. After the workflow ends, the workspace is no longer present.

Executors

This time I’m going to quote directly the official docs. It cannot be better expressed.

An executor defines the underlying technology or environment in which to run a job.

Knowing what an executor is, there are four types: docker, machine, macOS, and windows. Note that the machine type is a Linux virtual machine.

Depends on what type is chosen, different resources will be available. For example, if we’re trying to compile an iOS app, most probably a macOS type will be selected, so Xcode will be available.

You can read more on the subject in the official docs.

Below, there is an example of an executor. Notice an argument can be passed to choose the version of node.

executors:
  node-docker: # declares a reusable executor
  parameters:
    - version:
      description: "version tag"
      default: "lts"
      type: string
  docker:
    - image: cimg/node:<< parameters.version >>

Workflows

Workflows describe how jobs are going to be executed. It may well be filtering which job is processed by branch and/or tag, adding conditions, scheduling jobs, and running jobs concurrently.

When a job fails, it stops the execution and notifies immediately what happened. CircleCI permits rerunning workflows from the point it was stopped. Also, a failed workflow can re-run with access by ssh to the workspace the error took place.

As jobs may store temporary workspaces, workflows can store data to be reused between workflows. An example of this caching feature is by storing dependencies installed during the build process. Usually a pretty expensive action. Once cached the first time, the next workflows reuse the same dependencies and only download new dependencies that will be stored for the next workflows.

workflows:
  sayhellotoworld:
    jobs:
      - sayhello:
          to: World
  sayhellotouniverse:
    jobs:
      - sayhello:
          to: Universe
    requires:
      - sayhellotoworld
    filters:
      branches:
        only: master
  sayhellotoalternativeworld:
    jobs:
      - sayhello:
          to: Alternarive World
    filters:
      branches:
        only: alternative

In the example above, there are three workflows: sayhellotoworld, sayhellotouniverse and sayhellotoalternativeworld.

sayhellotouniverse is executed only when the current branch is master. sayhellotoalternativeworld will run when the branch is alternative. And lastly, sayhellotoworld is executed no matter what branch.

When the branch is master, both sayhellotoworld and sayhellotouniverse are performed but one after another. The requires statement specifies that sayhellotouniverse must wait until sayhellotoworld has finished.

On the other hand, if the branch happens to be alternative, both workflows sayhellotoworld and sayhellotoalternativeworld run in parallel. In fact this is the default behaviour.

Pipelines

All the described previous elements, once assembled together in a config.yml, orchestrate how CircleCI executes a new process each time modifications are detected in the repository. Each one of these processes is called a pipeline.

Each time a new pipeline is created, based on current context and worlflow conditions, executes one or another workflow. But pipelines can be triggered manually. By following the API contract described in the docs, a new pipeline can be executed from a current workflow. In fact, this is the approach used when working with monorepos: a default workflow executes new pipelines for different packages.

Orbs

Orbs can be seen as plugins or libraries. Instead of packing compiled sources as a node module or a ruby gem, it contains reusable snippets of configuration. Usually, orbs consist of jobs, commands, and executors.

Like any library, an orb can be versioned, too.

There are two types of orbs: Certified and 3rd-party. Usually, certified orbs are provided by CircleCI. On the other hand, 3rd-party orbs are not verified by CircleCI. To make use of 3rd-party orbs, the project settings must be updated to allow the use of them.

orbs:
  node: circleci/node@4.0.0 #orb version

workflows:
  test_my_app:
    jobs:
      - node/test: # job provided by orb
          version: 12.06

The configuration above sets the circleci/node to version 4. The orb has been named as node. In the jobs section, the test job provided by the node orb is used with the version parameter.

CircleCI also provides a repository for orbs. You can go there to check what orb suits your project needs.

Contexts

Contexts allow us to define environment variables for a specific scope. Variables are also cyphered. Managing context variables is done from the CircleCi app. Once a context is created, it can be used in any job from a workflow declaration, declaring the context property.

jobs:
  test:
     steps:
         - echo $TEST # var from context test-vars
         - echo "run tests"
  deploy:
     steps:
         - echo $DEPLOY_NAME # var from context deploy-vars
         - echo $AWS_TOKEN # var from context aws-vars
         - echo "run depoy"
workflows:
  test_and_deploy:
    jobs:
       - test:
          context: test-vars
       - deploy:
          context: 
            - deploy-vars
            - aws-vars

Full sample

Below we can peek at a full CircleCI configuration file. As we see, not all the features seen before have been used. For what the code does, there’s no need to declare executors nor even orbs. But we can get a neat idea of how things work by observing at this simple example.

version: 2.1
commands:
  saywhat:
    description: "A say what command"
    parameters:
      what:
        type: string
        default: "Something"
    steps:
      - run: echo << parameters.what >> 
jobs:
  sayhello:
    machine:
      image: ubuntu-1604:201903-01
    parameters:
      to:
        description: "To whom we say hello?"
        default: "Anonymus"
        type: string
    steps:
      - saywhat:
          what: Hello << parameters.to >>!
workflows:
  sayhellotoworld:
    jobs:
      - sayhello:
          to: World
    filters:
      branches:
        only: master
  sayhellotouniverse:
    jobs:
      - sayhello:
          to: Universe
    requires:
      - sayhellotoworld
    filters:
      branches:
        only: master
  sayhellotoalternativeworld:
    jobs:
      - sayhello:
          to: Alternative World
    filters:
      branches:
        only: alternative