🛸Flowcontrol

Appearance

🔄 CI/CD ​

The CI/CD pipeline of Flowcontrol consists of different types of workflows. These can be categorized into two groups:

Continuous Integration Actions (CI) ​

  • Trigger: CI actions are triggered by a push or pull request to any of the three branches of the repository—development, staging, and master.
  • Purpose: These actions are designed to automatically perform API tests and build the software whenever changes are committed, ensuring that new code integrates well with the existing codebase.

Continuous Deployment Actions (CD) ​

  • Trigger: CD actions are primarily triggered by a pull request from the staging branch to the master branch.
  • Purpose: These actions facilitate the automated deployment of tested and stable code to production environments, ensuring a smooth transition from testing to live deployment.

This structured approach to CI/CD ensures that each phase of the software lifecycle is carefully controlled and monitored, from development through to deployment.

🧪 Continuous Integration ​

The Continuous Integration (CI) process is divided into specialized workflows for different components of the application, specifically:

  • Documentation
  • Client
  • SJ

Workflow Descriptions ​

  • SJ Workflow: This reusable workflow caters to the article, farmer, transport and notification modules of the application. It is designed to be flexible and adaptable across these similar functionalities.
  • Client and Documentation Workflows: These workflows are distinct because their requirements do not align with the SJ workflow. Each has tailored steps that address their unique integration needs.

Integration into Main Workflows ​

These component-specific workflows are integrated into the main workflows (development, staging, master), performing their respective tasks before advancing to the next stages of the main workflow.

SonarQube Setup ​

  • Setup Process: Following the success of the initial CI workflows, SonarQube is configured to run according to the environment specified by the main workflow:
    • Development Environment: Utilizes the development-specific Docker Compose setup to create application containers and configure SonarQube. The SonarQube quality gate is not enforced here, allowing the workflow to pass even if the quality metrics fail.
    • Staging and Production Environments: In these environments, passing the SonarQube quality gate is mandatory. The setup ensures that only code that meets stringent quality standards progresses in these pipelines.

Quality Gate Enforcement ​

  • Development: The lack of enforcement on SonarQube metrics allows for greater flexibility and iterative improvements without blocking the development process.
  • Staging and Production: Strict adherence to the quality gate results ensures that only thoroughly vetted and stable code is moved forward, aligning with higher stakes in these environments.

This structured CI approach enhances the application's reliability by ensuring each component is individually tested and integrated before merging into larger workflows, with quality checks tailored to the needs of each environment.

🚀 Continuous Deployment ​

The continuous deployment process utilizes specific workflows for different environments, namely:

  • Development
  • Staging
  • Production

Workflow Triggers ​

Each workflow is triggered based on the target environment to which the code is being pushed or from which it is pulled. Upon pushing to these branches, Docker Compose is utilized, tailored to each specific branch.

Configuration and Environment Variables ​

  • Environment Variables: At the start of each Docker Compose file, environment variables (env) are specified. These are critical for configuring the services correctly in each environment.
  • Variable Storage:
    • Sensitive Variables: Stored securely in GitHub Secrets to ensure confidentiality.
    • Non-sensitive Variables: Stored openly as "vars" which are less critical and can be publicly visible if necessary.

Build and Quality Checks ​

  • Build Verification: The workflows include steps to verify whether the application compiles and builds correctly.
  • Quality Gate: SonarQube is used to ensure the code meets quality standards. If the quality gate fails, the workflow will halt, preventing deployment.
    • Exception: In the development environment, the SonarQube quality check is not enforced, allowing for more flexibility during the initial phases of development.

Notifications and Error Handling ​

If a workflow fails, a comment will be automatically posted on the commit detailing the specific part of the pipeline that failed, facilitating quick identification and resolution of issues.

This structured approach to continuous deployment ensures efficient management of deployments across different environments while maintaining high standards of code quality and security.

A successful start-up of each of these containers concludes the deployment workflow.

🏃 CI/CD Runners Overview ​

Our CI/CD pipeline exclusively utilizes self-hosted runners, enhancing control and customization of our workflows:

Self-Hosted Runners ​

  • Location: These runners are hosted on our infrastructure, providing direct integration with our production, staging, and development environments.
  • Purpose:
    • CI Workflows: Perform Maven builds, automated tests, and SonarQube analysis. SonarQube analysis is crucial as it is configured to run per module, which requires specific system settings available only on our production server.
    • CD Workflows: Handle the deployment processes, including the management of Docker containers directly on the production server where these containers ultimately run.

Runner Management and Maintenance ​

  • Daily Resets: A scheduled cron job resets the runners every day at 2 AM to prevent any issues related to inactivity or system timeouts.
  • Automatic Restart: Runners are configured to automatically restart upon boot if the staging machine is rebooted, ensuring that our pipeline remains active without manual intervention.
  • Scalability: The setup is designed to allow the easy addition of more runners as needed. This flexibility ensures that our CI/CD infrastructure can adapt to increased demands or expanding project scopes without disruption.

Current Configuration ​

  • Robust and Adaptable: Our infrastructure is tailored to meet the demands of our development cycles, providing robustness through dedicated resources and adaptability through maintenance practices that minimize downtime.

This configuration of self-hosted runners ensures that our CI/CD processes are tightly integrated with our operational requirements, offering both high performance and reliability.