SAS G-CLI Tools Documentation

Getting Started

About

I often get questions about how to get started with Continuous Integration and Continuous Delivery (CI/CD) in LabVIEW. Many developers see the value in it, but are unsure how to get started. The first step is usually to be able to use the CLI to automate various development tasks. That is what these tools are for.

SAS G-CLI Tools is a set of Open Source G-CLI tools for common LabVIEW Tasks. Each is provided in its own VIPM package, so you can install just the ones you want. Each G-CLI tool is also Open Source and hosted in it's own GitLab repository, so you can fork them and modify them all you want to match your specific workflow.

These tools are useful for automating your testing and build process in LabVIEW. Using these tools, you can write a simple bash script to run your unit tests and then if they pass build your executable and package it up. This helps you create a robust and repeatable process. These essential tools are the building blocks of a CI/CD pipeline.

Why?

CI/CD is the best investment you can make in risk management for your project. It prevents the 2 major sources of software defects: building tthe THING wrong and building the WRONG thing. Regularly running unit tests via CI helps to ensure that the software works as you intend and that you are building the THING right. Continuously delivering software to you users using CD ensures that you can get quick feedback from your users to make sure you are all on the same page and building the RIGHT thing.

Setting up CI/CD can be daunting. There are a lot of moving parts. There's also not a lot of good documentation out there. This project hopes to change that.

New to CI?

Starting out with CI can be overwhelming. If you are completely new to it all, here is how I recommend getting started.

  1. Checkout our introductory videos. It will give you a good overview of what is possible and what is involved.
  2. Map out your current process. Think about what happens after you make a change to your software. What steps do you take in order to verify that change works and didn't break anything. Then how do you build your application and distribute it. Write it all down.
  3. Look through the list of G-CLI tools here and connect the steps in your process to the G-CLI tools. Write a bash script that calls each of the steps in order. The goal is a single batch script that developers run locally that will run your tests and build your program and installer.
  4. Once you have that script, then head to the Setting up a GitLab Runner section. It will walk you through how to setup a GitLab Runner and get it to execute your script.
  5. Also checkout one of our Sample Projects. It will provide some inspiration and/or a useful starting point.
  6. Collect feedback and improve. Once you have a pipeline in place, then it easy to tweak existing steps and add now ones.

At some point after step 3 it is worth checking out the sample projects. They may do exactly what need. They will at least show you what is possible and point you in the right direction of putting all the pieces together.

Already using some form of CI?

Head to our sample projects to see these tools in use. The examples are all GitLab repositories that you can fork and experiment with. You'll also want to check out the section on Setting Up Dynamic Runners

Introductory Videos

These G-CLI tools are one of the first steps in setting up a CI/CD Pipeline. Here are some introductory videos to GitLab CI and CI/CD in general. None of these are specifically about SAS-GCLI-Tools, but they provide some good and necessary background information. These links are here to help get you up and running with the rest of that CI/CD pipeline setup.

Here are some other alternative options worth exploring:

Installation

Each of these G-CLI Tools is available on VIPM. That is the easiest way to install them.

Here are the links to download the latest packages from VIPM.

Liscensing

All of the tools are licensed under the MIT license. A license file is included in each VIPM installation package.

Verifying G-CLI Works

You need to make sure that G-CLI is in your PATH. Instructions for that differ based on what shell you are using.

To test G-CLI you can simply do the following:

g-cli echo -- "hello world"

Supported Versions

  • LabVIEW -These tools are packaged with VIPM using LabVIEW 2020, so they should work with any LabVIEW bitness, version 2020 or later.
  • OS - These tools are Windows only at the moment, although the latest G-CLI apparently also works on Linux, so perhaps that may work. If you try it and find out, please let me know. I'm still on Windows 10, but there is no reason they shouldn't work with Windows 11.
  • G-CLI - 2.4.0.4 or later.

Available Commands

  • clearlvcache - A G-CLI tool for clearing the LabVIEW Cache.

  • vipc - A G-CLI tool for applying VIPC files.

  • lvbuildspec - A G-CLI tool for building LabVIEW build specifications.

  • vitester - A G-CLI tool for running VI Tester Tests

  • lunit - A G-CLI tool for running LUnit Tests.

  • vipb - A G-CLI tool for building VIPM packages from .vipb build specifications.

  • vip - A G-CLI tool for installing VIPM packages.

  • switcheroo - A G-CLI tool for switching out source libraries to test PPLs and VIPM packages.

clearlvcache

image stars badge installs badge

Description

This tool clears both the AppBuilder and CompiledObject Caches. I generally use this as one of the first steps in a job, especially if you are running your job in a VM as opposed to a Docker image. It may not be as useful for jobs running in Docker, since presumably each run of the job starts with a fresh slate by spinning up a new container.

NOTE: in G-CLI version 3.0+ this can be replaced with the built in ClearCache command.

Options

This G-CLI Tool has no options. You just call it directly from G-CLI.

Example

# G-CLI options ignored for simplicity.
# it is really this simple.
g-cli clearlvcache

Repository

For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/clearlvcache

vipc

image stars badge installs badge

Description

This tool allows you to apply a VI Package Configuration file (vipc) using G-CLI. You point to the vipc file and the version of LabVIEW and this tool will check to see if it needs applied and apply it if necessary.

Options

This G-CLI Tool has the following options:

flag parameters required description
--lv-version, -v LabVIEW Version no LabVIEW version to install VIPC to. Defaults to internal version of VIPC file. If 64 bit LabVIEW you must specify. Example 22.3 (64-bit).
--timeout, -t VIPM API timeout no time to wait for VIPM API timeout. Defaults to 300s

Example

# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli vipc -- -v "22.3 (64-bit)" myvipcfile.vipc  # Applies myvipcfile.vipc to LabVIEW 22Q3 64 bit

Repository

For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/vipc

lvbuildspec

image stars badge installs badge

Description

This tool allows you to build LabVIEW build specifications using G-CLI. It allows to take advantage of automatic version calculation, manually supply a build number or use the built-in autoincrement functionality. For a description of the auto-increment functionality, see this blog post

Options

This G-CLI Tool has the following options:

flag parameters required description
--project, --proj, -p project path yes Project containing build specification
--buildspec, -b build spec to build yes Specific build specification to build
--target, -t target to build for no Target that contains build specification. Defaults to MyComputer
--version, -v version no Version number x.x.x.x to assign to build. If left empty just uses whatever is currently set in the LabVIEW Project. --manualversion and -mv also work for backwards compatibility, but are deprecated and will eventually be removed

Example

# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli lvbuildspec -- -p myproject.lvproj -b myexe -v 1.2.3.4 # builds my exe with version number set to 1.2.3.4

Repository

For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/lvbuildspec

vitester

image stars badge installs badge

Description

This tool runs a set of vitester tests and generates a JUnit report file.

Options

This G-CLI Tool has the following options:

flag parameters required description
--report, -r report path yes Path to write JUnit report to.

Example

# G-CLI options ignored for simplicity.
# it is really this simple.
g-cli vitester -- -r UnitTestReport.xml myproject.lvproj # this points to a project, you can also point to a Test Case or Test Suite Class

Repository

For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/vitester

lunit

image stars badge installs badge

Description

This tool allows you to run LUnit tests from the CLI using G CLI.

Options

This G-CLI Tool has the following options:

flag parameters required description
--report, -r, report path yes Path to the report
--parallel parallel no Flag to run tests in parallel

Trailing Arguments are interpreted as paths. Typically this is the LabVIEW project you want to scon for tests to run.

Example

# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli lunit -- -r UnitTestReport.xml mylabviewproject.lvproj # Runs alll lunit tests found in mylabviewproject.lvproj

Repository

For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/lunit

vipb

image stars badge installs badge

Description

This tool allows you to build VIPM Packages from the CLI using G CLI.

Options

This G-CLI Tool has the following options:

flag parameters required description
--buildspec, -b, build spec yes Path to the vipb file to build
--version, -v version no Version to build
--release-notes, -rn release notes file no Path to a file containing the release notes
--timeout, -t timeout no Timeout in seconds defaults to 1200 = 20 minutes

Example

# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli vipb -- -b MyBuildspec.vipb -v 1.2.3.4 # builds version 1.2.3.4

Repository

For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/vipb

vip

image stars badge installs badge

Description

This tool allows you to install an individual VIP file from the CLI using G CLI. This is useful for installing packages for testing after you build them. For general dependency management use the vipc tool

Options

This G-CLI Tool has the following options:

flag parameters required description
--local-vip-file package to install no Install specific package file
--timeout, -t VIPM API timeout no Timeout in seconds defaults to 1200 = 20 minutes
--lv-version, -v LabVIEW Version yes LabVIEW version to install VIPC to

Example

# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli vip -- --lv-version "20.0 (64-bit)" --local-vip-file "path_to_my_package.vip" # installs my package from the vip file.


Repository

For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/vip

switcheroo

image stars badge installs badge

Description

A G-CLI tool for switching source libraries to PPLs and installed VIPM packages for testing. Allows you to write tests once for the source and then reuse those tests against the built/installed library.

Options

This G-CLI Tool has the following options:

flag parameters required description
--test-dir, -t test directory yes Directory of Tests to load - these are the callers of the library you are going to replace.
--original, -o original library yes Library to be replaced
--replacement, -r replacement library yes Original Library will be replaced with this.

NOTE: for original and replacement you can use <vilib> which will get replaced with the path to vilib.

Example

# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli switcheroo -- -t "tests" -o "mylib.lvlib" -r "mylib.lvlibp"
or
g-cli switcheroo -- -t "tests" -o "mylib.lvib" -r "<vilib>/myrenamedlib.lvlib"

Repository

For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/switcheroo

Third Party CLI Tools for other common commands

These are tools I didn't bother to implement G-CLI commands for, since they are already provided. You may find these useful. I'm not going to document them here, since the developers have already done that.

Setting Up Static Runners

In order to fully take advantage of CI, at some point you are going to need to set up a runner. Static Runners are an easy starting point. This is a machine (VM or physical) that is constantly running. Your jobs run directly on this machine. This makes for easy setup and troubleshooting.

For CI, I typically use GitLab, so these instructions are tailored to GitLab (I'll also include some info for GitHub users). On my blog I have a few posts about setting up GitLab Runners. It's spread over a few articles and a little disjointed. I'll attempt summarize it all in one place here. If you prefer video I have a video of me walking Casey through the process. It was made before this tutorial so it might not match up exactly.

Here is a diagram showing the basic setup.

CI Diagram

How Many Runners Do I Need?

This is a common question. It is possible to have a single runner with multiple versions of LabVIEW installed. In the past multiple versions of LabVIEW have not always played well together so I recommend against this. I recommend one runner per version of LabVIEW or one per project if your projects have 3rd party drivers or other unique requirements. This keeps things better isolated. This can result in a lot of machines, so Virtual Machines are your friends here or Docker if you are brave, but that is a whole other can of worms. You can save yourself a little trouble by having a common Win10 or W11 machine that you copy as a starting point. You probably don't want to clone it - that can cause some problems. Better to just copy the VM Disk and create a new VM from it.

Assumptions

I'm making a few assumptions here. I am assuming you are using Windows (it should be similar for Win10 or Win11). I'm also going to be setting this up to use Git Bash instead of Powershell. I detest Microsoft and I don't know Powershell, but I do know Bash, so that is what I am going to use. If you want to use Powershell, the initial setup will be a little easier (because that is the default for GitLab shell runners on Windows). However you may have to tweak some of the yaml files and scripts. I am also assuming that you are using VIPM pro or the Community Edition. You need one of those in order for the vipc applier tool to work.

Starting Point

As a runner you will generally want a seperate machine from your dev machine. This could be a spare laptop or desktop, a VM running on your own desktop or server or a VPS running on something like Amazon, Vultr, or Digital Ocean. You'll need just enough horsepower to load your project and run a build. It will need access out to the internet to be able to ping gitlab.com (or if you are self-hosting, then whatever your url is for your GitLab server). You should consider keeping it on a seperate network isolated from your main network, because running a build basically opens up a reverse shell.

NOTE: If you are just starting out and dabbling with CI, an easy solution is to install GitLab Runner on your Development machine. It is much less than ideal, but lowers the barrier to entry and allows you to easily see what is going on. In that case you can probably skip the section on setting the script to run on startup and setting up autologon.

Access to Runner Machine

You will want to be able to logon to the runner and view the GUI for setup/installation and troubleshooting. For a physical machine that should be easy. RDP works best for remote machines, but VNC, MSP360 or Team Viewer will also work. Don't expose RDP or VNC to the internet. It is not secure. Run it through some encrypted tunnel like ssh, a vpn, wireguard, Nebula, etc.

Initial Software Installation

I'm assuming you are using Windows, so you will want to install the appropriate Windows Version and turn on Remote Desktop (if that is what you are using for access) You will want to install Git For Windows along with Git Bash. The default options should be fine. You will want to install NIPM and use that to install the proper version of LabVIEW, VIPM, DAQmx and any other drivers you need. You'll also want to install some basic VIPM packages such as G-CLI and the VIPM API. You'll also want to install whatever of the SAS-GCLI tools you want to use. If you have the Pro or Community edition of VIPM, then you can use the vipc applier tool to install packages as part of your CI pipeline. If you aren't doing that, then you should probably install the needed packages now. If you are using Python to run any build scripts or utilities, now is a good time to install that and any needed Python packages as well.

You really don't want to install a lot of extras. You are not going to be using this machine for anything other than running builds. Running a build basically opens up a reverse shell, so you don't want anything sensitive on there other than your code.

Initial Verification

You should manually open VIPM and LabVIEW and make sure no licensing dialogs pop up and take care of them if they do. You should verify that VIPM can connect to your version of LabVIEW by opening VIPM and going to the settings dialog box. You should also verify G-CLI is installed correctly and can connect to LabVIEW by typing g-cli echo -- "Hello World" into Git Bash and verifying that it returns "Hello World". If it doesn't, check your $PATH and make sure it contains the G-CLI directory. You may need to add an export line to your .bashrc file.

Setting Up GitLab Runner

Installing GitLab Runner

The instructions for installing the gitlab-runner program provided by GitLab are pretty good. Make sure you select shell as your executor when it asks. It will also ask you to give it a name and assign a tag to it. You can change these later through the web interface. Generally you'll want to tag a runner either with the name of a specifc project or based on the software versions installed. The tags are used to determine what jobs GitLab can run on this particular runner. You should just follow the instructions up until the point where it tells you to install it as a service. If you want to be able to actively debug and watch what is going on, you don't want to do this. So stop after step 5 and don't proceed to step 6.

gitlab-runner installation instructions

Setting up GitLab Runner to use Git Bash Instead of PowerShell

NOTE: This section describes how to set up GitLab Runner to use Git Bash. If you want to use PowerShell you can skip this.

First, we need to help GitLab Runner find GitBash. Add a bash.cmd file into your user directory (the user that runs the GitLab Runner) with the following contents:

@"C:\Program Files\Git\usr\bin\bash.exe" -l

Next, we have to make sure that our starting paths are set up correctly with Linux-style formatting. In your config.toml file for your GitLab runner (it’s in the directory where you installed GitLab Runner), change the shell to bash and add Linux style paths for the builds_dir and cache_dir. That’s it. See the example below.

[[runners]]
  name = "windows"
  url = "https://your.server.name"
  token = "YOUR_SECRET_TOKEN"
  executor = "shell"
  shell = "bash"
  builds_dir="/c/gitlab-runner/builds/"
  cache_dir="/c/gitlab-runner/cache/"

NOTE on Paths: We are using linux paths here, but when passing paths as parameters to the g-cli tools (ie the parameters after the -- ) you will need to use Windows Paths.

Running as a User

Create a batch file with the following line in it.

"C:\Program Files\Git\bin\sh.exe" --login -i -c "cd /c/Gitlab-Runner && ./gitlab-runner.exe run"

Save it somewhere convenient, like the Desktop and give it a meaningful name like "Start GitLab Runner.bat" Running this batch file will open a console window as seen below. Once a job starts, then you’ll see all the windows that it opens. You’ll see the LabVIEW splash screen and any VIs it opens. So if it hangs on some dialog, you will know about it.

Set to run at Startup

In order to run the script at startup (when the current user logs in), drop a link to into the Windows Startup Folder. You can find the Windows startup folder by hitting the windows key + R and typing shell:startup into the resulting dialog box.

Setting User to Autologon

The last thing I typically do is set the user to autologon, otherwise every time the machine boots up, you have to login to the machine before gitlab-runner can start. The way to do that in windows is to use netplwiz. Hit windows key + R and type in netplwiz. It will open a User Accounts window. You should see a checkbox at the top that says "Users must enter a user name and password to use this computer." Simply uncheck it.

netplwiz

NOTE: You may not see that checkbox. If you don't you can try adding a registry key. Copy the code below. Open a command prompt as administrator. Paste and run the code. Reboot. The box should show up now. If not, there are some other potential fixes, just google it.

reg ADD “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\PasswordLess\Device” /v DevicePasswordLessBuildVersion /t REG_DWORD /d 0 /f

At this point I usually reboot the runner and just sit and watch to make sure when it boots up it doesn't hang waiting for a Windows password and that the batch script runs on startup.

Verifying in GitLab

Verifying the Connection

If you log into your project and go to the CI/CD settings and the runner is up and running (ie you have ran the batch file we created), you should see it in the list of assigned runners.

CICD Runners Settings Page]

Assigning Runners and changing tags

Once you have a runner up and connected to your gitlab server, it is easy to make changes to it through the web interface. You can change the name and the tags. You can also enable it for different projects (by default they are locked to a specific project, so you have to unlock it first). You can do all this by clicking on the edit button next to the runner to open the runner configuration editor.

Runner Configuration

Success!

Once you have done all the above steps, your runner should be good to go.

Setting Up Dynamic Runners

This section is based on this blog article. It's about how to use the GitLab Custom Executor to spawn KVM VMs on demand and run jobs on them. It should work for any VM provider. The code is in this repository. You'll see that it uses some shell scripts to launch the VMs and then connects to the VMs over SSH to run the commands. There is no reason the shell scripts couldn't me modified to use another VM provider or some cloud provider.

Here is a diagram of the overall architecture that may be helpful.

KVM Executor

NOTE This is an advanced topic so I assume you have some idea of what you are doing. I make some assumptions and leave some details out.

Disadvantages

  • The main disadvantage of this approach is it is slightly more complex
  • As-is the scripts only work with GitLab and KVM on Linux. It should be portable to other platforms. For GitHub, I looked into it and there is a webhook that fires when a new job is queued. So that could be used to execute a shell script to start a VM and register it with GitHub as an ephemeral runner. If you get that to work, let me know.

Advantages

This approach offers several advantages.

  • Clean VM each time - avoid residue being left over from one job to the next. Ensures you have included all dependencies.
  • Scaling and Running in Parallel - This allows you to run multiple different (or same) VMs in parallel.
  • Ease of setting up New Base VMs
  • Better Resource Utilization - If you have several projects, you only have 1 runner listening.

Setting Up The Host

I use KVM on Linux. Google how to set that up. Then I clone these scripts and place them somewhere accessible by the user that runs KVM. You also need a directory full of Base VMs and a directory for clones. Putting all clones in one directory makes housekeeping easier if something get's orphaned. Getting the permissions right can be a challenge. I suggest starting with a project with a really simple hello world and getting that to work first. Also you'll need to setup an SSH key to ssh into the VMs without a password - so don't make the key password protected!

Setting Up The Base VM Images

This is for Windows machines. I generally install Git For Windows and NIPM. Then I set up SSH and everything else. Then I clone that VM and install LabVIEW and VIPM. That gives me 2 base VMs. One with just Git and NIPM so I can test installs and one with LabVIEW, VIPM, etc for running my builds.

Windows SSH

You need to setup SSH server on your Windows VM. See this blog article

Necessary Packages For BootStrapping

You'll need the G-CLI, VIPM API and the VIPC Applier packages as a minimum. With that you can use the VIPC Applier to install anything else you need. Remember this happens on every job. If it starts taking too long, you can create a seperate image for a particular project with the vipc preinstalled. I usually keep the VIPC step in that case anyway, just incase I change something like add or upgrade a package.

GitLab Runner

You only technically need GitLab Runner on the KVM host. However if you want to upload artifacts then you need gitlab-runner on the base VM and in the path. You don't register it, but it does have to be in the path.

Setting Up The gitlab-ci.yml File

You have 2 things that you need to add to your yml file in order to run a job this way. First you need a tag to tell GitLab which runner to dispatch the job to. I use "kvm" for this. You also need a $BASE_VM variable to tell the runner which VM to spin up and run the job on.

Parallel Jobs

You can enable parallel jobs by editing the runner config on the KVM host. Just change the concurrent setting.

Sample Projects

Once you have a runner setup, the next step is to test it out. An easy way to do that is to use one of our sample projects. You can simply fork the project. Then make sure that the tag in the CI matches your runner and your runner is assigned to the project and it should be good to go. Make a change, commit, and push and it should kick off a pipeline. If you aren't using VIPM pro, you may need to manually apply the VIPC to the runner, but other than that it should work.

Why Bash scripts?

You'll notice in the sample projects I put all the commands in Bash scripts. Why is that? It gives us 2 things.

  1. It allows us to run the commands locally (ie ./build.sh)
  2. It makes your CI more portable. Almost any CI system can run bash scripts. The difference between GitHub and GitLab is just in how we tell the system which script to run and when.

New to Bash?

I chose Bash because I am familiar with it (more than PowerShell) and because it is cross-platform. If you are getting started, I recommend purchasing this cheatsheet. The biggest takeaway is to make sure you add set -euo pipefail at the top of your bash scripts so the error handling and exit codes work correctly.

A Note On Variables

In the GitLab examples, you'll notice this line:

. <(sed -nr '/variables:/,$ s/  ([A-Z_]+): (.*)/\1=\2/ p' vars.yml)

What this line does is that it reads in the vars.yml file and takes the variables section and sets the variables there as environment variables. This is so we can define the variables once and use them when running locally or on the CI server (this file is included in the .gitlab-ci.yml file). The sed command is kind of black magic but it works. I got it here: https://unix.stackexchange.com/questions/539009/export-environment-variables-parsed-from-yaml-text-file