SAS G-CLI Tools Documentation
Getting Started
About
I often get questions about how to get started with Continuous Integration and Continuous Delivery (CI/CD) in LabVIEW. Many developers see the value in it, but are unsure how to get started. The first step is usually to be able to use the CLI to automate various development tasks. That is what these tools are for.
SAS G-CLI Tools is a set of Open Source G-CLI tools for common LabVIEW Tasks. Each is provided in it's own VIPM package, so you can install just the ones you want. Each G-CLI tool is also Open Source and hosted in it's own GitLab repository, so you can fork them and modify them all you want to match your specific workflow.
These tools are useful for automating your testing and build process in LabVIEW. Using these tools, you can write a simple bash script to run your unit tests and then if they pass build your executable and package it up. This helps you create a robust and repeatable process. These essential tools are the building blocks of a CI/CD pipeline.
Why?
CI/CD is the best investment you can make in risk management for your project. It prevents the 2 major sources of software defects: building the thing wrong and building the wrong thing. Regularly running unit tests via CI helps to ensure that the software works as you intend. Continuously delivering software to you users using CD ensures that you can get quick feedback from your users to make sure you are all on the same page.
Setting up CI/CD can be daunting. There are a lot of moving parts. There's also not a lot of good documentation out there. This project hopes to change that.
New to CI?
Starting out with CI can be overwhelming. If you are completely now to it all, here is how I recommend getting started.
- Checkout our introductory videos. It will give you a good overview of what is possible and what is involved.
- Map out your current process. Think about what happens after you make a change to your software. What steps do you take in order to verify that change works and didn't break anything. Then how do you build your application and distribute it. Write it all down.
- Look through the list of G-CLI tools here and connect the steps in your process to the G-CLI tools. Write a bash script that calls each of the steps in order. The goal is a single batch script that developers run locally that will run your tests and build your program and installer.
- Once you have that script, then head to the Setting up a GitLab Runner section. It will walk you through how to setup a GitLab Runner and get it to execute your script.
- Also checkout one of our Sample Projects. It will provide some inspiration and/or a useful starting point.
- Collect feedback and improve. Once you have a pipeline in place, then it easy to tweak existing steps and add now ones.
At some point after step 3 it is worth checking out the sample projects. They may do exactly what need. They will at least show you what is possible and point you in the right direction of putting all the pieces together.
Already using some form of CI?
Head to our sample projects to see these tools in use. The examples are all GitLab repositories that you can fork and experiment with.
Introductory Videos
These G-CLI tools are one of the first steps in setting up a CI/CD Pipeline. Here are some introductory videos to GitLab CI and CI/CD in general. None of these are specifically about SAS-GCLI-Tools, but they provide some good and necessary background information. These links are here to help get you up and running with the rest of that CI/CD pipeline setup.
- NI Connect 2023 Panel on CI - A panel discussion about CI at NI Connect, featuring Sam Taggart, Jim Kring, Joerg Hampel, and Chris Roebuck
- GDevCon #2 Panel Discussion on CI - Some good general thoughts and an overview of CI along with some use cases.
- LAF Presentation on Getting Quicker Feedback - Getting Started With CI - A good overview of CI.
- GLA 2022 - What is this CD thing? - A demo of a couple options for extending your CI pipeline to actually deploying code.
- Setting Up a GitLab Runner - A walkthrough of setting up a GitLab Runner
Here are some other alternative options worth exploring:
- Exploring BLT - A demonstration of BLT, which is an alternative way to do a lot of CI type things.
- Getting Started with GitHub Actions - A handson presentation that walks you through setting up a GitHub agent (similar to a GitLab Runner)
Installation
Each of these G-CLI Tools is available on VIPM. That is the easiest way to install them.
Here are the links to download the latest packages from VIPM.
Liscensing
All of the tools are licensed under the MIT license. A license file is included in each VIPM installation package.
Verifying G-CLI Works
You need to make sure that G-CLI is in your PATH. Instructions for that differ based on what shell you are using.
To test G-CLI you can simply do the following:
g-cli echo -- "hello world"
Supported Versions
- LabVIEW -These tools are packaged with VIPM using LabVIEW 2020, so they should work with any LabVIEW bitness, version 2020 or later.
- OS - These tools are Windows only at the moment, although the latest G-CLI apparently also works on Linux, so perhaps that may work. If you try it and find out, please let me know. I'm still on Windows 10, but there is no reason they shouldn't work with Windows 11.
- G-CLI - 2.4.0.4 or later.
Available Commands
-
clearlvcache - A G-CLI tool for clearing the LabVIEW Cache.
-
vipc - A G-CLI tool for applying VIPC files.
-
lvbuildspec - A G-CLI tool for building LabVIEW build specifications.
-
vitester - A G-CLI tool for running VI Tester Tests
-
lunit - A G-CLI tool for running LUnit Tests.
-
vipb - A G-CLI tool for building VIPM packages from .vipb build specifications.
clearlvcache
Description
This tool clears both the AppBuilder and CompiledObject Caches. I generally use this as one of the first steps in a job, especially if you are running your job in a VM as opposed to a Docker image. It may not be as useful for jobs running in Docker, since presumably each run of the job starts with a fresh slate by spinning up a new container.
NOTE: in G-CLI version 3.0+ this can be replaced with the built in ClearCache command.
Options
This G-CLI Tool has no options. You just call it directly from G-CLI.
Example
# G-CLI options ignored for simplicity.
# it is really this simple.
g-cli clearlvcache
Repository
For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/clearlvcache
vipc
Description
This tool allows you to apply a VI Package Configuration file (vipc) using G-CLI. You point to the vipc file and the version of LabVIEW and this tool will check to see if it needs applied and apply it if necessary.
Options
This G-CLI Tool has the following options:
| flag | parameters | required | description |
|---|---|---|---|
| --lv-version, -v | LabVIEW Version | no | LabVIEW version to install VIPC to. Defaults to internal version of VIPC file. If 64 bit LabVIEW you must specify. Example 22.3 (64-bit). |
| --timeout, -t | VIPM API timeout | no | time to wait for VIPM API timeout. Defaults to 300s |
Example
# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli vipc -- -v "22.3 (64-bit)" myvipcfile.vipc # Applies myvipcfile.vipc to LabVIEW 22Q3 64 bit
Repository
For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/vipc
lvbuildspec
Description
This tool allows you to build LabVIEW build specifications using G-CLI. It allows to take advantage of automatic version calculation, manually supply a build number or use the built-in autoincrement functionality. For a description of the auto-increment functionality, see this blog post
Options
This G-CLI Tool has the following options:
| flag | parameters | required | description |
|---|---|---|---|
| --project, --proj, -p | project path | yes | Project containing build specification |
| --buildspec, -b | build spec to build | yes | Specific build specification to build |
| --target, -t | target to build for | no | Target that contains build specification. Defaults to MyComputer |
| --version, -v | version | no | Version number x.x.x.x to assign to build. If left empty just uses whatever is currently set in the LabVIEW Project. --manualversion and -mv also work for backwards compatibility, but are deprecated and will eventually be removed |
Example
# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli lvbuildspec -- -p myproject.lvproj -b myexe -v 1.2.3.4 # builds my exe with version number set to 1.2.3.4
Repository
For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/lvbuildspec
vitester
Description
This tool runs a set of vitester tests and generates a JUnit report file.
Options
This G-CLI Tool has the following options:
| flag | parameters | required | description |
|---|---|---|---|
| --report, -r | report path | yes | Path to write JUnit report to. |
Example
# G-CLI options ignored for simplicity.
# it is really this simple.
g-cli vitester -- -r UnitTestReport.xml myproject.lvproj # this points to a project, you can also point to a Test Case or Test Suite Class
Repository
For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/vitester
lunit
Description
This tool allows you to run LUnit tests from the CLI using G CLI.
Options
This G-CLI Tool has the following options:
| flag | parameters | required | description |
|---|---|---|---|
| --report, -r, | report path | yes | Path to the report |
| --parallel | parallel | no | Flag to run tests in parallel |
Trailing Arguments are interpreted as paths. Typically this is the LabVIEW project you want to scon for tests to run.
Example
# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli lunit -- -r UnitTestReport.xml mylabviewproject.lvproj # Runs alll lunit tests found in mylabviewproject.lvproj
Repository
For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/lunit
vipb
Description
This tool allows you to VIPM Packages from the CLI using G CLI.
Options
This G-CLI Tool has the following options:
| flag | parameters | required | description |
|---|---|---|---|
| --buildspec, -b, | build spec | yes | Path to the vipb file to build |
| --version, -v | version | no | Version to build |
| --release-notes, -rn | release notes file | no | Path to a file containing the release notes |
| --timeout, -t | timeout | no | Timeout in seconds defaults to 1200 = 20 minutes |
Example
# G-CLI options ignored for simplicity.
# -- seperates the g-cli options from the options for this specific tool.
g-cli vipb -- -b MyBuildspec.vipb -v 1.2.3.4 # builds version 1.2.3.4
Repository
For more information and to see the source code, check out the repository here: https://gitlab.com/sas-gcli-tools/vipb
Third Party CLI Tools for other common commands
These are tools I didn't bother to implement G-CLI commands for, since they are already provided. You may find these useful. I'm not going to document them here, since the developers have already done that.
Setting Up Runners
In order to fully take advantage of CI, at some point you are going to need to set up a runner. For this I typically use GitLab, so these instructions are tailored to GitLab (I'll also include some info for GitHub users). On my blog I have a few posts about setting up GitLab Runners. It's spread over a few articles and a little disjointed. I'll attempt summarize it all in one place here. If you prefer video I have a video of me walking Casey through the process. It was made before this tutorial so it might not match up exactly.
How Many Runners Do I Need?
This is a common question. It is possible to have a single runner with multiple versions of LabVIEW installed. In the past multiple versions of LabVIEW have not always played well together so I recommend against this. I recommend one runner per version of LabVIEW or one per project if your projects have 3rd party drivers or other unique requirements. This keeps things better isolated. This can result in a lot of machines, so Virtual Machines are your friends here or Docker if you are brave, but that is a whole other can of worms. You can save yourself a little trouble by having a common Win10 or W11 machine that you copy as a starting point. You probably don't want to clone it - that can cause some problems. Better to just copy the VM Disk and create a new VM from it.
Assumptions
I'm making a few assumptions here. I am assuming you are using Windows (it should be similar for Win10 or Win11). I'm also going to be setting this up to use Git Bash instead of Powershell. I detest Microsoft and I don't know Powershell, but I do know Bash, so that is what I am going to use. If you want to use Powershell, the initial setup will be a little easier (because that is the default for GitLab shell runners on Windows). However you may have to tweak some of the yaml files and scripts. I am also assuming that you are using VIPM pro or the Community Edition. You need one of those in order for the vipc applier tool to work.
Starting Point
As a runner you will generally want a seperate machine from your dev machine. This could be a spare laptop or desktop, a VM running on your own desktop or server or a VPS running on something like Amazon, Vultr, or Digital Ocean. You'll need just enough horsepower to load your project and run a build. It will need access out to the internet to be able to ping gitlab.com (or if you are self-hosting, then whatever your url is for your GitLab server). You should consider keeping it on a seperate network isolated from your main network, because running a build basically opens up a reverse shell.
NOTE: If you are just starting out and dabbling with CI, an easy solution is to install GitLab Runner on your Development machine. It is much less than ideal, but lowers the barrier to entry and allows you to easily see what is going on. In that case you can probably skip the section on setting the script to run on startup and setting up autologon.
Access to Runner Machine
You will want to be able to logon to the runner and view the GUI for setup/installation and troubleshooting. For a physical machine that should be easy. RDP works best for remote machines, but VNC, MSP360 or Team Viewer will also work. Don't expose RDP or VNC to the internet. It is not secure. Run it through some encrypted tunnel like ssh, a vpn, wireguard, Nebula, etc.
Initial Software Installation
I'm assuming you are using Windows, so you will want to install the appropriate Windows Version and turn on Remote Desktop (if that is what you are using for access) You will want to install Git For Windows along with Git Bash. The default options should be fine. You will want to install NIPM and use that to install the proper version of LabVIEW, VIPM, DAQmx and any other drivers you need. You'll also want to install some basic VIPM packages such as G-CLI and the VIPM API. You'll also want to install whatever of the SAS-GCLI tools you want to use. If you have the Pro or Community edition of VIPM, then you can use the vipc applier tool to install packages as part of your CI pipeline. If you aren't doing that, then you should probably install the needed packages now. If you are using Python to run any build scripts or utilities, now is a good time to install that and any needed Python packages as well.
You really don't want to install a lot of extras. You are not going to be using this machine for anything other than running builds. Running a build basically opens up a reverse shell, so you don't want anything sensitive on there other than your code.
Initial Verification
You should manually open VIPM and LabVIEW and make sure no licensing dialogs pop up and take care of them if they do. You should verify that VIPM can connect to your version of LabVIEW by opening VIPM and going to the settings dialog box. You should also verify G-CLI is installed correctly and can connect to LabVIEW by typing g-cli echo -- "Hello World" into Git Bash and verifying that it returns "Hello World". If it doesn't, check your $PATH and make sure it contains the G-CLI directory. You may need to add an export line to your .bashrc file.
Setting Up GitLab Runner
Installing GitLab Runner
The instructions for installing the gitlab-runner program provided by GitLab are pretty good. Make sure you select shell as your executor when it asks. It will also ask you to give it a name and assign a tag to it. You can change these later through the web interface. Generally you'll want to tag a runner either with the name of a specifc project or based on the software versions installed. The tags are used to determine what jobs GitLab can run on this particular runner. You should just follow the instructions up until the point where it tells you to install it as a service. If you want to be able to actively debug and watch what is going on, you don't want to do this. So stop after step 5 and don't proceed to step 6.
Setting up GitLab Runner to use Git Bash Instead of PowerShell
NOTE: This section describes how to set up GitLab Runner to use Git Bash. If you want to use PowerShell you can skip this.
First, we need to help GitLab Runner find GitBash. Add a bash.cmd file into your user directory (the user that runs the GitLab Runner) with the following contents:
@"C:\Program Files\Git\usr\bin\bash.exe" -l
Next, we have to make sure that our starting paths are set up correctly with Linux-style formatting. In your config.toml file for your GitLab runner (it’s in the directory where you installed GitLab Runner), change the shell to bash and add Linux style paths for the builds_dir and cache_dir. That’s it. See the example below.
[[runners]]
name = "windows"
url = "https://your.server.name"
token = "YOUR_SECRET_TOKEN"
executor = "shell"
shell = "bash"
builds_dir="/c/gitlab-runner/builds/"
cache_dir="/c/gitlab-runner/cache/"
NOTE on Paths: We are using linux paths here, but when passing paths as parameters to the g-cli tools (ie the parameters after the -- ) you will need to use Windows Paths.
Running as a User
Create a batch file with the following line in it.
"C:\Program Files\Git\bin\sh.exe" --login -i -c "cd /c/Gitlab-Runner && ./gitlab-runner.exe run"
Save it somewhere convenient, like the Desktop and give it a meaningful name like "Start GitLab Runner.bat" Running this batch file will open a console window as seen below. Once a job starts, then you’ll see all the windows that it opens. You’ll see the LabVIEW splash screen and any VIs it opens. So if it hangs on some dialog, you will know about it.
Set to run at Startup
In order to run the script at startup (when the current user logs in), drop a link to into the Windows Startup Folder. You can find the Windows startup folder by hitting the windows key + R and typing shell:startup into the resulting dialog box.
Setting User to Autologon
The last thing I typically do is set the user to autologon, otherwise every time the machine boots up, you have to login to the machine before gitlab-runner can start. The way to do that in windows is to use netplwiz. Hit windows key + R and type in netplwiz. It will open a User Accounts window. You should see a checkbox at the top that says "Users must enter a user name and password to use this computer." Simply uncheck it.
NOTE: You may not see that checkbox. If you don't you can try adding a registry key. Copy the code below. Open a command prompt as administrator. Paste and run the code. Reboot. The box should show up now. If not, there are some other potential fixes, just google it.
reg ADD “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\PasswordLess\Device” /v DevicePasswordLessBuildVersion /t REG_DWORD /d 0 /f
At this point I usually reboot the runner and just sit and watch to make sure when it boots up it doesn't hang waiting for a Windows password and that the batch script runs on startup.
Verifying in GitLab
Verifying the Connection
If you log into your project and go to the CI/CD settings and the runner is up and running (ie you have ran the batch file we created), you should see it in the list of assigned runners.
]
Assigning Runners and changing tags
Once you have a runner up and connected to your gitlab server, it is easy to make changes to it through the web interface. You can change the name and the tags. You can also enable it for different projects (by default they are locked to a specific project, so you have to unlock it first). You can do all this by clicking on the edit button next to the runner to open the runner configuration editor.
Success!
Once you have done all the above steps, your runner should be good to go.
Sample Projects
Once you have a runner setup, the next step is to test it out. An easy way to do that is to use one of our sample projects. You can simply fork the project. Then make sure that the tag in the CI matches your runner and your runner is assigned to the project and it should be good to go. Make a change, commit, and push and it should kick off a pipeline. If you aren't using VIPM pro, you may need to manually apply the VIPC to the runner, but other than that it should work.
Why Bash scripts?
You'll notice in the sample projects I put all the commands in Bash scripts. Why is that? It gives us 2 things.
- It allows us to run the commands locally (ie
./build.sh) - It makes your CI more portable. Almost any CI system can run bash scripts. The difference between GitHub and GitLab is just in how we tell the system which script to run and when.
New to Bash?
I chose Bash because I am familiar with it (more than PowerShell) and because it is cross-platform. If you are getting started, I recommend purchasing this cheatsheet. The biggest takeaway is to make sure you add set -euo pipefail at the top of your bash scripts so the error handling and exit codes work correctly.
A Note On Variables
In the GitLab examples, you'll notice this line:
. <(sed -nr '/variables:/,$ s/ ([A-Z_]+): (.*)/\1=\2/ p' vars.yml)
What this line does is that it reads in the vars.yml file and takes the variables section and sets the variables there as environment variables. This is so we can define the variables once and use them when running locally or on the CI server (this file is included in the .gitlab-ci.yml file). The sed command is kind of black magic but it works. I got it here: https://unix.stackexchange.com/questions/539009/export-environment-variables-parsed-from-yaml-text-file