Stop Putting Your CI/CD Code in YAML

Stop putting programming logic in YAML

We are developers. We need to stop putting our CI/CD logic in YAML.

Yes, almost everyone does it this way. Yes, github and gitlab (and all the others) want to lock you into their particular syntax so you can’t jump ship easily.

But it’s wrong.

• it’s hard to run locally.
• you’re a programmer, but YAML isn’t a programming language.
• you’re on the wrong side of history.

Hard to run locally

If you saw a colleague writing Java and then, instead of running locally, they:

• git add
• git commit
• git push
• log into CI/CD web app
• look to see if change worked

…you would laugh at them. This would be an insane way to develop with programming language. The modern equivalent of using punched cards and waiting overnight to find out if it worked.

(Or image if they did all their development directly in the CI/CD web app? Eek.)

But it’s what I see happen over and over again at my day job when developers work on CI/CD pipelines.

Yes, there are ways to run locally. And if you can use them, you definitely should. Because running things locally is a better experience than committing, pushing, and waiting.

But at my corporate job, most developers use Windows and connect via VPN. Running a docker image to execute a pipeline locally is very difficult in this environment and getting hold of a docker desktop license is tricky. And my suspicion is it is difficult for many corporate developers for similar reasons.

So I get to see developers become YAML jockeys.

Don’t do this:

- if [[ ! -z "ARTIFACT_REPO_TOKEN" ]];then echo "set ARTIFACT_REPO_TOKEN";exit 1;fi
- if [[ ! -z "ARTIFACT_REPO_URL" ]];then echo "set ARTIFACT_REPO_URL";exit 1;fi
- if [[ ! -z "CODE_SCAN_TOKEN" ]];then echo "set CODE_SCAN_TOKEN";exit 1;fi
- if [[ ! -z "PATH_TO_DEPENDENCIES" ]];then echo "set PATH_TO_DEPENDENCIES";exit 1;fi
- if [[ ! -z "DOTNET_VERSION" ]];then echo "set DOTNET_VERSION";exit 1;fi
- export PATH=$PATH:/path/to/dotnet/sdk/$DOTNET_VERSION
- dotnet restore
- dotnet build

If you build it this way, every bug you have and every change you make has to be tested up in your CI/CD runner.

You’re a programmer

Instead, build and test your CI/CD in your IDE on that powerful development machine you have. When you push it up, the command that ran locally should also run correctly remotely. Put all that logic in a build tool (e.g. make, Bazel, or literally one of hundreds of others). Or into a container. Or even into a script. Or a combination of scripts and build commands.

Rules of thumb:

• if it has an “if”, put it in a script
• if it has sequencing, put it in a build tool
• if it has dependencies, put them in a container

And above all:

• if your CI/CD stage has more than one command, refactor those steps into a script/build tool/container.

e.g. any of the following are reasonable ways to build your app locally and not put your logic in YAML:

- make build

or

- docker run myapp

or

- build.sh

Just make sure it’s a real programmers tool and not YAML.

Your debug cycle time will be faster and you won’t end up with 67 ugly failed builds cluttering your pipeline history.

If you use YAML for programming logic, you’re on the wrong side of history

I lived through programming in the early 00’s when XML configuration was the new hotness and every enterprise developer wrote more XML than Java. This was not a fun time. I know of no serious developer who thinks “let’s go back to programming in XML, it was pretty good.”

20 years from now, no one will be saying “let’s go back to programming in YAML , it was pretty good.”

• rant
• yaml
• ci/cd
• how to do it