How to specify promotionTemplate for each warehouse separately in the same stage?

[ddeath]

We would like to have multiple microservices in 1 project with shared stages but different promotionTemplates

Lets assume we have 2 service

• template-service
• testing-playground

and we want Freight deploy to int (integration) stage:

If the each service is sitting in different repository, it will need to update helm values.yaml files in different repo for each service. But I dont see an option how to specify different promotionTemplate for each Warehouse (or freight coming from different warehouse) and looks like I cannot say that step A should be executed only for Freight coming from Warehouse B

apiVersion: kargo.akuity.io/v1alpha1
kind: Stage
metadata:
  name: int
  namespace: testing-playground
spec:
  requestedFreight:
    - origin:
        kind: Warehouse
        name: Warehouse_A
      sources:
        direct: true
    - origin:
        kind: Warehouse
        name: Warehouse_B
      sources:
        direct: true
  promotionTemplate:
    spec:
      steps:
        # How to make sure, this will get executed only for Warehouse A Freights?
        - uses: git-clone
          config:
            repoURL: https://gitlab.com/warehouse_A_service.git
            checkout:
              - branch: main
                path: ./src
        - uses: helm-update-image
          config:
            path: ./src/services/warehouse-a-service/values/values.yaml
            images:
            - image: registry.gitlab.com/warehouse-a
              key: imageTag
              value: Tag
        - uses: git-commit
          config:
            path: ./src
            message: Testing Deployment
        - uses: git-push
          config:
            path: ./src

        # How to make sure, this will get executed only for Warehouse B Freights?
        - uses: git-clone
          config:
            repoURL: https://gitlab.com/warehouse_B_service.git
            checkout:
              - branch: main
                path: ./src
        - uses: helm-update-image
          config:
            path: ./src/services/warehouse-b-service/values/values.yaml
            images:
            - image: registry.gitlab.com/warehouse-b
              key: imageTag
              value: Tag
        - uses: git-commit
          config:
            path: ./src
            message: Testing Deployment
        - uses: git-push
          config:
            path: ./src

Alternative approach would be to create 2 different stages and point each warehouse to different stage, but then I would need to have stages:

• INT_warehouse_A
• INT_warehouse_B

Since having 2 stages named INT would be conflict in k8s crds

[krancour]

It wouldn't necessarily be wrong to have two "int" stages. I think it looks wrong because your naming scheme is odd.

If you have two microservices, we'll call them "foo" and "bar," then you can have Warehouses named foo and bar that will each discover all artifacts (images and manifests) for one of those or the other and bundle them up as Freight whenever new artifacts are found.

You can then have two stages, quite reasonably named foo-int and bar-int. These Warehouses and Stages would essentially be the beginnings of two parallel pipelines for your two microservices that are, in all ways, independent of one another.

This is one way of doing things.

Now suppose that although "foo" and "bar" are separate services, there's some tight coupling between them and when you roll out changes to one, you usually have to roll out complementary changes to the other, in tandem. In this case, you would not wish to move artifacts for one service and the other through your pipeline(s) independently of one another. In a case such as this, perhaps you make a single foobar Warehouse subscribed to multiple git repos and multiple image repos. Each piece of Freight produced by that Warehouse would now contain all four artifacts required for the two services. You could create stages like foobar-int, foobar-prod, etc. (Or even just int and prod if there are no other services in the Project.) Artifacts for both services would move from one end of the pipeline together as a unit in this case.

Still, another possibility is one that looks like your screenshot where you have two Warehouses producing Freight for "foo" and "bar" separately from one another and you have Stages like int, prod, etc. that will use both of those, but won't require "foo" artifacts and "bar" artifacts to move through the pipeline as a unit. You could freely mix-and-match any version of "foo" and any version of "bar" in a single Stage. However, in this case, the first option I laid out accomplishes the exact same thing whilst being substantially less complicated. (There are legitimate cases where it makes sense for Stages to get Freight from multiple Warehouses, as in your screenshot, but the use case you've described isn't one of them. I won't digress further.)

So... not that I recommend the last approach, but zeroing in on it, since that's what you asked about, and circling back to your original question -- you don't need a second promotion template. You need to define your promotion process differently. "Instead of clone this repo and do this, this and this," it is more like, "Clone these two repos and do this and that, etc."

Bottom line is that you have a lot of options and there isn't any one-size-fits all answer. You need to design a pipeline (or pipelines) that meets your needs.

Does this help?

[ddeath]

Yeah it make sense. For our need this will kind of do:

I think I was just confused and trapped in certain way of thinking. For us we want to have stage which would represent dev as an environment (as a location where you are deploying microservices) so it kind of does not make sense to have different stages because we dont have different locations for dev env. Everything runs in the same "location", same env.

Then I also saw this in documentation:

Stages may also request Freight from multiple sources. The following example illustrates a Stage that requests Freight from both a microservice-a and microservice-b Warehouse

apiVersion: kargo.akuity.io/v1alpha1
kind: Stage
metadata:
  name: test
  namespace: kargo-demo
spec:
  requestedFreight:
  - origin:
      kind: Warehouse
      name: microservice-a
    sources:
      direct: true
  - origin:
      kind: Warehouse
      name: microservice-b
    sources:
      direct: true
  # ...

So in the example in the docs https://docs.kargo.io/concepts the above yaml will result in situation shown as on picture in my original post. But since those microservice-a and microservice-b are "independend" and just sits in the same location (env) I was confused because based on the example it looked like we should have 1 stage, but it does not really supports those multiple promotion templates so it just added to confusion....

Anyway just wanted to explain my thinking a little bit.

After reading your answer I got a little bit different perspective and those multiple "pipelines" will work for us but I think it is not ideal and could be improved on it. But for that I will open different discussion.

[krancour]

Stages could be "environments," but we don't encourage people to look at it that way. We actually avoid the term "environment," wherever we can because it's too overloaded. Its meaning is different depending on your perspective.

If you are an infra person, to you, "environment" might means a specific cluster or region. If you are a developer, "environment" might be a specific instance of your app. i.e. When someone says to you "QA" is down, you probably don't jump to thinking the QA cluster is down. You probably assume they mean your app is down IN QA.

So we don't encourage this stage == environment way of looking at things.

Our documentation suggests that stages be thought of an instance of an application or service with a discrete purpose rather than aligning stages directly with a location.

[ddeath]

🤔 Now when I am reading it again after some time. You are saying that how you interpret the stuff depends also on who you are - infra person, developer or someone else.

So the question could be: Is it ok to have different view for each persona? Because developers "usually" think about env as place where you are deploying stuff, multiple microservices sits in same env. So would it make sense to allow them to use it like that?

I assume that developers and infra guys will have their own projects and there is not much need for developer person to be looking at infra project and vice versa 🤔

[krancour]

You're overcomplicating this. We don't need different views when it's already up to you to define "Stage" however you see fit. It is a purposely generic term.