[QUESTION] apiops strategies for multiple teams and repos, scaling ?

[detox3r]

REPOS & MULTIPLE TEAMS

Since apiops relies on gitops. what would be a good strategy to be applied to a business that has more than 100 teams each with several apis each and 4 environments ?.

My thought process would be something like this:

1. Have 1 repo per team with apiops in place (so operators people can be off hands)
2. Have 1 repo that has apiops in place ( that allows management and promotion of all 100 apis) - self centered approach
3. have 1 main repo that has all apis control (for operators) and also have 1 repo per team with apiops in them (kinda conflicts with gitops approach because overwriting api code)

I found something in the apiops docs but is kinda not clear.

does this mean the recommendation is to have option 3 but instead of extracting it into the main repo ( the one operators control) it will actually be extracted to the other teams repo ?... kinda confused there

[github-actions]

  Thank you for opening this issue! Please be patient while we will look into it and get back to you as this is an open source project. In the meantime make sure you take a look at the [closed issues](https://github.com/Azure/apiops/issues?q=is%3Aissue+is%3Aclosed) in case your question has already been answered. Don't forget to provide any additional information if needed (e.g. scrubbed logs, detailed feature requests,etc.).
  Whenever it's feasible, please don't hesitate to send a Pull Request (PR) our way. We'd greatly appreciate it, and we'll gladly assess and incorporate your changes.

[waelkdouh]

It's up to you to decide how you wanna structure your repositories. Apiops will be more than happy to extract your artifacts into the repository of your choice. Having said that we had a recommend that you take a look at work Spaces in the future please keep in mind that we have an updated APIops to support workspaces yet and we don't have a timeline for now. Also as the documentation discusses, you can always use Gore to ensure that certain artifacts are not extracted into specific repositories that you specify. Also, remember that we do provide you with the ability to utilize the configuration, extractor file where you can specify specific artifacts that you can extract them into specific repose. Referred to the wiki for more details around that.

[sa-pietronegri]

one question here as I could not find a way to create brand new questions. Sorry for that, I am very new to these procedures.

Generally speaking, how could we support a scenario in which in dev 2 apis (api-1, api-2) were updated but only 1 (api-1) needs to be promoted? I thought by using the config extract.
So I put api-1 in the config extract and I saw I could "extract" one specific api, api-1, and create PR-A.

Now, my prod apim is already in place and tracked in the git repository. When I look to PR-A this is proposing to delete all the other Apis (which are missing in the branch created by the run - extract pipeline) and update api-1. If I understand this correctly, this will give me in the end a prod apim with api-1 updated and all the other ones deleted.
What I would have expected was to keep all the other apis and update api-1. I hope I made myself clear.

[guythetechie]

Hi @sa-pietronegri:

The extractor pipeline does a few things:

• ...
• Downloads extractor
• ...
• Runs extractor
• ...
• Creates an artifact that includes the extractor changes
• ...
• Creates a PR with the extracted changes.

The PR step ensures that your Git repo is in sync with your APIM instance, adding or deleting resources as needed. That's not what you want though; you're only interested in selecting specific changes. I would:

• Clone my artifacts git repo on my local computer.
• Download the artifact generated by the extractor prior to the PR step.
• Copy/paste the parts of that artifact I want into my local git repo.
• Push my git changes.
• Abandon the autogenerated extractor PR.

Hope this helps.

[sa-pietronegri]

Hi @guythetechie,
this helps indeed, thanks a lot!
I honestly thought about this approach, but it seemed to me -how can I say- a bit error prone.
I hoped I would not have to touch the code at all during the promotion process.
But it makes sense and I will give it a try. Thanks again :-)

[sa-pietronegri]

hi @guythetechie I was trying to publish 3 named values with the approach you mentioned.
I get in the "Run publisher for Dev environment" task the message:
System.InvalidOperationException: Failed to get files for commit *** in directory /home/vsts/work/1/s/artifacts. Error message is 'fatal: cannot change to '/home/vsts/work/1/s/artifacts': No such file or directory

but the 3 named values files are in the last commit...

they are 3 KV values with overriding in the config.prod.yaml

What could be gone wrong here? thanks in advance

Ps. Afterwards I tried with an akv secret with no overriding, but I got the same kind of error

[sa-pietronegri]

that was happening because I named the folder wrongly ("aritfacts" rather then "artifacts") -.-

[anish2808]

REPOS & MULTIPLE TEAMS

Since apiops relies on gitops. what would be a good strategy to be applied to a business that has more than 100 teams each with several apis each and 4 environments ?.

My thought process would be something like this:

1. Have 1 repo per team with apiops in place (so operators people can be off hands)
2. Have 1 repo that has apiops in place ( that allows management and promotion of all 100 apis) - self centered approach
3. have 1 main repo that has all apis control (for operators) and also have 1 repo per team with apiops in them (kinda conflicts with gitops approach because overwriting api code)

I found something in the apiops docs but is kinda not clear.

does this mean the recommendation is to have option 3 but instead of extracting it into the main repo ( the one operators control) it will actually be extracted to the other teams repo ?... kinda confused there

In a similar situation. I would like to get some clarity on this as well.

[waelkdouh]

@anish2808 workspaces will be the feature to look forward to once we implement it in apiops.

[anish2808]

@anish2808 workspaces will be the feature to look forward to once we implement it in apiops.

@waelkdouh - What is the recommended approach until Workspaces becomes GA?
Thinking from the perspective of how to run smoothly until then and also how efficiently can we pivot the strategy to using workspaces once it becomes available on Azure and ApiOps.

Taking a mono-repo approach where multiple teams work on the same repo becomes complicated quickly. And it has to be a manual process of extracting artifacts and publishing them to other environments through pipelines.
I'm leaning towards separating out the repos/pipelines for each individual team makes things easier in the meantime. But it defeats the principle of GitOps.

A video walkthrough would be beneficial for us where you showcase a recommended procedure of supporting multiple teams currently.

[waelkdouh]

We don't know what the final shape of workspaces will be so I can't speak to that yet. In the meantime, you can check out this discussion.

[DibyaRanjan1]

Hi @sa-pietronegri:

The extractor pipeline does a few things:

• ...
• Downloads extractor
• ...
• Runs extractor
• ...
• Creates an artifact that includes the extractor changes
• ...
• Creates a PR with the extracted changes.

The PR step ensures that your Git repo is in sync with your APIM instance, adding or deleting resources as needed. That's not what you want though; you're only interested in selecting specific changes. I would:

• Clone my artifacts git repo on my local computer.
• Download the artifact generated by the extractor prior to the PR step.
• Copy/paste the parts of that artifact I want into my local git repo.
• Push my git changes.
• Abandon the autogenerated extractor PR.

Hope this helps.

Hi @guythetechie , @waelkdouh

I am facing the same issue and your proposed solution did not help me. Here is my scenario.

1. I have extracted all API metadata data in dev without configuring the Publisher pipeline. I have committed the changes to the Main branch. Intially I do not want to run the publisher pipeline as It will override all the API in QA.
2. Now I configured the Publisher pipeline setup with no prior run.
3. My Dev APIM instance has 1000 api, QA has 800 and prod has 500 API. It is common in the organization that people develop, test and then only releases to Prod. That's why prod has less API.
4. Now I want to extract one API from dev and deploy to QA. However the main branch already has the metadata of the API. As it is already pushed from starting to the main branch, git is not taking as a modification file. How can I include this file in the git commit, there is no modification to this file but It is only present in APIM dev.

I tried git rm --cached apim_artifacts/apis/apiminitiativeprivate command and
git add git add apim_artifacts/apis/apiminitiativeprivate but git thinks ,It is an unmodified folder and did not allow me to commit it.
5. I tried all possible ways but nothing helped. I have cloned the main branch to a feature branch. My aim is to submit a PR merge to the Main branch with only one API metadata that I want to deploy to QA and prod. The only workaround is to go to all the required files and make some dummy changes. So that git will include the change as a modified folder/file.

I don't need to run the extractor pipeline here as the metadata is already present in the main branch.
Could you find any other way to include the folder/files and submit a PR to the main branch to make the publisher pipeline run.

[Mohid-A]

Hi @waelkdouh I thought of using the same thread since it is open instead of creating a new question on the workspaces.

Is there any additional update on the workspace support availability?

I agree with others, in an environment with 10's of APIs it's becoming challenging to deal with moving changes to high environments, where we want to push changes for some apis and not all as part of commit-based push. Can you guys please advise what is the viable solution for now?

Thank you and we appreaciate you guys.

[waelkdouh]

We are currently in the process of adding support for workspaces in apiops. Please keep in mind that workspaces feature itself is still public preview so we are usually careful with rushing implementation for public preview features as they are subject to certain limitations and changes. So please stay tuned.