Note that the Origin Manager is unmaintained and considered mostly obsolete in its current form by https://www.suse.com/c/closing-the-leap-gap-src/
The primary function of origin manager is, as the name implies, to manage the origin of packages. In other words to keep track of from what project a package originates, submit updates, review requests to detect origin changes, and enforce origin specific policies like adding appropriate reviews.
The primary configuration for all parts of the tool is the OSRT:OriginConfig attribute to be placed on the target project. Once the config is created all relevant caches, listeners, and cron jobs will automatically include the project. The only exception is the origin-manager user should be added as a project reviewer in order to activate the review portion of the tool. Conversely either removing the config or locking the project will disable all services. Locking the project will preserve the caches used by the web interface, but not update them which allows for archived viewing.
Additionally, there are some other OSRT:Origin* attributes used to override the OSRT:OriginConfig outside of the target project and standard OSRT:Config relevant options.
The following core concepts are important for understanding the operation of the tool and its configuration.
The primary concept within origin manager is an origin which is simply a project from which a package may originate. The origin config will contain a prioritized list of possible origins. In order to determine the origin for a package each origin is searched in priority order for the package and a matching source revision. The first origin containing a match is considered the origin of the package.
An origin can change for a package without the source of the package changing in the target project in one of two ways. Either a devel project can be set on the target package in conjunction with an allowed <devel> origin or the package may change in a potential origin project. If the matching sources are accepted into a higher priority origin then the package will be consider as coming from the new origin. This automatic switching properly encapsulates the rather complicated interrelated product workflow.
Alternatively, if the OSRT:OriginConfig is changed that may also change a package's origin.
A workaround is a pseudo origin used to track a package that used to match an origin, but had a source change accepted that matched no origin. The change to a workaround requires human review via the fallback-group. If the source later shows up in the origin project it will automatically switch back to not being considered a workaround.
As noted above all configuration for the tool is read from OBS attributes.
Any target project that expects origin manager to manage it must contain this config. The config is expected to be valid yaml of the following format.
origins:
- origin1:
policy_option1: policy_option1_value
global_option1: global_option1_valueNote the origins are a list of single key-value pairs. The list allows the order to be maintained by yaml parsers over a simple key-value set, but makes for a bit more cumbersome markup. The following demonstrates the default config.
# Wait instead of declining an initial source submission that cannot be found in
# any origin.
unknown_origin_wait: false
# List of prioritized origins and their policy overrides.
origins: []
# User that will perform reviews (ie. user under which ./origin-manager.py will
# run). If not specified nor available from default reference 'origin-manager'
# will be used.
review-user: <config:origin-manager-review-user>
# Group from which a review will be requested as a fallback whenever automated
# approval is not possible.
fallback-group: <config:origin-manager-fallback-group>
# Fallback workaround project primarily used for migration to origin-manager.
fallback-workaround: {}The following is the default policy for an origin. The policy of the new origin is applied to requests and when determining the current origin (especially relevant to pending* options).
# Additional by_group reviews to be added to requests from this origin. Special
# keys supported are: fallback and maintainer. The fallback key will be replaced
# by the fallback-group global configuration value and the maintainer key will
# be replaced by the maintainer from the origin package (by_package review) or
# the devel project for the origin package if configured. Maintainer reviews
# are skipped if the request creator is a maintainer.
additional_reviews: []
# Submit updates for packages from this origin and allow forward (based on
# revision order) submissions without fallback review. If disabled updates will
# not be submitted automatically and manually submitted updates will require
# fallback review.
automatic_updates: true
# Submit new packages from this origin (highest priority, enabled, origin wins).
# New packages will only be submitted once. Any prior requests will block future
# submission of the package until it is accepted.
automatic_updates_initial: false
# Submit updates even if an existing request is still pending.
automatic_updates_supersede: true
# Delay update submission by a minimum of N seconds after a source change.
automatic_updates_delay: 0
# Cap updates at a minimum of N seconds since the last request.
automatic_updates_frequency: 0
# Always add a review for the package maintainer.
maintainer_review_always: false
# Add a review for the package maintainer on the initial submission of package.
maintainer_review_initial: true
# Approve source submissions that match pending submissions in the origin
# project of either the new state or with only allowed reviews
# (pending_submission_allowed_reviews). Otherwise, consider as originating from
# the project containing the pending request and wait for the conditions to be
# met. Disable to require submissions to be accepted before accepting review.
pending_submission_allow: false
# Consider pending submissions for the purposes of determining origin, but wait
# for requests to be accepted before accepting review.
pending_submission_consider: false
# List of reviews to allow for the purposes of accepting review when
# pending_submission_allow is enabled.
pending_submission_allowed_reviews:
# Non-maintenance projects:
- <config_source:staging>*
# Maintenance projects:
- <config_source:review-install-check>
- <config_source:review-openqa>
# Control how early pending requests in an origin project are considered for
# submission to the target project. Submit pending requests with a set of
# allowed reviews, but still wait for the above reviews before being accepted.
# For example !<config:review-team> could be added to require the review team
# approval before mirroring a pending request.
'pending_submission_allowed_reviews_update':
- '!maintenance_incident'The following special origins are supported where * denotes all and ~ indicates a workaround (explained later).
*: applied to all origins above it*~: applied to all workaround origins above it and generates them if they do not existprefix*suffix: describes a product family which is expanded into specific projectsorigin1~: describes a specific origin workaround<devel>: replaced with the devel project for a given package if set
To add an origin with a completely default policy use {} as shown below.
origins:
- projectFoo: {}As one may have noticed, both the global options and policy defaults contain config references denoted by <config:*> or <config_source:*>. References are replaced with the corresponding value from the OSRT:Config for the relevant project.
config: read from the target projectconfig_source: read from the source project (relative to a source change request)
If no value is found in the relevant config the value is replaced by a blank which in the case of lists removes the entry.
A reference may be used in conjunction with additional characters. This is useful for the pending_submission_allowed_reviews policy option which defaults to including <config_source:staging>*. This will allow any review that starts with <config_source:staging>. For example, when evaluated in the context of openSUSE:Leap:15.2 for a request from openSUSE:Factory this will be evaluated as openSUSE:Factory:Staging* which will allow staging reviews.
The reference abstraction is especially useful for cross product family origins. For example SLE can use openSUSE.org:openSUSE:Factory as an origin and automatically have the appropriate staging reviews ignored based on the cross-OBS-instance config.
When a package is considered a workaround for an origin a ~ will be appended to the origin. Workarounds, just like any origin are only allowed if they are found in the list of origins from the config. Workarounds may be configured with different settings, such as additional_reviews, than the non-workaround origin.
Workarounds of any origin may be allowed by adding *~ origin at the end of the list.
Instead of defining specific origin projects for an entire family the config supports family expansion. Note that this has become less useful since layered projects such as maintenance projects have their history stacked on parents for origin finding purposes. As such it is typically sufficient to include just the previous product's :Update project instead of the whole family.
Looking at the config from openSUSE:Leap:15.1 helps explain this feature best. The config contained the following family expansions.
SUSE:SLE-15*openSUSE:Leap:15*
These were expanded into the following.
- SUSE:SLE-15-SP1:Update
- SUSE:SLE-15-SP1:GA
- SUSE:SLE-15:Update
- SUSE:SLE-15:GA
- openSUSE:Leap:15.0:Update
- openSUSE:Leap:15.0
Note that the Leap expansion automatically did not include the target project nor its :Update project as they are considered above it. The *~ origin in the config meant that each of these was also duplicated for a workaround origin.
Family expansion can also be used with a suffix like what was done in openSUSE:Leap:15.1:NonFree.
- openSUSE:Leap:15*:NonFree
The above expanded into the following.
- openSUSE:Leap:15.0:NonFree:Update
- openSUSE:Leap:15.0:NonFree
The fallback-workaround global option is used as a fallback project which will be considered a workaround for another origin. This was designed for the migration to origin manager before which a :SLE-workarounds subproject was used to house workaround sources.
The option requires two keys to be set: project and origin. The project key is the project to search for matching sources and the origin is what origin of that will be used if matched. For example, openSUSE:Leap:15.1 used the following.
fallback-workaround:
origin: 'SUSE:SLE-15-SP1:GA~'
project: 'openSUSE:Leap:15.1:SLE-workarounds'The above config meant that if sources matched no origin, but matched those in openSUSE:Leap:15.1:SLE-workarounds the origin was considered SUSE:SLE-15-SP1:GA~.
The following attributes are provided to override their corresponding policy option. These attributes are searched for in the following order of precedence.
- origin package
- origin project
- target package
- target project
If no attributes are found the OSRT:OriginConfig policy values for the matched origin are used.
OSRT:OriginUpdateSkip:automatic_updates = falseOSRT:OriginUpdateSupersede:automatic_updates_supersedeOSRT:OriginUpdateDelay:automatic_updates_delayOSRT:OriginUpdateFrequency:automatic_updates_frequency
The OSRT:OriginUpdateSkip attribute can also be added to a target project to disable updates entirely, otherwise if the OSRT:OriginConfig includes at least one origin with automatic_updates then the project will be included in the update jobs.
For target projects containing origins with automatic_updates_initial enabled the OSRT:OriginUpdateInitialBlacklist attribute can be utilized to blacklist new packages from being submitted. The blacklist does not stop those packages from being submitted manually or updated if they are included in the target project. The blacklist only limits the new packages considered for initial submission.
The blacklist supports one entry per line where entries can be a python compatible regular expression and is placed on the target project to be applied to all origins. For example, the following config would be relevant for Leap which has automatic_updates_initial enabled for SLE, but does not want SLE specific packages.
.*SLE.*
.*SLED.*
.*SLES.*
.*suse-manager.*
.*spacewalk.*
kernel-livepatch-.*
system-role-.*
sca-.*
patterns-server-enterprise
unified-installer-release
A few options relevant to origin manager are standard options and thus available via the OSRT:Config attribute.
# Group whose members are allowed to issue override commands. The staging-group
# is also included regardless of what value is set.
originmanager-override-group = ''
# Minimum time in seconds to wait before reviewing source submissions. The
# default is 30 minutes with the intention of allowing parrallel submissions
# to origin projects and downstream projects that allow pending requests.
# Without the wait downstream projects can be reviewed before the upstream
# requests have been created. 30 minutes is also less than the 50 minutes that
# staging-bot waits which insure the quick strategy can still be utilized.
originmanager-request-age-min = 1800A CLI is provided for origin manager via the osc-plugin-origin package. See osc origin --help for complete reference. A few especially useful examples are included below.
The config command shows the expanded configuration for a given project. The --origins-only flag is useful for ensuring origin expansions are working as expected, but the full output is useful for ensuring policy overrides are working.
$ osc origin -p openSUSE:Leap:15.2 config --origins-only
<devel>
<devel>~
SUSE:SLE-15-SP2:GA
SUSE:SLE-15-SP2:GA~
openSUSE:Leap:15.1:Update
openSUSE:Leap:15.1:Update~
openSUSE:Leap:15.1
openSUSE:Leap:15.1~
openSUSE:Factory
openSUSE:Factory~
It can be helpful to understand the origin search path which can be done via the --debug flag on the package command.
$ osc origin -p openSUSE:Leap:15.2 package --debug adminer
[D] origin_find: openSUSE:Leap:15.2/adminer with source 3bd0d78 (True, True, True)
[D] source_contain: openSUSE:Leap:15.1:Update 16c519a == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1:Update 440c7f2 == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1:Update 440c7f2 == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1:Update 4c26333 == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1:Update eb15f78 == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1 16c519a == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1 440c7f2 == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1 440c7f2 == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1 4c26333 == 3bd0d78
[D] source_contain: openSUSE:Leap:15.1 eb15f78 == 3bd0d78
[D] source_contain: openSUSE:Factory 3bd0d78 == 3bd0d78 (match)
openSUSE:Factory
Which shows that the latest revision from openSUSE:Factory is the first match and thus the origin.
Understanding the potential origins of a package is also useful.
$ osc origin -p openSUSE:Leap:15.2 potentials kernel-source
origin version
Kernel:openSUSE-15.2 5.3.8
openSUSE:Leap:15.1:Update unknown
openSUSE:Leap:15.1 4.12.14
openSUSE:Factory 5.3.8
Knowning the origin history of a package can also be enlightening.
$ osc origin -p openSUSE:Leap:15.2 history kernel-source
origin state request
Kernel:openSUSE-15.2 declined 745644
Kernel:openSUSE-15.2 review 745323
Kernel:openSUSE-15.2 accepted 744385
Kernel:openSUSE-15.2 declined 743969
Kernel:openSUSE-15.2 declined 743419
Kernel:openSUSE-15.2 declined 743414
Kernel:openSUSE-15.2 accepted 742724
Kernel:openSUSE-15.2 accepted 737241
Kernel:openSUSE-15.2 accepted 733609
Kernel:openSUSE-15.2 superseded 733094
Kernel:openSUSE-15.2 superseded 733005
Kernel:openSUSE-15.2 accepted 732312
Kernel:openSUSE-15.2 accepted 731459
openSUSE:Leap:15.1:Update~ accepted 728396
openSUSE:Leap:15.1:Update~ accepted 725631
openSUSE:Leap:15.1:Update accepted 724147
openSUSE:Leap:15.1:Update accepted 721159
As with all ReviewBot based bots they can be debugged locally by using the --debug and --dry options. Origin manager provides detailed debug output for understanding both the origin search and policy evaluation. The following example demonstrates a request that is changing the origin from openSUSE:Leap:15.1:Update to openSUSE:Factory.
$ ./origin-manager.py --dry --debug id 746065
[I] checking 746065
[D] origin_find: openSUSE:Leap:15.2/virtualbox with source d79bf01 (False, True, True)
[D] source_contain: openSUSE:Leap:15.1:Update 1655653 == d79bf01
[D] source_contain: openSUSE:Leap:15.1:Update f642fe0 == d79bf01
[D] source_contain: openSUSE:Leap:15.1:Update 22bd811 == d79bf01
[D] source_contain: openSUSE:Leap:15.1:Update 48ab129 == d79bf01
[D] source_contain: openSUSE:Leap:15.1:Update dd5c97b == d79bf01
[D] source_contain: openSUSE:Leap:15.1 48ab129 == d79bf01
[D] source_contain: openSUSE:Leap:15.1 dd5c97b == d79bf01
[D] source_contain: openSUSE:Leap:15.1 da823e6 == d79bf01
[D] source_contain: openSUSE:Leap:15.1 28ae8a7 == d79bf01
[D] source_contain: openSUSE:Leap:15.1 0bc96d1 == d79bf01
[D] source_contain: openSUSE:Factory d79bf01 == d79bf01 (match)
[D] origin_find: openSUSE:Leap:15.2/virtualbox with source 1655653 (True, True, True)
[D] source_contain: openSUSE:Leap:15.1:Update 1655653 == 1655653 (match)
[D] policy_evaluate:
# policy
additional_reviews: []
automatic_updates: true
automatic_updates_delay: 0
automatic_updates_frequency: 0
automatic_updates_initial: false
automatic_updates_supersede: true
maintainer_review_always: false
maintainer_review_initial: true
pending_submission_allow: false
pending_submission_allowed_reviews:
- openSUSE:Factory:Staging*
pending_submission_allowed_reviews_update:
- '!maintenance_incident'
pending_submission_consider: true
# inputs
direction: unknown
higher_priority: false
new_package: false
origin_change: true
pending_submission: 'False'
same_family: false
PolicyResult(wait=False, accept=True, reviews={'fallback': 'Changing to a lower priority origin.'}, comments=[])
[D] skipped adding duplicate review for origin-reviewers
[D] broadening search to include any state on 746065
[D] no previous comment to replace on 746065
[I] 746065 accepted: origin: openSUSE:Factory
origin_old: openSUSE:Leap:15.1:Update
[D] 746065 review not changed
From the PolicyResult it is clear that the review will be accepted with the addition of a review for the fallback group since Changing to a lower priority origin..
There are a two commands that can be used to control the review bot from an OBS request. The commands follow the standard reply syntax @<user> <message> and can be placed in the request description or a comment on the request. Commands issued by users not part of either the staging-group or originmanager-override-group will be ignored. In order to be accepted commands must match the following regular expression.
^@(?P<user>[^ ,:]+)[,:]? (?P<args>.*)$The standard override command is available to force acceptance when the submission does not match any allowed origin, but this should be avoided if possible. The unknown_origin_wait global option can be enabled to make it easier to utilize the override feature.
The origin manager specific command change_devel is also available with an expanded user pool to include the request creator. The command tells origin manager to consider the origin as a devel project from the request source project. Alternatively, a different origin project can be optionally specified.
For example, a request originating from projectDevel would consider the origin for review purposes as if the devel on the package was set to projectDevel using the following command.
@origin-manager change_devel
The same request would consider the origin for review purposes as if the devel on the package was set to projectFoo using the following command.
@origin-manager change_devel projectFoo
If a source submission is accepted, but during update an origin cannot be matched the annotation on the most recent request will be checked to see if it was considered as devel project. If such an annotation exists a change_devel request will automatically be created to match the annotation. After the request is accept future updates will be submitted as appropriate.