Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ubuntu/jammy #4286

Merged
merged 84 commits into from
Jul 25, 2023
Merged

Ubuntu/jammy #4286

merged 84 commits into from
Jul 25, 2023

Conversation

TheRealFalcon
Copy link
Member

@TheRealFalcon TheRealFalcon commented Jul 24, 2023

Includes:

Fixed up patches to be quilt compatible and include changelog entries.

igalic and others added 30 commits May 24, 2023 13:06
apply more ignorance.

Sponsored by: FreeBSD Foundation
Since the config-refresh_rmc_and_interface module was removed in canonical#2148,
we also need to update the reporting test.
…nonical#4128)

Some module names in cloud.cfg.tmpl use minus signs, others use
underscores. Standardise on only using underscores for module names
as this aligns with modules using underscores, rather than minus
signs for the names of their settings.

Also remove the blank line created at the top of the resultant
cloud.cfg file - strictly speaking it may not be valid to start
a YAML file with a blank line.
The heuristic is assuming that the URL will contain an IP, and that the
route explicitly lists that IP (eg: 0.0.0.0/0 should match but doesn't).
In order for the heuristic to be 100% reliable, it would have to
replicate exactly what the system is doing both in terms of DNS and
route resolution.

Because the HTTP request below is already exercising the python and
system resolution, it is simpler to just remove this check and lean on
the HTTP request to provide the answer if the network is up or not.
…nonical#4153)

Commit e0a96d7 standardized config module names as underscore-delimited.

Correct config module names in cloud.cfg.tmpl: salt_minion and set_password
Also update docs for all config module references.
Update integration test for expected skip on salt-minion to salt_minion

Co-authored-by: Alberto Contreras <[email protected]>
Co-authored-by: dermotbradley <[email protected]>
- remove Hash key redirection
- copy basic multi-distro config from cc_ntp
- add optional installation
- enable and start services
- on BSD: disable Base system syslog

Sponsored by: The FreeBSD Foundation

Fixes canonicalGH-3250, canonicalGH-4118
LP: #1798055
…ical#2157)

OpenBSD: Added support for static routes
FreeBSD: Fixed creation of static routes

Corrected syntax in usage of route(8)
Added the needed space separated list of route names in the /etc/rc.conf-variable "static_routes"
Currently ephemeral.py uses hardcoded iproute2
calls to setup ephemeral networking, which is
incompatible with BSDs. Refactor to support
pluggable network interface operations.
why wasn't this failing before?

Sponsored by: The FreeBSD Foundation
…anonical#4161)

When building untagged commits from tip of main, brpm uses
git describe to create a versioned subdirectory.

For untagged commits, limit git describe hash to 8 characters to
align with the expected hash length of 8 that rpmbuild expects.

This avoids missing file errors during %prep stage resulting in
the following type or build error:
  + cd cloud-init-23.2-15-g95364bbf7
  /var/tmp/rpm-tmp.S7mQ7Y: line 40: cd: cloud-init-23.2-15-g95364bbf7
  : No such file or directory
If the minion is launched in masterless mode, then it should not be
launched as a daemon.

Signed-off-by: Denis Halturin <[email protected]>
This limited list was put into place, because in the past, something
wasn't working right. Whatever it was, seems to be resolved. Removing it
means BSDs have the same chance on all clouds like other distros do.

People can always create images that customize the selection, but the
out of box experience should be that cloud-init works as advertised on
all supported distros and all available clouds.

Sponsored by: The FreeBSD Foundation
…anonical#4162)

netplan 0.106 introduced PermanentMACAddress matching for networkd
configuration. LXD containers us veth interfaces which cannot
be matched by PermanentMACAddress.

Fix test_wakeonlan to prefer match by macadddress in network-config
for lxd_vm platform and prefer match by name on lxd_container.

Leave both test types in place to validate netplan plumbing for
both cases.
The regular expressions used to parse BSD mount(8) are both
overly complicated and limited.

- Combine the two regular expressions into one
- clarify regular expressions with named groups
- add parsing of mount options
- Attempt to extract mount-parent sub-path finding code from  parse_mount_info()
- Failing that, duplicate it in parse_mount()

This leaves parse_mtab() as only mount info provider without options parsing, for now.

Sponsored by: The FreeBSD Foundation
netplan 0.106 networkd match clause of PermanentMACAddress prevents
networkd from matching virtual network devices when setting up config.

Since LXD containers use virtual NICs, routes are not brought up.

Change test_gh668 to match by name: eth0 on lxd_container, retain
macaddress matching for lxd_vm.
Currently, network manager is disabled on c9s and therefore sysconfig is used as the primary renderer for network configuration. We do not want to change this for c9s even when network-manager renderer is re-enabled as it would mean a big behaviour change for cloud-init in the centos 9 stream.

This change bumps up the priority for sysconfig renderer so that it is used as the primary renderer on c9s and other downstream distributions derived from it. In the next major centos stream release, we may use network manager as the default renderer and make changes accordingly.

RHBZ: 2209349

Signed-off-by: Ani Sinha <[email protected]>
Implement ephemeral networking for BSD

After canonical#2127 lay the foundation, we now implement the BSD side of this

Sponsored by: The FreeBSD Foundation
…l#4184)

pytest 7.3.2 changed treatment of `testpaths` config which forced
test/unittest/conftest.py module load in our integration-tests
toxenv. Upstream bug is tracking their planned revert/fix for this
regression. Pin 7.3.1 in the meantime.

Fixes canonical#4183
)

While setting up the ephemeral network config for fetching the IMD at
init-local timeframe on multi-NIC instances, fulfilling:

- All NICs without carrier flag or more than one NIC with carrier flag
- systemd's predictable interface names is enabled

net.find_fallback_nic could select a NIC that is not the primary NIC,
leaving the instance without access to the network, as on GCE, only the
primary NIC can talk to the IMDS.

At this point in time, there is yet not access to
{instance,vendor,user}-data. Thus, it is tricky to dynamically inject a
breadcrum pointing to the primary NIC.

Two actions have been taken to fix this situation on the
DataSourceGCE:

1. Substitute eth0 with ens4 as the default primary NIC candidate.
2. Try through the list of candidate NICs and use the first that can
reach the IMDS.
jfroche and others added 20 commits July 18, 2023 16:37
The currently used dhcp client, dhclient, is coming from the unmaintained package, isc-dhcp-client (refer https://www.isc.org/dhcp/) which ended support in 2022.

This change introduce support for the dhcp client, udhcpc, from the busybox project. Busybox advantages are that it is available across many distributions and comes with lightweight executables.
…g errors (canonical#4234)

git describe may not return version/tags in the format that the read-version
tool expects. Make the tool robust so that it can gracefully handle
version strings that are not in the regular format.
We use regex to capture the details we care about, but if we cannot find them,
we won't traceback and will continue to use version and version_long as
expected.

Signed-off-by: Ani Sinha <[email protected]>
This simplifies the documentation and improves the hierarchy
under Development.

Fixes canonicalGH-4242
Clarify that the new datasource doc page has to be referenced in
doc/rtd/reference/datasources.rst to be rendered.
The cloud-init template for Ubuntu currently includes membership in
numerous groups. Many of these are groups that were removed from the
default list over a decade ago on the desktop, and which are better
served by udev rules today. Specifically: audio, dialout, floppy,
netdev, plugdev, video.

LP: #1923363
…new mode (canonical#4250)

By default, the cloud init log file is created with mode 0o644 with
`preserve_mode` parameter of `write_file()` set to False. This means that when
an existing log file is found, its mode will be unconditionally reset to the
mode 0o644. It is possible that this might cause the change of the mode of the
log file from the current more stricter mode to a less strict mode
(when the new mode 0o644 is less strict than the existing mode of the file).

In order to mitigate the above issue, check the current mode of the log file
and if the current mode is stricter than the default new mode 0o644, then
preserve the current mode of the file.

Fixes canonicalGH-4243

Signed-off-by: Ani Sinha <[email protected]>
crypt is deprecated in Python 3.11 and will be removed in 3.13.
This uses the passlib module to replace it. passlib is a
pure-Python package.

See: https://passlib.readthedocs.io
testing: Clear all LRU caches after each test

Some unit tests were missing necessary mocks because a previous test
had mocked the behavior and mock stuck around because of an LRU cache.
This means that if tests got run out of a specific order, they would
fail due to missing mocks. Because of this, a new autouse fixture was
added to automatically clear any lru cache after every test.

Additionally, updated unit tests accordingly so that they have the
necessary mocks, and convert `test_create_users.py` to pytest as
I was getting some transient errors from it.
Currently >2 keys in NoCloud datasource are not
supported. Previously 250280 attempted to correct
no keys being supported, however that fix was only
partial since more than one key breaks datasource
detection.

Additionally add test coverage.

Fixes canonicalGH-4271
…cal#4275)

Mock out the `which` checks for presence of ovs-vsctl cmd on system.
This prevents test leaks which eventually call subp ovs-vsctl commands.
The crypt module suffices on old versions of Python.
Use it when available. New passlib dependency will 
be required only with Python version > 3.12.
It was previously ignoring keys with numbers in them.

Additionally, update tests so that they don't have to be patched
downstream any time we add a new feature flag. Since the whole point
of the feature flags is to create overrides, that seems like a lot
of needless churn.
@holmanb
Copy link
Member

holmanb commented Jul 25, 2023

@TheRealFalcon want to pull in this fix too?

@TheRealFalcon
Copy link
Member Author

want to pull in #4292 too?

I don't think it's really necessary to fix dailies. It can make its way in through some future upstream snapshot. Hotfix will happen on a separate branch, so it's not really necessary here.

Copy link
Member

@holmanb holmanb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like build-package fails for a couple of reasons. The control file needs a comma (see inline), and it doesn't like the patch.

build-package
Preparing worktree (detached HEAD 9d16934d0)
HEAD is now at 9d16934d0 update changelog
Found UNRELEASED in debian/changelog. Do you really want to build-package ? (y/n)  y
pkg_name=cloud-init pkg_ver=23.2-0ubuntu0~22.04.2 upstream_ver=23.2 suite=UNRELEASED
wrote ../dl/cloud-init_23.2.orig.tar.gz
========================================
 Using orig tarball in ../dl/cloud-init_23.2.orig.tar.gz
048f28779fea6f163331afc13bd2e60a9d8440ad5b68c9e886bd798bc51987aa  ../dl/cloud-init_23.2.orig.tar.gz
========================================
Proceed with build-package ? (y/n)  y
building with: -d -S -nc
 dpkg-buildpackage -d -us -uc -ui -S -nc
dpkg-buildpackage: info: source package cloud-init
dpkg-buildpackage: info: source version 23.2-0ubuntu0~22.04.2
dpkg-buildpackage: info: source distribution UNRELEASED
dpkg-buildpackage: info: source changed by James Falcon <[email protected]>
 dpkg-source --before-build .
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying expire-on-hashed-users.patch
dpkg-source: info: applying retain-netplan-world-readable.patch
dpkg-source: info: applying retain-old-groups.patch
dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -E -b -B .pc/retain-old-groups.patch/ --reject-file=- < wtd/debian/patches/retain-old-groups.patch subprocess returned exit status 1
patching file config/cloud.cfg.tmpl
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored
dpkg-source: info: the patch has fuzz which is not allowed, or is malformed
dpkg-source: info: if patch 'retain-old-groups.patch' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: info: if the file is present in the unpacked source, make sure it is also present in the orig tarball
dpkg-buildpackage: error: dpkg-source --before-build . subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -d -us -uc -ui -S -nc failed
FAILED: debuild -d -S -nc

@@ -23,6 +23,7 @@ Build-Depends: debhelper-compat (= 13),
python3-setuptools,
python3-yaml,
python3-responses
python3-passlib
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comma after python3-responses is required

@TheRealFalcon TheRealFalcon merged commit ff72c61 into canonical:ubuntu/jammy Jul 25, 2023
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.