[enhancement]: Automatically linkcheck in CI

[itsaviral2609]

Proposed Commit Message

workflow for linkcheck in CI

Note for Maintainers

Workflow subject to changes by maintainers.

Fixes GH- #4460

Checklist:

• My code follows the process laid out in the documentation
• I have updated or added any unit tests accordingly
• I have updated or added any documentation accordingly

[blackboxsw]

Thanks for this first contribution to cloud-init @itsaviral2609!

I've confirmed you have signed the CLA and have a minor suggestion or two to discuss here.

Our linkchecker target seems to raise a bunch of errors on links which shouldn't be valid, and some links we shouldn't be checking every time we run CI.

I think it's worth adding the following to our sphinx conf to ignore a few things:

diff --git a/doc/rtd/conf.py b/doc/rtd/conf.py
index fcb8f2e30..5c9be14a6 100644
--- a/doc/rtd/conf.py
+++ b/doc/rtd/conf.py
@@ -119,6 +119,16 @@ html_extra_path = ["googleaf254801a5285c31.html"]
 autosectionlabel_prefix_document = True
 autosectionlabel_maxdepth = 2
 
+# Sphinx-linkcheck config
+linkcheck_ignore = [
+    r"http://instance-data.*", r"https://powersj.io.*",
+    r"http://169.254.169.254.*", r"http://10.10.0.1.*"
+]
+linkcheck_anchors_ignore_for_url = (
+    r"https://github.com/canonical/cloud-init.*",
+    r"https://github.com/canonical/ubuntu-pro-client.*"
+)
+
 # Sphinx-copybutton config options:
 notfound_body = (
     "<h1>Page not found</h1><p>Sorry we missed you! Our docs have had a"

[blackboxsw]

Thanks for the constructive workflow_dispatch suggestion here for readability of continue-on-error below.

[blackboxsw]

For the moment it looks like we are going to have a few broken links, hence good to have the continue-on-error: false. I think it may be worth grep broken on the linkcheck and tee int the output of broken lines to another file and performing a line-count and assert that the broken link line count is < 6.

This way we could set a high-water mark that we can reduce in subsequent PRs as we sort external stale documentation links and update our docs.

[itsaviral2609]

@blackboxsw I have made the requested changes!
Do ping me if I missed out on any change :)

[blackboxsw]

Looks great @itsaviral2609. Last review round and we'll turn on CI and land this branch, many thanks.

[blackboxsw]

Great thanks @itsaviral2609, one more ignore to add and then I think we can drop the high-water mark for # of broken links from 6 to 3

diff --git a/doc/rtd/conf.py b/doc/rtd/conf.py
index 0dc3b4839..03c8688d2 100644
--- a/doc/rtd/conf.py
+++ b/doc/rtd/conf.py
@@ -120,8 +120,9 @@ autosectionlabel_prefix_document = True
 autosectionlabel_maxdepth = 2
 
 # Sphinx-linkcheck config
-linkcheck_ignore = [   r"http://instance-data.*", r"https://powersj.io.*",
-                      r"http://169.254.169.254.*", r"http://10.10.0.1.*"
+linkcheck_ignore = [
+    r"http://\[fd00:ec2::254.*", r"http://instance-data.*",
+    r"https://powersj.io.*", r"http://169.254.169.254.*", r"http://10.10.0.1.*"
 ]
 linkcheck_anchors_ignore_for_url = (
     r"https://github.com/canonical/cloud-init.*",

[blackboxsw]

Thank your for this extra work here. This looks good. We also want to avoid the internal ec2 ipv6 instance metadata service r"http://\[fd00:ec2::254.*" and we should be good to drop the high-water mark from 6 to 3 and merge this PR. Thanks again for the swift work here.

Suggested change

	if [[ $broken_count -ge 6 ]]; then
	if [[ $broken_count -ge 3 ]]; then

[itsaviral2609]

Done with the requested changes!

I'm working on issue #4453 too and was facing issues to generate template while running this command
make render-template command. Wanted a brief discuss on what should be the variant(List of variants given like ubuntu,fedora,suse.....)

I will integrate this issue to CI too if I get more clear objectives of this task @blackboxsw

[blackboxsw]

Looks good to me. Welcome to cloud-init! Pushed a commit that fixes black formatting style on conf.py, by running: tox -e do_format on this branch. I'll land when CI is happy