Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ISSUE-21: Index all pages (not just blog pages) #27

Merged
merged 1 commit into from
Oct 6, 2023

Conversation

fbuys
Copy link
Contributor

@fbuys fbuys commented Oct 3, 2023

Previously we only included links that end with .html from the different sitemaps. Now we include all pages.

The same conditions used to exclude some specifig pages from our index has been left in place.
Code: .reject {|x| x.include?("page") || x.include?("/tags/") || x.include?("author") }

This comment show all the links that will now be included for fastruby.io for example.

See: #21

@fbuys fbuys requested a review from etagwerker October 3, 2023 17:26
@fbuys
Copy link
Contributor Author

fbuys commented Oct 3, 2023

Hey @etagwerker could you confirm that for fastruby we want to include all these pages from now on?

Link.find_urls(sitemap)
=>
["https://fastruby.io/blog",
 "https://fastruby.io/blog/10-strategies-for-upgrading-ruby-or-rails-applications-with-low-test-coverage.html",
 "https://fastruby.io/blog/bundler/memory-leaks/introducing-bundler-leak.html",
 "https://fastruby.io/blog/case-study/doxo-upgrades-ruby-on-rails-increases-application-performance.html",
 "https://fastruby.io/blog/case-study/power-hrg-tune-report-increases-performance.html",
 "https://fastruby.io/blog/case-study/smile.io-upgrades-ruby-on-rails-increases-speed-agility-and-revenue.html",
 "https://fastruby.io/blog/case-study/soundcloud-upgrades-monolithic-ruby-and-rails-system.html",
 "https://fastruby.io/blog/code-quality/an-intro-to-software-quality.html",
 "https://fastruby.io/blog/code-quality/calculate-tech-debt-using-skunk-on-github-actions.html",
 "https://fastruby.io/blog/code-quality/churn-vs-complexity-vs-coverage.html",
 "https://fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0-simplecov-support.html",
 "https://fastruby.io/blog/code-quality/escaping-the-tar-pit-at-rubyconf.html",
 "https://fastruby.io/blog/code-quality/intruducing-skunk-stink-score-calculator.html",
 "https://fastruby.io/blog/custom-deprecation-behavior.html",
 "https://fastruby.io/blog/docker/docker-ssh-keys.html",
 "https://fastruby.io/blog/esbuild/webpacker/javascript/migrate-from-webpacker-to-esbuild.html",
 "https://fastruby.io/blog/exploring-ruby-warnings.html",
 "https://fastruby.io/blog/fix-sneaky-argument-error-when-upgrading-ruby.html",
 "https://fastruby.io/blog/fortify-rails-security-webinar.html",
 "https://fastruby.io/blog/gem/release/automate-the-gem-release-process.html",
 "https://fastruby.io/blog/gem/release/how-to-manually-release-a-gem-in-rubygems.html",
 "https://fastruby.io/blog/gems/dry/ruby/gemifying-your-styleguide.html",
 "https://fastruby.io/blog/heroku/upgrades/Heroku-22-Stack-Upgrade-Guide.html",
 "https://fastruby.io/blog/how-to-improve-debugging-skills-in-rails.html",
 "https://fastruby.io/blog/how-to-run-multiple-versions-of-rails-on-heroku.html",
 "https://fastruby.io/blog/index.html",
 "https://fastruby.io/blog/introduction-to-rails-engines.html",
 "https://fastruby.io/blog/monthly-maintenance-services.html",
 "https://fastruby.io/blog/navigating-asset-management-madness.html",
 "https://fastruby.io/blog/open-source/guide-for-unmaintained-open-source-projects.html",
 "https://fastruby.io/blog/open-source/introducing-dotenv-validator.html",
 "https://fastruby.io/blog/open-source/ombulabs-open-source-guidelines.html",
 "https://fastruby.io/blog/performance/optimizing-images-2.html",
 "https://fastruby.io/blog/performance/optimizing-images-part-3.html",
 "https://fastruby.io/blog/performance/optimizing-images.html",
 "https://fastruby.io/blog/performance/rails/writing-fast-rails.html",
 "https://fastruby.io/blog/rails/code-quality/An-introduction-to-race-condition.html",
 "https://fastruby.io/blog/rails/code-quality/how-we-estimate-rails-application-size.html",
 "https://fastruby.io/blog/rails/how-to-extend-rails-associations.html",
 "https://fastruby.io/blog/rails/how-to-split-rails-routes.html",
 "https://fastruby.io/blog/rails/javascript/code-coverage/js-code-coverage-in-rails.html",
 "https://fastruby.io/blog/rails/naming-things-is-hard.html",
 "https://fastruby.io/blog/rails/performance/ruby/best-ec2-instance-for-rails-apps.html",
 "https://fastruby.io/blog/rails/performance/ruby/hows-the-performance-of-ruby-3.0.0-preview1.html",
 "https://fastruby.io/blog/rails/performance/writing-fast-rails-part-2.html",
 "https://fastruby.io/blog/rails/performance/writing-fast-rails-part-3.html",
 "https://fastruby.io/blog/rails/rspec/testing-ajax-controller-requests.html",
 "https://fastruby.io/blog/rails/ruby/performance/how-fast-is-ruby-3-on-rails.html",
 "https://fastruby.io/blog/rails/ruby/rubygems/announcing-audit-tool.html",
 "https://fastruby.io/blog/rails/security/dangerous-query-method-deprecation.html",
 "https://fastruby.io/blog/rails/security/ruby-security-toolkit.html",
 "https://fastruby.io/blog/rails/simplecov/upgrades/merging-multiple-simpleCov-coverage-results.html",
 "https://fastruby.io/blog/rails/tips-for-upgrading-rails-3-2-to-4.html",
 "https://fastruby.io/blog/rails/upgrade/dual-booting-with-conditionals.html",
 "https://fastruby.io/blog/rails/upgrade/how-to-pitch-an-upgrade-to-your-boss.html",
 "https://fastruby.io/blog/rails/upgrade/our-rails-upgrade-process.html",
 "https://fastruby.io/blog/rails/upgrade/prepare-for-rails-upgrade.html",
 "https://fastruby.io/blog/rails/upgrade/rails-upgrade-series.html",
 "https://fastruby.io/blog/rails/upgrade/testing/how-to-upgrade-rails-without-tests.html",
 "https://fastruby.io/blog/rails/upgrade/zeitwerk/upgrading-to-zeitwerk.html",
 "https://fastruby.io/blog/rails/upgrades/7-common-mistakes-made-while-doing-rails-upgrades.html",
 "https://fastruby.io/blog/rails/upgrades/active-record-5-1-api-changes.html",
 "https://fastruby.io/blog/rails/upgrades/assessing-rails-upgrades.html",
 "https://fastruby.io/blog/rails/upgrades/case-study/procore-case-study.html",
 "https://fastruby.io/blog/rails/upgrades/case-study/upgrading-a-large-rails-application-from-4.2-to-5.2.html",
 "https://fastruby.io/blog/rails/upgrades/case-study/upgrading-a-monolith.html",
 "https://fastruby.io/blog/rails/upgrades/case-study/upgrading-a-rails-app.html",
 "https://fastruby.io/blog/rails/upgrades/deprecation-warnings-rails-guide.html",
 "https://fastruby.io/blog/rails/upgrades/how-to-migrate-from-capybara-webkit-to-webdrivers.html",
 "https://fastruby.io/blog/rails/upgrades/how-to-stay-up-to-date.html",
 "https://fastruby.io/blog/rails/upgrades/our-rails-upgrade-process.html",
 "https://fastruby.io/blog/rails/upgrades/rails-merge-deprecation.html",
 "https://fastruby.io/blog/rails/upgrades/rails-upgrade-shims.html",
 "https://fastruby.io/blog/rails/upgrades/rails-upgrade-strategies.html",
 "https://fastruby.io/blog/rails/upgrades/simplecov/rails-upgrades-simplecov.html",
 "https://fastruby.io/blog/rails/upgrades/southeast-ruby-upgrade-rails-workshop.html",
 "https://fastruby.io/blog/rails/upgrades/strong-parameters-migration-guide.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-6-1-to-7-0.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-3-2-to-4-0.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-4-0-to-4-1.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-4-1-to-4-2.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-4-2-to-5-0.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-5-0-to-5-1.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-5-1-to-5-2.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-5-2-to-6-0.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-6-0-to-6-1.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-to-rails-3-1.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-to-rails-3-2.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-to-rails-3.html",
 "https://fastruby.io/blog/rails/upgrades/what-i-learned-first-rails-upgrade.html",
 "https://fastruby.io/blog/rails/upgrades/why-upgrade-your-rails-application.html",
 "https://fastruby.io/blog/rails/webpack/from-sprockets-to-webpacker.html",
 "https://fastruby.io/blog/rails/what-does-load-defaults-do.html",
 "https://fastruby.io/blog/rails/what-is-code-coverage-ruby-on-rails.html",
 "https://fastruby.io/blog/railsconf/upgrade-rails-101-the-roadmap-to-smooth-upgrades.html",
 "https://fastruby.io/blog/railsconf-2023-upgrade-workshop.html",
 "https://fastruby.io/blog/readonly-associations-upgrading-rails-4.html",
 "https://fastruby.io/blog/rspec/debug/how-to-debug-non-deterministic-specs.html",
 "https://fastruby.io/blog/rubocop/code-quality/create-a-custom-rubocop-cop.html",
 "https://fastruby.io/blog/ruby/code-quality/how-we-use-rubocop-and-standardrb.html",
 "https://fastruby.io/blog/ruby/dual-boot/dual-booting-gems.html",
 "https://fastruby.io/blog/ruby/dual-boot/generating-gemfile-next-lock.html",
 "https://fastruby.io/blog/ruby/performance/how-fast-are-ractors.html",
 "https://fastruby.io/blog/ruby/performance/why-wasnt-ruby-3-faster.html",
 "https://fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html",
 "https://fastruby.io/blog/ruby/rails/versions/compatibility-table.html",
 "https://fastruby.io/blog/ruby/upgrades/upgrade-ruby-from-2.7-to-3.0.html",
 "https://fastruby.io/blog/ruby/uuid/friendlier-uuid-urls-in-ruby.html",
 "https://fastruby.io/blog/ruby-3-i18n-gem-and-beyond.html",
 "https://fastruby.io/blog/rubyconf/rubyconf-scholars-and-guides.html",
 "https://fastruby.io/blog/testing/javascript/mocking-js-requests.html",
 "https://fastruby.io/blog/testing/the-aaa-pattern-writing-robust-tests-for-any-project-with-confidence.html",
 "https://fastruby.io/blog/the-assets-pipeline-history.html",
 "https://fastruby.io/blog/upgrade-rails/dual-boot/dual-boot-with-rails-6-0-beta.html",
 "https://fastruby.io/blog/upgrade-rails/legacy-rails-silently-judging-you.html",
 "https://fastruby.io/blog/upgrade-rails/workshop/workshop-rails-upgrade.html",
 "https://fastruby.io/blog/upgrades/active-storage/gradually-migrate-from-paperclip.html",
 "https://fastruby.io/blog/upgrades/two-different-approaches.html",
 "https://fastruby.io/blog/why-you-might-not-want-to-run-rails-app-update.html",
 "https://fastruby.io",
 "https://fastruby.io/#clients",
 "https://fastruby.io/#faq",
 "https://fastruby.io/#contact-us",
 "https://fastruby.io/team",
 "https://fastruby.io/blog",
 "https://fastruby.io/roadmap",
 "https://fastruby.io/twelve-factor",
 "https://fastruby.io/security-risks",
 "https://fastruby.io/static-code-analysis",
 "https://fastruby.io/minimum-viable-product",
 "https://fastruby.io/monolith",
 "https://fastruby.io/newsletter",
 "https://fastruby.io/rails-3-0-vulnerabilities",
 "https://fastruby.io/rails-3-1-vulnerabilities",
 "https://fastruby.io/rails-3-2-vulnerabilities",
 "https://fastruby.io/rails-4-0-vulnerabilities",
 "https://fastruby.io/rails-4-1-vulnerabilities",
 "https://fastruby.io/rails-4-2-vulnerabilities",
 "https://fastruby.io/rails-5-0-vulnerabilities",
 "https://fastruby.io/rails-5-1-vulnerabilities",
 "https://fastruby.io/rails-5-2-vulnerabilities",
 "https://fastruby.io/newsletter/70-n-1-problems-railsconf-talks-puma-6-1-and-more",
 "https://fastruby.io/newsletter/1-escaping-tar-pit",
 "https://fastruby.io/newsletter/2-audit-tool-improvements",
 "https://fastruby.io/newsletter/3-security-vulnerability-tools",
 "https://fastruby.io/newsletter/41-latest-updates-from-ruby-on-rails-rails-6-0-4-encrypted-attributes-in-rails-7",
 "https://fastruby.io/newsletter/50-javascript-test-code-coverage-in-rails-business-logic-in-rails-with-operators-enum-in-rails-explained",
 "https://fastruby.io/newsletter/51-ruby-3-1-0-preview-1-under-the-hood-of-the-includes-method-in-rails",
 "https://fastruby.io/newsletter/5-importance-of-rails-upgrades",
 "https://fastruby.io/newsletter/6-open-source-projects",
 "https://fastruby.io/newsletter/9-open-source-guide",
 "https://fastruby.io/newsletter/13-rails-upgrade-strategies",
 "https://fastruby.io/newsletter/23-ruby-3-changes",
 "https://fastruby.io/newsletter/14-rails-performance",
 "https://fastruby.io/newsletter/15-rails-upgrade-case-study",
 "https://fastruby.io/newsletter/17-rails-assets-precompile-performance",
 "https://fastruby.io/newsletter/19-rails-6-1-changes",
 "https://fastruby.io/newsletter/21-news-rails-releases",
 "https://fastruby.io/newsletter/45-hey-inspired-pop-ups-using-hotwire-mastering-rails-time-operations",
 "https://fastruby.io/newsletter/46-rails-7-moves-activerecord-base-logger-to-a-class_attribute-responsible-monkeypatching-in-ruby",
 "https://fastruby.io/newsletter/47-vanillanested-gem-friendlier-uuid-urls-in-ruby",
 "https://fastruby.io/newsletter/24-rails-6-1-changes",
 "https://fastruby.io/newsletter/26-ruby-ractors",
 "https://fastruby.io/newsletter/27-rails-6-1-preview",
 "https://fastruby.io/newsletter/29-ruby-3-release",
 "https://fastruby.io/newsletter/28-ruby-3-0-rc1-released",
 "https://fastruby.io/newsletter/48-gradually-migrate-from-paperclip-to-active-storage-how-to-reduce-memory-bloat-and-rails-7-news",
 "https://fastruby.io/newsletter/49-upgrading-to-zeitwerk-rails-7-supports-nullsfirst-for-all-databases",
 "https://fastruby.io/newsletter/30-how-fast-is-ruby-3",
 "https://fastruby.io/newsletter/31-solving-dual-booting-issues",
 "https://fastruby.io/newsletter/32-why-wasnt-ruby-3-faster",
 "https://fastruby.io/newsletter/34-upgrade-ruby",
 "https://fastruby.io/newsletter/37-building-with-hotwire-rails-6-1-new-defaults",
 "https://fastruby.io/newsletter/52-ruby-code-quality-with-ernesto-tagwerker-design-patterns-in-ruby",
 "https://fastruby.io/newsletter/54-railsconf2022-schedule-generating-the-gemfile-next-lock-custom-changes-in-ruby-and-more",
 "https://fastruby.io/newsletter/60-build-a-ruby-compiler-learn-about-ruby-symbols-ruby-ractors-and-more",
 "https://fastruby.io/newsletter/61-learn-about-fastruby-io-make-2d-games-with-ruby-jit-compilers-and-more",
 "https://fastruby.io/newsletter/57-the-results-are-in-2022-ruby-on-rails",
 "https://fastruby.io/newsletter/84-rails-versions-7-0-7-2-6-1-7-6-released-yjit-actioncable-and-optimizing-code",
 "https://fastruby.io/newsletter/59-a-tool-to-help-rerun-ci-failures-how-to-track-memory-leaks-and-more",
 "https://fastruby.io/newsletter/4-disallowed-deprecations",
 "https://fastruby.io/newsletter/7-vulnerability-puma",
 "https://fastruby.io/newsletter/8-paperclip-is-back",
 "https://fastruby.io/newsletter/10-concurrency-in-ruby",
 "https://fastruby.io/newsletter/11-ruby-new-syntax",
 "https://fastruby.io/newsletter/12-code-quality-metrics",
 "https://fastruby.io/newsletter/16-rails-active-storage-changes",
 "https://fastruby.io/newsletter/18-rails-deprecations",
 "https://fastruby.io/newsletter/20-ruby-3-0",
 "https://fastruby.io/newsletter/22-rails-6-1-deprecations",
 "https://fastruby.io/newsletter/25-rails-shims",
 "https://fastruby.io/newsletter/33-ruby-code-coverage",
 "https://fastruby.io/newsletter/76-save-the-date-new-security-webinar-with-expedited-security-on-june-12th-12pm-est",
 "https://fastruby.io/newsletter/38-behind-the-scenes-of-rails-ujs-best-http-clients-for-2021-rails-7-add-enumerable-in_order_of",
 "https://fastruby.io/newsletter/39-railsconf-2021-talks-solid-principles-rails-7-and-ruby-3-1-additions",
 "https://fastruby.io/newsletter/40-upgrade-rails-from-6-0-to-6-1-upgrading-from-sprockets-to-webpacker",
 "https://fastruby.io/newsletter/42-ruby-3-0-2-released-the-includes-method-in-rails",
 "https://fastruby.io/newsletter/43-rails-6-1-4-has-been-released-the-i-in-solid",
 "https://fastruby.io/newsletter/44-webpack-all-the-assets-using-activerecord-s-update_counters",
 "https://fastruby.io/newsletter/53-rails-server-side-analytics-support-for-deferrable-foreign-key-constraints-more",
 "https://fastruby.io/newsletter/55-check-vulnerabilities-quickly-building-with-rails-7-and-more",
 "https://fastruby.io/newsletter/56-icymi-here-s-what-happened-at-railsconf2022",
 "https://fastruby.io/newsletter/58-migrate-from-webpacker-to-esbuild-add-feature-flags-and-more",
 "https://fastruby.io/newsletter/62-sidekiq-7-0-beta-ruby-3-2-s-data-class-pitchfork-a-new-rack-http-server-by-shopify-and-more",
 "https://fastruby.io/newsletter/63-soundcloud-s-ruby-rails-upgrade-project-puma-6-release-rack-3-0-and-more",
 "https://fastruby.io/newsletter/64-rutie-scientist-dns-inquiries-and-more",
 "https://fastruby.io/newsletter/65-rails-foundation-twitter-drama-heroku-pricing-and-more",
 "https://fastruby.io/newsletter/66-render-deployment-ruby-3-2-0-multi-threaded-web-crawlers-and-more",
 "https://fastruby.io/newsletter/67-ruby-3-2-benchmarking-2022-review-upcoming-camps-conferences-and-more",
 "https://fastruby.io/newsletter/68-rails-7-1-features-new-rails-versions-better-mailers-and-more",
 "https://fastruby.io/newsletter/69-using-openai-with-ruby-the-geocoder-gem-and-a-1-million-user-one-woman-dev-team",
 "https://fastruby.io/newsletter/35-test-rails-5-2-5-6-0-3-6-and-6-1-3-1-have-been-released-docker-basics-for-rails-developers",
 "https://fastruby.io/newsletter/77-reminder-ruby-on-rails-security-webinar-happens-on-monday-at-12pm-est",
 "https://fastruby.io/newsletter/78-in-2-hours-ruby-on-rails-security-webinar-happening-soon",
 "https://fastruby.io/newsletter/79-icymi-our-first-webinar-on-ruby-on-rails-security-was-action-packed-with-information-you-might-need",
 "https://fastruby.io/newsletter/71-a-railscasts-retrospective-new-rubygems-command-ractors-and-more",
 "https://fastruby.io/newsletter/81-work-through-that-technical-debt-sooner-than-later",
 "https://fastruby.io/newsletter/82-database-triggers-asset-management-activerecord-and-the-sunken-cost-fallacy",
 "https://fastruby.io/newsletter/83-rails-7-0-7-released",
 "https://fastruby.io/newsletter/72-new-ruby-versions-metaprogramming-render-and-more",
 "https://fastruby.io/newsletter/36-ruby-3-0-1-released-rails-7-introduces-active-record-encryption",
 "https://fastruby.io/newsletter/85-ruby-3-3-s-yjit-performance-improvements-fixed-cost-upgrade-services-for-javascript-and-deploying-with-kamal",
 "https://fastruby.io/newsletter/73-railsconf-2023-workshops-easy-performance-wins-rack-deflater-and-more",
 "https://fastruby.io/newsletter/74-ruby-3-2-s-yjit-rails-7-1-tune-report-a-new-front-end-language",
 "https://fastruby.io/newsletter/75-new-fastruby-now-offering-fixed-cost-monthly-maintenance-services",
 "https://fastruby.io/monthly-rails-maintenance",
 "https://fastruby.io/security-audit"]

Gemfile.lock Outdated
@@ -434,6 +434,7 @@ PLATFORMS
aarch64-linux
x86_64-darwin-20
x86_64-darwin-21
x86_64-darwin-22
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automatically added when I bundled

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

["https://fastruby.io/blog",
 "https://fastruby.io/blog/10-strategies-for-upgrading-ruby-or-rails-applications-with-low-test-coverage.html",
 "https://fastruby.io/blog/bundler/memory-leaks/introducing-bundler-leak.html",
 "https://fastruby.io/blog/case-study/doxo-upgrades-ruby-on-rails-increases-application-performance.html",
 "https://fastruby.io/blog/case-study/power-hrg-tune-report-increases-performance.html",
 "https://fastruby.io/blog/case-study/smile.io-upgrades-ruby-on-rails-increases-speed-agility-and-revenue.html",
 "https://fastruby.io/blog/case-study/soundcloud-upgrades-monolithic-ruby-and-rails-system.html",
 "https://fastruby.io/blog/code-quality/an-intro-to-software-quality.html",
 "https://fastruby.io/blog/code-quality/calculate-tech-debt-using-skunk-on-github-actions.html",
 "https://fastruby.io/blog/code-quality/churn-vs-complexity-vs-coverage.html",
 "https://fastruby.io/blog/code-quality/code-coverage/rubycritic-4-2-0-simplecov-support.html",
 "https://fastruby.io/blog/code-quality/escaping-the-tar-pit-at-rubyconf.html",
 "https://fastruby.io/blog/code-quality/intruducing-skunk-stink-score-calculator.html",
 "https://fastruby.io/blog/custom-deprecation-behavior.html",
 "https://fastruby.io/blog/dealing-with-schema-changes-in-rails-7.html",
 "https://fastruby.io/blog/docker/docker-ssh-keys.html",
 "https://fastruby.io/blog/esbuild/webpacker/javascript/migrate-from-webpacker-to-esbuild.html",
 "https://fastruby.io/blog/exploring-ruby-warnings.html",
 "https://fastruby.io/blog/fix-sneaky-argument-error-when-upgrading-ruby.html",
 "https://fastruby.io/blog/fortify-rails-security-webinar.html",
 "https://fastruby.io/blog/gem/release/automate-the-gem-release-process.html",
 "https://fastruby.io/blog/gem/release/how-to-manually-release-a-gem-in-rubygems.html",
 "https://fastruby.io/blog/gems/dry/ruby/gemifying-your-styleguide.html",
 "https://fastruby.io/blog/heroku/upgrades/Heroku-22-Stack-Upgrade-Guide.html",
 "https://fastruby.io/blog/how-to-improve-debugging-skills-in-rails.html",
 "https://fastruby.io/blog/how-to-run-multiple-versions-of-rails-on-heroku.html",
 "https://fastruby.io/blog/index.html",
 "https://fastruby.io/blog/introduction-to-rails-engines.html",
 "https://fastruby.io/blog/monthly-maintenance-services.html",
 "https://fastruby.io/blog/navigating-asset-management-madness.html",
 "https://fastruby.io/blog/open-source/guide-for-unmaintained-open-source-projects.html",
 "https://fastruby.io/blog/open-source/introducing-dotenv-validator.html",
 "https://fastruby.io/blog/open-source/ombulabs-open-source-guidelines.html",
 "https://fastruby.io/blog/performance/optimizing-images-2.html",
 "https://fastruby.io/blog/performance/optimizing-images-part-3.html",
 "https://fastruby.io/blog/performance/optimizing-images.html",
 "https://fastruby.io/blog/performance/rails/writing-fast-rails.html",
 "https://fastruby.io/blog/rails/code-quality/An-introduction-to-race-condition.html",
 "https://fastruby.io/blog/rails/code-quality/how-we-estimate-rails-application-size.html",
 "https://fastruby.io/blog/rails/how-to-extend-rails-associations.html",
 "https://fastruby.io/blog/rails/how-to-split-rails-routes.html",
 "https://fastruby.io/blog/rails/javascript/code-coverage/js-code-coverage-in-rails.html",
 "https://fastruby.io/blog/rails/naming-things-is-hard.html",
 "https://fastruby.io/blog/rails/performance/ruby/best-ec2-instance-for-rails-apps.html",
 "https://fastruby.io/blog/rails/performance/ruby/hows-the-performance-of-ruby-3.0.0-preview1.html",
 "https://fastruby.io/blog/rails/performance/writing-fast-rails-part-2.html",
 "https://fastruby.io/blog/rails/performance/writing-fast-rails-part-3.html",
 "https://fastruby.io/blog/rails/rspec/testing-ajax-controller-requests.html",
 "https://fastruby.io/blog/rails/ruby/performance/how-fast-is-ruby-3-on-rails.html",
 "https://fastruby.io/blog/rails/ruby/rubygems/announcing-audit-tool.html",
 "https://fastruby.io/blog/rails/security/dangerous-query-method-deprecation.html",
 "https://fastruby.io/blog/rails/security/ruby-security-toolkit.html",
 "https://fastruby.io/blog/rails/simplecov/upgrades/merging-multiple-simpleCov-coverage-results.html",
 "https://fastruby.io/blog/rails/tips-for-upgrading-rails-3-2-to-4.html",
 "https://fastruby.io/blog/rails/upgrade/dual-booting-with-conditionals.html",
 "https://fastruby.io/blog/rails/upgrade/how-to-pitch-an-upgrade-to-your-boss.html",
 "https://fastruby.io/blog/rails/upgrade/our-rails-upgrade-process.html",
 "https://fastruby.io/blog/rails/upgrade/prepare-for-rails-upgrade.html",
 "https://fastruby.io/blog/rails/upgrade/rails-upgrade-series.html",
 "https://fastruby.io/blog/rails/upgrade/testing/how-to-upgrade-rails-without-tests.html",
 "https://fastruby.io/blog/rails/upgrade/zeitwerk/upgrading-to-zeitwerk.html",
 "https://fastruby.io/blog/rails/upgrades/7-common-mistakes-made-while-doing-rails-upgrades.html",
 "https://fastruby.io/blog/rails/upgrades/active-record-5-1-api-changes.html",
 "https://fastruby.io/blog/rails/upgrades/assessing-rails-upgrades.html",
 "https://fastruby.io/blog/rails/upgrades/case-study/procore-case-study.html",
 "https://fastruby.io/blog/rails/upgrades/case-study/upgrading-a-large-rails-application-from-4.2-to-5.2.html",
 "https://fastruby.io/blog/rails/upgrades/case-study/upgrading-a-monolith.html",
 "https://fastruby.io/blog/rails/upgrades/case-study/upgrading-a-rails-app.html",
 "https://fastruby.io/blog/rails/upgrades/deprecation-warnings-rails-guide.html",
 "https://fastruby.io/blog/rails/upgrades/how-to-migrate-from-capybara-webkit-to-webdrivers.html",
 "https://fastruby.io/blog/rails/upgrades/how-to-stay-up-to-date.html",
 "https://fastruby.io/blog/rails/upgrades/our-rails-upgrade-process.html",
 "https://fastruby.io/blog/rails/upgrades/rails-merge-deprecation.html",
 "https://fastruby.io/blog/rails/upgrades/rails-upgrade-shims.html",
 "https://fastruby.io/blog/rails/upgrades/rails-upgrade-strategies.html",
 "https://fastruby.io/blog/rails/upgrades/simplecov/rails-upgrades-simplecov.html",
 "https://fastruby.io/blog/rails/upgrades/southeast-ruby-upgrade-rails-workshop.html",
 "https://fastruby.io/blog/rails/upgrades/strong-parameters-migration-guide.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-6-1-to-7-0.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-3-2-to-4-0.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-4-0-to-4-1.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-4-1-to-4-2.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-4-2-to-5-0.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-5-0-to-5-1.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-5-1-to-5-2.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-5-2-to-6-0.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-rails-from-6-0-to-6-1.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-to-rails-3-1.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-to-rails-3-2.html",
 "https://fastruby.io/blog/rails/upgrades/upgrade-to-rails-3.html",
 "https://fastruby.io/blog/rails/upgrades/what-i-learned-first-rails-upgrade.html",
 "https://fastruby.io/blog/rails/upgrades/why-upgrade-your-rails-application.html",
 "https://fastruby.io/blog/rails/webpack/from-sprockets-to-webpacker.html",
 "https://fastruby.io/blog/rails/what-does-load-defaults-do.html",
 "https://fastruby.io/blog/rails/what-is-code-coverage-ruby-on-rails.html",
 "https://fastruby.io/blog/railsconf/upgrade-rails-101-the-roadmap-to-smooth-upgrades.html",
 "https://fastruby.io/blog/railsconf-2023-upgrade-workshop.html",
 "https://fastruby.io/blog/readonly-associations-upgrading-rails-4.html",
 "https://fastruby.io/blog/rspec/debug/how-to-debug-non-deterministic-specs.html",
 "https://fastruby.io/blog/rubocop/code-quality/create-a-custom-rubocop-cop.html",
 "https://fastruby.io/blog/ruby/code-quality/how-we-use-rubocop-and-standardrb.html",
 "https://fastruby.io/blog/ruby/dual-boot/dual-booting-gems.html",
 "https://fastruby.io/blog/ruby/dual-boot/generating-gemfile-next-lock.html",
 "https://fastruby.io/blog/ruby/performance/how-fast-are-ractors.html",
 "https://fastruby.io/blog/ruby/performance/why-wasnt-ruby-3-faster.html",
 "https://fastruby.io/blog/ruby/quality/code-quality-ruby-gems.html",
 "https://fastruby.io/blog/ruby/rails/versions/compatibility-table.html",
 "https://fastruby.io/blog/ruby/upgrades/upgrade-ruby-from-2.7-to-3.0.html",
 "https://fastruby.io/blog/ruby/uuid/friendlier-uuid-urls-in-ruby.html",
 "https://fastruby.io/blog/ruby-3-i18n-gem-and-beyond.html",
 "https://fastruby.io/blog/rubyconf/rubyconf-scholars-and-guides.html",
 "https://fastruby.io/blog/testing/javascript/mocking-js-requests.html",
 "https://fastruby.io/blog/testing/the-aaa-pattern-writing-robust-tests-for-any-project-with-confidence.html",
 "https://fastruby.io/blog/the-assets-pipeline-history.html",
 "https://fastruby.io/blog/upgrade-rails/dual-boot/dual-boot-with-rails-6-0-beta.html",
 "https://fastruby.io/blog/upgrade-rails/legacy-rails-silently-judging-you.html",
 "https://fastruby.io/blog/upgrade-rails/workshop/workshop-rails-upgrade.html",
 "https://fastruby.io/blog/upgrades/active-storage/gradually-migrate-from-paperclip.html",
 "https://fastruby.io/blog/upgrades/two-different-approaches.html",
 "https://fastruby.io/blog/why-you-might-not-want-to-run-rails-app-update.html",
 "https://fastruby.io",
 "https://fastruby.io/team",
 "https://fastruby.io/blog",
 "https://fastruby.io/roadmap",
 "https://fastruby.io/twelve-factor",
 "https://fastruby.io/security-risks",
 "https://fastruby.io/static-code-analysis",
 "https://fastruby.io/minimum-viable-product",
 "https://fastruby.io/monolith",
 "https://fastruby.io/newsletter",
 "https://fastruby.io/rails-3-0-vulnerabilities",
 "https://fastruby.io/rails-3-1-vulnerabilities",
 "https://fastruby.io/rails-3-2-vulnerabilities",
 "https://fastruby.io/rails-4-0-vulnerabilities",
 "https://fastruby.io/rails-4-1-vulnerabilities",
 "https://fastruby.io/rails-4-2-vulnerabilities",
 "https://fastruby.io/rails-5-0-vulnerabilities",
 "https://fastruby.io/rails-5-1-vulnerabilities",
 "https://fastruby.io/rails-5-2-vulnerabilities",
 "https://fastruby.io/newsletter/70-n-1-problems-railsconf-talks-puma-6-1-and-more",
 "https://fastruby.io/newsletter/1-escaping-tar-pit",
 "https://fastruby.io/newsletter/2-audit-tool-improvements",
 "https://fastruby.io/newsletter/3-security-vulnerability-tools",
 "https://fastruby.io/newsletter/41-latest-updates-from-ruby-on-rails-rails-6-0-4-encrypted-attributes-in-rails-7",
 "https://fastruby.io/newsletter/50-javascript-test-code-coverage-in-rails-business-logic-in-rails-with-operators-enum-in-rails-explained",
 "https://fastruby.io/newsletter/51-ruby-3-1-0-preview-1-under-the-hood-of-the-includes-method-in-rails",
 "https://fastruby.io/newsletter/5-importance-of-rails-upgrades",
 "https://fastruby.io/newsletter/6-open-source-projects",
 "https://fastruby.io/newsletter/9-open-source-guide",
 "https://fastruby.io/newsletter/13-rails-upgrade-strategies",
 "https://fastruby.io/newsletter/23-ruby-3-changes",
 "https://fastruby.io/newsletter/14-rails-performance",
 "https://fastruby.io/newsletter/15-rails-upgrade-case-study",
 "https://fastruby.io/newsletter/17-rails-assets-precompile-performance",
 "https://fastruby.io/newsletter/19-rails-6-1-changes",
 "https://fastruby.io/newsletter/21-news-rails-releases",
 "https://fastruby.io/newsletter/45-hey-inspired-pop-ups-using-hotwire-mastering-rails-time-operations",
 "https://fastruby.io/newsletter/46-rails-7-moves-activerecord-base-logger-to-a-class_attribute-responsible-monkeypatching-in-ruby",
 "https://fastruby.io/newsletter/47-vanillanested-gem-friendlier-uuid-urls-in-ruby",
 "https://fastruby.io/newsletter/24-rails-6-1-changes",
 "https://fastruby.io/newsletter/26-ruby-ractors",
 "https://fastruby.io/newsletter/27-rails-6-1-preview",
 "https://fastruby.io/newsletter/29-ruby-3-release",
 "https://fastruby.io/newsletter/28-ruby-3-0-rc1-released",
 "https://fastruby.io/newsletter/48-gradually-migrate-from-paperclip-to-active-storage-how-to-reduce-memory-bloat-and-rails-7-news",
 "https://fastruby.io/newsletter/49-upgrading-to-zeitwerk-rails-7-supports-nullsfirst-for-all-databases",
 "https://fastruby.io/newsletter/30-how-fast-is-ruby-3",
 "https://fastruby.io/newsletter/31-solving-dual-booting-issues",
 "https://fastruby.io/newsletter/32-why-wasnt-ruby-3-faster",
 "https://fastruby.io/newsletter/34-upgrade-ruby",
 "https://fastruby.io/newsletter/37-building-with-hotwire-rails-6-1-new-defaults",
 "https://fastruby.io/newsletter/52-ruby-code-quality-with-ernesto-tagwerker-design-patterns-in-ruby",
 "https://fastruby.io/newsletter/54-railsconf2022-schedule-generating-the-gemfile-next-lock-custom-changes-in-ruby-and-more",
 "https://fastruby.io/newsletter/60-build-a-ruby-compiler-learn-about-ruby-symbols-ruby-ractors-and-more",
 "https://fastruby.io/newsletter/61-learn-about-fastruby-io-make-2d-games-with-ruby-jit-compilers-and-more",
 "https://fastruby.io/newsletter/57-the-results-are-in-2022-ruby-on-rails",
 "https://fastruby.io/newsletter/84-rails-versions-7-0-7-2-6-1-7-6-released-yjit-actioncable-and-optimizing-code",
 "https://fastruby.io/newsletter/59-a-tool-to-help-rerun-ci-failures-how-to-track-memory-leaks-and-more",
 "https://fastruby.io/newsletter/4-disallowed-deprecations",
 "https://fastruby.io/newsletter/7-vulnerability-puma",
 "https://fastruby.io/newsletter/8-paperclip-is-back",
 "https://fastruby.io/newsletter/10-concurrency-in-ruby",
 "https://fastruby.io/newsletter/11-ruby-new-syntax",
 "https://fastruby.io/newsletter/12-code-quality-metrics",
 "https://fastruby.io/newsletter/16-rails-active-storage-changes",
 "https://fastruby.io/newsletter/18-rails-deprecations",
 "https://fastruby.io/newsletter/20-ruby-3-0",
 "https://fastruby.io/newsletter/22-rails-6-1-deprecations",
 "https://fastruby.io/newsletter/25-rails-shims",
 "https://fastruby.io/newsletter/33-ruby-code-coverage",
 "https://fastruby.io/newsletter/76-save-the-date-new-security-webinar-with-expedited-security-on-june-12th-12pm-est",
 "https://fastruby.io/newsletter/38-behind-the-scenes-of-rails-ujs-best-http-clients-for-2021-rails-7-add-enumerable-in_order_of",
 "https://fastruby.io/newsletter/39-railsconf-2021-talks-solid-principles-rails-7-and-ruby-3-1-additions",
 "https://fastruby.io/newsletter/40-upgrade-rails-from-6-0-to-6-1-upgrading-from-sprockets-to-webpacker",
 "https://fastruby.io/newsletter/42-ruby-3-0-2-released-the-includes-method-in-rails",
 "https://fastruby.io/newsletter/43-rails-6-1-4-has-been-released-the-i-in-solid",
 "https://fastruby.io/newsletter/44-webpack-all-the-assets-using-activerecord-s-update_counters",
 "https://fastruby.io/newsletter/53-rails-server-side-analytics-support-for-deferrable-foreign-key-constraints-more",
 "https://fastruby.io/newsletter/55-check-vulnerabilities-quickly-building-with-rails-7-and-more",
 "https://fastruby.io/newsletter/56-icymi-here-s-what-happened-at-railsconf2022",
 "https://fastruby.io/newsletter/58-migrate-from-webpacker-to-esbuild-add-feature-flags-and-more",
 "https://fastruby.io/newsletter/62-sidekiq-7-0-beta-ruby-3-2-s-data-class-pitchfork-a-new-rack-http-server-by-shopify-and-more",
 "https://fastruby.io/newsletter/63-soundcloud-s-ruby-rails-upgrade-project-puma-6-release-rack-3-0-and-more",
 "https://fastruby.io/newsletter/64-rutie-scientist-dns-inquiries-and-more",
 "https://fastruby.io/newsletter/65-rails-foundation-twitter-drama-heroku-pricing-and-more",
 "https://fastruby.io/newsletter/66-render-deployment-ruby-3-2-0-multi-threaded-web-crawlers-and-more",
 "https://fastruby.io/newsletter/67-ruby-3-2-benchmarking-2022-review-upcoming-camps-conferences-and-more",
 "https://fastruby.io/newsletter/68-rails-7-1-features-new-rails-versions-better-mailers-and-more",
 "https://fastruby.io/newsletter/69-using-openai-with-ruby-the-geocoder-gem-and-a-1-million-user-one-woman-dev-team",
 "https://fastruby.io/newsletter/35-test-rails-5-2-5-6-0-3-6-and-6-1-3-1-have-been-released-docker-basics-for-rails-developers",
 "https://fastruby.io/newsletter/77-reminder-ruby-on-rails-security-webinar-happens-on-monday-at-12pm-est",
 "https://fastruby.io/newsletter/78-in-2-hours-ruby-on-rails-security-webinar-happening-soon",
 "https://fastruby.io/newsletter/79-icymi-our-first-webinar-on-ruby-on-rails-security-was-action-packed-with-information-you-might-need",
 "https://fastruby.io/newsletter/71-a-railscasts-retrospective-new-rubygems-command-ractors-and-more",
 "https://fastruby.io/newsletter/81-work-through-that-technical-debt-sooner-than-later",
 "https://fastruby.io/newsletter/82-database-triggers-asset-management-activerecord-and-the-sunken-cost-fallacy",
 "https://fastruby.io/newsletter/83-rails-7-0-7-released",
 "https://fastruby.io/newsletter/72-new-ruby-versions-metaprogramming-render-and-more",
 "https://fastruby.io/newsletter/36-ruby-3-0-1-released-rails-7-introduces-active-record-encryption",
 "https://fastruby.io/newsletter/85-ruby-3-3-s-yjit-performance-improvements-fixed-cost-upgrade-services-for-javascript-and-deploying-with-kamal",
 "https://fastruby.io/newsletter/73-railsconf-2023-workshops-easy-performance-wins-rack-deflater-and-more",
 "https://fastruby.io/newsletter/74-ruby-3-2-s-yjit-rails-7-1-tune-report-a-new-front-end-language",
 "https://fastruby.io/newsletter/75-new-fastruby-now-offering-fixed-cost-monthly-maintenance-services",
 "https://fastruby.io/monthly-rails-maintenance",
 "https://fastruby.io/security-audit"]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fbuys This list looks good to me

@@ -8,7 +8,7 @@ class Link < ApplicationRecord

class << self
def find_urls(sitemap)
sitemap.to_a.select {|x| x.end_with? ".html" }.reject {|x| x.include?("page") || x.include?("/tags/") || x.include?("author") }
sitemap.to_a.reject {|x| x.include?("page") || x.include?("/tags/") || x.include?("author") }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add more specific pages to exclude here if needed.

Copy link
Member

@JuanVqz JuanVqz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need Ernesto's input to confirm. To me, this seems correct so far.

@etagwerker
Copy link
Member

@fbuys The list is accurate but it has an error by including these:

 "https://fastruby.io/#clients",
 "https://fastruby.io/#faq",
 "https://fastruby.io/#contact-us",

Those are not new pages, they are sections of https://fastruby.io/

Copy link
Member

@JuanVqz JuanVqz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@FionaDL FionaDL temporarily deployed to librarian-staging October 4, 2023 19:19 Inactive
@fbuys fbuys temporarily deployed to librarian-fbuys-21-inde-gue7qi October 5, 2023 15:56 Inactive
@fbuys fbuys temporarily deployed to librarian-fbuys-21-inde-gue7qi October 5, 2023 16:43 Inactive
@fbuys fbuys temporarily deployed to librarian-fbuys-21-inde-gue7qi October 5, 2023 16:51 Inactive
@fbuys fbuys temporarily deployed to librarian-staging October 6, 2023 13:14 Inactive
Copy link
Member

@etagwerker etagwerker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fbuys I think there is going to be a problem with this PR because of this: https://github.com/fastruby/librarian/blob/main/app/controllers/links_controller.rb#L11

We are using published_at to list links later in a sorted way.

You didn't do any changes to

def find_published_time(page)
element = page.at('meta[property="article:published_time"]')
element.nil? ? "" : element[:content]
end

So what's the plan with that?

Have you tested this locally? What happens to the list of links?

We want to see the latest articles up top, so I don't know if this PR will affect that.

@fbuys fbuys temporarily deployed to librarian-staging October 6, 2023 14:41 Inactive
Previously we only included links that end with `.html` from the
different sitemaps. Now we include all pages.

The same conditions used to exclude some specifig pages from our index
has been left in place. We also exclude URLs with a hash param.
Reject condition: `x.match?(/#.+\z/) || x.include?('page') || x.include?('/tags/') || x.include?('author')`

We removed the where condition in the links_controller.
Now links without published_at will show.
This also means that links without  published_at will appear at the top
of the links page.
See: https://stackoverflow.com/a/44912964

See: #21
@fbuys fbuys temporarily deployed to librarian-staging October 6, 2023 14:56 Inactive
@fbuys
Copy link
Contributor Author

fbuys commented Oct 6, 2023

@fbuys I think there is going to be a problem with this PR because of this: https://github.com/fastruby/librarian/blob/main/app/controllers/links_controller.rb#L11

We are using published_at to list links later in a sorted way.

You didn't do any changes to

def find_published_time(page)
element = page.at('meta[property="article:published_time"]')
element.nil? ? "" : element[:content]
end

So what's the plan with that?

Have you tested this locally? What happens to the list of links?

We want to see the latest articles up top, so I don't know if this PR will affect that.

Hey @etagwerker that is true, thanks for highlighting it.
This comment should address the concerns you raise: https://github.com/fastruby/librarian/pull/27/files#r1348836707

@fbuys fbuys requested a review from etagwerker October 6, 2023 14:59
Copy link
Member

@etagwerker etagwerker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fbuys Looks good, thanks! 👍🏼

@etagwerker etagwerker merged commit 08a1e6b into main Oct 6, 2023
1 check passed
@etagwerker etagwerker deleted the fbuys/21-index-all-pages branch October 6, 2023 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants