Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update toolchains to newer compilers #25327

Draft
wants to merge 47 commits into
base: bugfix-2.1.x
Choose a base branch
from

Conversation

quiret
Copy link
Contributor

@quiret quiret commented Feb 1, 2023

Description

This pull request updates the platform and package versions of build configurations to their latest version. The build configurations have been rigorously tested (mostly under my HAL SPI rework).

Improved generic_create_variant.py to not mess with official board variant configurations (only touch if board_build.variant value is prefixed with MARLIN_).

Benefits

Newer compilers emit better code. Newer compilers provide newer C++ language features thus more powerful software models can be implemented. Newer toolchains may come with support for even more boards.

Related Issues

#24911 (depends on this PR)

…future

- improved generic_create_variant.py to not mess with official board variant configurations (only touch if board_build.variant value is prefixed with MARLIN_)
…e STM32 framework (it is excluded by default) [chitu_f103 fix]
@ellensp
Copy link
Contributor

ellensp commented Feb 2, 2023

I really hope you have thoroughly tested these new compilers on all boards effected, and are not just blindly upgrading them... beyond it still compiles...

@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

I really hope you have thoroughly tested these new compilers on all boards effected, and are not just blindly upgrading them... beyond it still compiles...

The upgrade process is being performed slowly! You can see that I have left compatibility .ini configurations as fallback. The following boards have been tested IRL:

  • BTT SKR V1.4
  • MKS TinyBee V1.0
  • MKS Robin E3D V1.1

Also please do not seed distrust into PlatformIO official registry packages. They have been released for a good reason and PlatformIO targets embedded devs. I will nevertheless make sure everything works ;) Progress is necessary!

…32 to fix compilation issues on the latest Espressif toolchains

When the PR luc-github/ESP3DLib#51 has been merged then please change it back!

- (temporarily) updated the "ESP Async WebServer" dependency for MKS TinyBee / ESP32 to fix compilation issues on the latest Espressif toolchains

Please watch PR me-no-dev/ESPAsyncWebServer#1142 and change back once merged.

- fixed another ESP32 bug due to toolchain update
@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

Please watch PRs me-no-dev/ESPAsyncWebServer#1142 and luc-github/ESP3DLib#51 for merging, then change back dependencies if the official packages work.

@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

I really hope you have thoroughly tested these new compilers on all boards effected, and are not just blindly upgrading them... beyond it still compiles...

@ellensp Hey I need some help testing the things that run on CI locally. Do you know how I can do that? I'd greatly appreciate it! Need to fix the issue for the MKS TinyBee which fails in an unknown Linux script...

Never mind! Got it figured out. I simply took another crash course into Marlin FW, specifically the automated build system scripts and stuff. Really interesting and worth a video! That system looks very stable so users should get an insight if they want to contribute to Marlin FW.

…uilding again (shell passthrough special characters)
@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

Please monitor PR espressif/arduino-esp32#7744 + check for the release of arduino-espressif32 version 2.0.7. Then remove the custom dependency from esp32.ini !

…spressif32 (please fix dependency once 2.0.7 of that package is out on PlatformIO registry!)
@ellensp
Copy link
Contributor

ellensp commented Feb 2, 2023

I noticed that you added gnu++1z to avr env

According to this document https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html

‘gnu++17’
‘gnu++1z’
`GNU dialect of -std=c++17. This is the default for C++ code. The name ‘gnu++1z’ is deprecated.`

So perhaps you should be using gnu++17 instead ?

The document also state "This is the default" is this define even needed?

It does build slightly differently

example ramps build.
With gnu++1z
RAM: [====== ] 60.3% (used 4938 bytes from 8192 bytes)
Flash: [===== ] 51.1% (used 129668 bytes from 253952 bytes)

Without gnu++1z
RAM: [====== ] 60.3% (used 4938 bytes from 8192 bytes)
Flash: [===== ] 51.0% (used 129630 bytes from 253952 bytes)

38 bytes less flash...

@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

I noticed that you added gnu++1z to avr env
(...)
The document also state "This is the default" is this define even needed?
(...)
38 bytes less flash...

Thanks for pointing this out! I am still trying to figure out the proper C++ standard targets for each platform. The C++17 standard is already pretty good, but there are really amazing features in C++20 that we should not miss out on.

It is sometimes necessary to override -std flags from dependency packages / platform options, so that is also a reason.

…tions

Please check PR rhapsodyv/Arduino_Core_STM32#1 and if @rhapsodyv has merged then adjust the dependency to an official one!
@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

@ellensp Please note that the currently available GCC for AVR version is 7.3 on PlatformIO and you have mentioned the default -std=c++17 option that is only from GCC version 11.

https://registry.platformio.org/tools/platformio/toolchain-atmelavr/versions
https://gcc.gnu.org/projects/cxx-status.html#cxx17

Worth pointing out because GCC versions differ between platforms! Basically GCC maintenance is a huge mess due to the distributed nature of the compiler.

@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

As part of my work I can fix the following issue: #24717

@EvilGremlin
Copy link
Contributor

Marlin is reluctant to bump toolchain versions because it will break things, potentially for thousands people. Can you test at least dozens configurations for each MCU? And that's just hardware, not software feature conmbinations. Do you at least cople dozens most popular boards, dozen different displays and machines of every kinematic so you can debug and fix stuff you will break with toolchain updates? As long as current toolchain is satisying - we'd better not touch it.

@ellensp
Copy link
Contributor

ellensp commented Feb 2, 2023

"As part of my work I can fix the following issue: #24717"

That is trivial, main issue is pins_debugging needs re written for stm32 platform, it doesn't support many MPU's properly at all.

Most Issues are with analog pins.

And this should be in a separate PR.

@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

"As part of my work I can fix the following issue: #24717"

That is trivial, main issue is pins_debugging needs re written for stm32 platform, it doesn't support many MPU's properly at all.

Most Issues are with analog pins.

And this should be in a separate PR.

@ellensp I think that a proper implementation of pin debugging should be in a seperate PR. But I will adjust some things for compatibility sake.

@quiret
Copy link
Contributor Author

quiret commented Feb 2, 2023

Marlin is reluctant to bump toolchain versions because it will break things, potentially for thousands people. Can you test at least dozens configurations for each MCU? And that's just hardware, not software feature conmbinations. Do you at least cople dozens most popular boards, dozen different displays and machines of every kinematic so you can debug and fix stuff you will break with toolchain updates? As long as current toolchain is satisying - we'd better not touch it.

@EvilGremlin As I have said, PlatformIO packages are there for a reason. They are tried & tested by the embedded community. Thus it is safe to assume that they are ready to be deployed on real hardware. Nevertheless I will perform big tests on real hardware.

…moving Marlin internals out of variants-headers and making them optional (new platdefs STM32 HAL folder for Marlin specific board specializations + _STM32_PLATDEFS define)

@ellensp said that PINS_DEBUGGING support should be rewritten in another PR; I agree but this is a necessity to unbreak compilation + PINS_DEBUGGING is not a release feature so don't obstruct progress, please!
…t is a bad idea to rely on such internal thus adjusted some things.
@thinkyhead
Copy link
Member

@sjasonsmith — I agree that this should not be merged all at once, so I've been slowly integrating small parts of this as time allows. Today I'm just bringing this PR up to date with a new merge, and then following up to see if I can make the CI tests pass. I see ESP32 is up to version 6.4.0 now, so this PR might also be updated to point to that version.

Over time we can continue to pick up small parts of this and integrate them as we have time to test, perhaps starting with ESP32 since it will have the smallest impact on the smallest number of boards. I would definitely like to get updated to the more modern STM32 platform too since it apparently has some nice improvements.

@thinkyhead thinkyhead force-pushed the bugfix-2.1.x branch 2 times, most recently from 9c65146 to 4f65466 Compare January 26, 2024 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants