Unified STM32 Network Interface

[HTRamsey]

New Unified STM32 driver for F4, F7, H5, H7.

Description

Implementation of new STM32 drivers. Does not support f2.

Test Steps

Checklist:

• I have tested my changes. No regression in existing tests.
• I have modified and/or added unit-tests to cover the code changes in this Pull Request.

Related Issue

#766

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[HTRamsey]

@htibosch @AniruddhaKanhere Looks like this one NetworkInterface should work for all of Fxx & Hxx. The biggest difference between the HAL drivers are the usage of a few register macros that are named differently, but we can easily abstract that out. I've tested on a F767ZI and can test a H755ZI-Q, but don't have a way to test H5 at the moment.

[htibosch]

@holden-zenith wrote:

I've tested on a F767ZI and can test a H755ZI-Q, but don't have a way to test H5 at the moment.

If I'm not mistaken, I have an H747, F407, and F746 for testing.

So e.g. when testing F407, I use the latest HAL drivers for that part?

[HTRamsey]

@htibosch yes, last time I checked the latest F4 & F7 Eth drivers were 100% the same, so it should work the same. Before you test, I just noticed the call to HAL_ETH_ReleaseTxPacket needs to be moved to EMAC_IF_TX_EVENT because it calls the non interrupt safe version of vReleaseNetworkBufferAndDescriptor. I was using vNetworkBufferReleaseFromISR in the HAL_ETH_TxFreeCallback, but I guess it needs to be handled in the task to be compatible with both buffer allocation methods.

[htibosch]

but I guess it needs to be handled in the task to be compatible with both buffer allocation methods.

As you may have read, I also prefer to handle network buffers from a normal task, rather than from an interrupt.

I just came across this test:

    bool isInterrupt()
    {
        return (SCB->ICSR & SCB_ICSR_VECTACTIVE_Msk) != 0 ;
    }

which could be used to choose the correct network buffer function.

last time I checked the latest F4 & F7 Eth drivers were 100% the same

Doesn't F7 have multiple TX buffers each with their own hardware priority? I never used that feature, but I did have to disable it explicitly.

Would you mind to send me an email so I can ask you little things now and then? hein [at] htibosch [dot] net

Thanks,

[htibosch]

Are you sure that HAL_ETH_RxAllocateCallback() will only be called from a task-context?

And also, do you always use BufferAllocation_2.c ? I think we'll have to use that to make sure that uncached memory will be used by DMA.

[shubnil]

HI @holden-zenith thanks for creating the change.
I wanted to get a little bit of context. Please help in explaining the thought behind coming up with this new driver.
I see that the older H and F series portable layers are moved inside legacy and the new driver has an integrated NetworkInterface.c file. Is the idea to have a single file to handle all the multiple stm32 SKUs.

[HTRamsey]

@shubnil Historically the STM32 HAL Eth drivers have been awful, which is part of the reason why edited HAL drivers were provided with the NetworkInterface.c. The newer HAL drivers are 'better' and all STM32's from F4, F7, H5, H7 essentially use the same driver now. So with this, edited HAL drivers are no longer necessary, it will be much easier to update to accommodate for future HAL updates, only one NetworkInterface is needed, and I am getting about 20mbps higher on iperf. I left the legacy drivers for now since anything prior to F4 won't be getting the HAL update, so technically I suppose only the Fxx driver would be necessary to leave.

[htibosch]

@holden-zenith , do you have a new version of NetworkInterface.c to test?

[HTRamsey]

@htibosch I'll probably push it in the next few days. I've been able to max out the bandwidth at 95mbps but it's not consistent and depends on compilation optimization. So I need to spend some time looking through the disassembly before I push it.

[bjsowa]

@holden-zenith What's the status of this? I have a Nucleo-H563ZI board and would like to use FreeRTOS+TCP but the current port implementation does not support H5 series.

[HTRamsey]

@bjsowa Actually it would be great if you could test it. I have used it successfully on an F7 but there is potential concern with how it compares to the older driver in terms of performance, at least when tested with an F4. I do not have an H5 at the moment to try it.

[bjsowa]

@holden-zenith I made the driver work for my nucleo board (at least DHCP and ping replies seem to work).
I posted the modified NetworkInterface.c with some comments here:
HTRamsey#1

I also ported it to work for version 4.0.0 of FreeRTOS-Plus-TCP (as I want IPv6 support) by mimicking the changes made in #602 for the current driver.

[bjsowa]

On bjsowa#1 I integrated the new driver with the CMake build system. You can use this for reference.

[HTRamsey]

@bjsowa oh nice that's great. I am working on testing an H7 too so all of F4, F7, H5, & H7 will have been tested. Seems to be at least usable on all of them for the time being. In theory it should perform at least as well as the last driver but iperf apparently has had mixed results so far.

[htibosch]

The earlier drivers for STM32Fxx and STM32Hxx used a copy of ST's HAL driver. The driver was optimised for efficiency/speed, and support for zero-copy was added.

@holden-zenith created a unified driver that is based on today's HAL drivers. The driver files must be obtained from ST.

I tried the new driver on STM32F4 and STM32F7, and found that iperf3 shows lower speeds, both TX and RX. I wonder what the influence is of the extra mutex?

I've got no testing equipment where I am now. In August I can do a real-time analyses of both drivers.

[amazonKamath]

@holden-zenith I made the driver work for my nucleo board (at least DHCP and ping replies seem to work). I posted the modified NetworkInterface.c with some comments here: holden-zenith#1

I also ported it to work for version 4.0.0 of FreeRTOS-Plus-TCP (as I want IPv6 support) by mimicking the changes made in #602 for the current driver.

@bjsowa Thanks for trying out the release candidate versions of 4.0.0. Did you face any difficulties or issues with IPv6? Also curious to know if you use the multiple endpoint feature?
PS: We will be soon doing a formal release (targeting mid August). The stack is currently undergoing security review and penetration testing.

[HTRamsey]

@htibosch Here are the ping results of the latest version. It seems to have much better latency, at least for me.
Old Driver:

New Driver:

[htibosch]

@holden-zenith wrote:

Here are the ping results of the latest version. It seems to have much better latency, at least for me.

That looks very promising. Dit you already commit and push the changes?
Do you want me to test it on a particular platform?

[chinglee-iot]

@htibosch @tony-josi-aws is it possible to ignore spellings and formatting for the included HAL drivers?

"source/portable/NetworkInterface/STM32" folder is excluded from spelling and formatting check.

[bjsowa]

@HTRamsey Could you look at HTRamsey#13 ?

[HTRamsey]

@ActoryOu Is there any plan to check this out? If not, it at least serves as a good example I guess

[ActoryOu]

Hi @HTRamsey,
Sorry for late response.
We had a internal discussion on this PR, then continue to review this.
Here are my comments. Please help take a look.

Thank you.

[bjsowa]

@HTRamsey What's the reasoning for removing HAL drivers? This makes the FREERTOS_PLUS_TCP_STM32_IF_DRIVER feature in CMakeLists.txt obsolete.

[HTRamsey]

I guess that depends on if we can add the hal drivers to be ignored by formatting and spell checks

[HTRamsey]

@ActoryOu @tony-josi-aws What should I do from here? I'd like to get this PR closed if not accepted

[ActoryOu]

@ActoryOu @tony-josi-aws What should I do from here? I'd like to get this PR closed if not accepted

Hi @HTRamsey,
Sorry for late. I'd like to approve this PR.
But why do you remove all *hal_eth.c files in the latest commit?

Thank you.

[tony-josi-aws]

Hi @HTRamsey,
Thanks for this contribution, and we are good with this change.
We plan to merge this PR by early October and are still running some tests.

Few observations that were found:

1. Build error with ipconfigETHERNET_DRIVER_FILTERS_PACKETS enabled [line]:
   1. error: conflicting type qualifiers for 'ulRxDesc'
2. Comparitvely low iperf performance with TCP compared to the legacy STM32 network interface (still investigating if its configuration/tuning issue).

[HTRamsey]

@ActoryOu
I can re-add, but then we have to be able to ignore those files with spelling & formatting.

@tony-josi-aws
will fix that error, and I will check the low performance issue, I've had significant performance differences based on config values. I can max out around 97mbps depending on the config. There have been a couple of HAL ethernet driver updates that have not been released yet (but can be found on their github repos) that may help too.

[ActoryOu]

@ActoryOu I can re-add, but then we have to be able to ignore those files with spelling & formatting.

Got it. Then please re-add them and I'll approve this PR. Thank you!

[Mixaill]

@tony-josi-aws

The performance of iperf depends heavily on the configuration of the PC and MCU. As mentioned above, I was able to achieve a speed of ~93.3 Mbps.

#804 (comment)

[tony-josi-aws]

@HTRamsey

I have tested the PR on STM32 F4, H5, H7, and the new network interface seems to be working fine. Here are few things that I noted:

• eConsiderPacketForProcessing not defined when ipconfigETHERNET_DRIVER_FILTERS_PACKETS. I suppose eConsiderPacketForProcessing will be part of a future change?
• Similar to the old network interface it will be good to print the resource statistics while the debug printf is enabled:

  •   #if ( ipconfigHAS_DEBUG_PRINTF != 0 )
              {
                  /* Call a function that monitors resources: the amount of free network
                   * buffers and the amount of free space on the heap.  See FreeRTOS_IP.c
                   * for more detailed comments. */
                  vPrintResourceStats();
              }
      #endif /* ( ipconfigHAS_DEBUG_PRINTF != 0 ) */
    

[HTRamsey]

@tony-josi-aws Oh yeah I see that. It's been a long while but there was a discussion about adding some kind of generic check 'eConsiderPacketForProcessing' (similar to the one for the ethernet frame) that all network interfaces can use.