Future of .NETMF (take #999999) #527
Comments
|
The thumbs up is for never giving up and try over and over again to put life into this repo |
|
If someone knows him, please tell this guy to release it, please :) |
|
There are a couple of basic problems!
Problem has always been the porting process and the steep learning curve etc. This is more critical now with the appearance of lots of low cost embedded modules capable of running Linux/Android. We are lucky in that we can customise a port for a customer fairly painlessly, and as a result we have completed a number of commercial projects based on MF, and the pain of the port is out weighed by the gains in speed of application development, ease of debugging and overall system reliability. For MF5 to succeed, the porting process needs to be documented and easier to build - both points have already been acknowledged. It kind of makes a lot of sense to maintain the track it seems to have gone down - focusing on Cortex M base processors - even tho it was supposed to be processor independent. However all this is a lot of work - and if we rely on Steve M - then frankly he is never going to have enough free time to do this. The only way this is going to happen is if some people can commit some time to helping out. |
|
I agree with most, not all of your points. One of the points I disagree with is about GHI not supporting MF anymore. Although this is probably closed source but it is NOT dead, on the contrarary read here for more details ... https://www.ghielectronics.com/community/forum/topic?id=23934&page=1 |
|
@SytechDesigns you bring up very good points. To be perfectly clear: I have not – now or ever – complaint on what Steve Maillet has contributed for .NETMF. On the matter of evaluation boards for hobbyists or companies working on a proof of concept, I believe that the offer from ST (being currently the only fully supported SoC) is quite abundant, cheap and with all the possible variations (different series, more or less peripherals, networking, low end devices, high end devices, etc). Meaning that it’s probably a waste of resources for any company out there to be offering an evaluation board. The SoC manufacturers already to that very well. So let’s set the focus of .NETMF on the firmware to load on those board (evaluation or OEM). I agree 200% with you on the porting aspects. Being so “painful” and the lack of documentation is a serious (if not the main) blocking aspect of adoption for new platforms/variants/adaptations to custom or evaluation boards. Once you get it is quite easy actually, but you need to get there… Adapting the HAL/PAL to CMSIS would be a very good approach on “kind of” standardizing the architecture and making it easy to move between different series and even chip makers. As time permits, me and a colleague of mine, we’ve been working on exactly that approach, along with making the porting and feature management much easier and understandable. Take a look at #434. On the toolchain (for native code and building a .NETMF image) using VS all the way would be awesome. Maintaining both VS and uVision projects seems complicated and requires an extra effort. Regarding volunteering: more than volunteering and/or committing to it there are already a number of us actually and actively contributing with code and work. The more people willing to do it the better, but I don’t think that the lack of contributions or people willing/wanting to contribute is the reason for the non-progress of .NETMF… |
Volunteers for what exactly? As @josesimoes has mentioned already, there are people willing to commit their resources to various kinds of contributions (issues, code, help etc.), but there is nobody at the other side. I don't blame anybody for turning away after their effort is left without any response for weeks, months or even years.
I don't question their technical knowledge, but IMVHO it has been clear for quite some time they are not able to manage NETMF project in any reasonable way. I understand delays, change of priorities, personal reasons etc., this happens to everybody - but going completely silent for months without any hint has reached the level of being almost insulting. Especially, considering the status of other projects under .NET Foundation umbrella. As I see it, NETMF is now at the edge of the cliff, already put a foot forward and looking back at @smaillet-ms - unfortunately, there is not much time left... |
TL;DRAfter writing this all it came out a bit long so I'll summarize. The success or failure of NETMF (the current interpreter as well as the general concept of .NET for micros) depends entirely on the people who choose to work on it. There is a ton of work that can be done, most of it isn't writing code for an STM32 and may not be of interest or within the capabilities of everyone. The success of NETMF doesn't depend on me, Lorenzo, our employer or any other entity to "manage it". If NETMF is to survive it will do so because enough people are willing, able, and step up to make it useful enough to a sufficiently large enough group of people.
What other side are you talking about? Where are these people? Where are their design proposals for any of the open issues? Why haven't any of them bothered to fix the commits that have already been allowed in and broke the GGC builds? (#508) I'm not trying to start a Grand Urinary Olympics here but this isn't so one sided as you seem to think it is. There are a number of pending PRs that I haven't merged because they don't address the busted build. Let's get that fixed then we can bring in the other pieces. There's plenty of work to be done if people want to do it they need to step up. If there is an existing issue filed then propose a solution, discuss or comment on proposals, then implement a PR, engage in reviews of PRs. If there isn't an issue filed, then submit one to define the problem. If everyone waits for someone else to assign tasks to them things will just stall. If there was a huge team with financial interests in the project maybe we could afford someone to manage tasks at that level, but whether we like it or not - that's not the reality here. A major tenant of Agile Development is the idea of self organizing teams, rather then strict rigid hierarchies. I think NETMF needs that, but it takes a certain kind of commitment and discipline that's quite different from what most people are used to in their professional employment. Now, to the future of NETMF. Some are painting a lot of doom and gloom and bleak dark pictures on NETMF - they are entitled to their opinions. They may even be correct in the end. If you don't want such views to be reality then you need to ask yourself - what are you willing to do to prevent it? If you only need some functionality to continue shipping your own commercial products and don't care if such changes are relevant to anyone else (or even are counter productive to them) then by all means fork the code and have at it. If on the other hand you want/need it to be a platform in and of itself with a vibrant and active user base then ask yourself what is it that prevents that and what are you willing and able to do to get it there. I for one would like to see NETMF as a platform in and of itself. Though I need to make it clear that doesn't necessarily mean the current code base. More specifically I mean the concept of .NET and especially C# as a language/platform for programming micro controllers in embedded systems. After I posted a set of ideas and a branch for vNext there was a lot of criticism about spending any time on NETMF instead of Llilum (e.g. why not let the interpreter "out to pasture" and push on with AOT compilation with Llilium). Thus, it wasn't clear to me that what I was proposing was really where things should go. While it is work I'd always wanted to see done for NETMF I'm not really interested in doing it as an academic exercise for myself. (I've got other projects for that) In addition to the pushback at the same time, the basic technologies to build off of weren't quite publicly available or ready for use. This includes the new VS "15" project system support, which as of the latest preview is now pretty stable to build off of. The other core piece for the foundation was CMSIS, since we are looking to re-define the entire HAL/PAL concept for the platform (AOT , interpreted or both, doesn't really matter) there wasn't a lot of value doing that based on the V4.x flavors of CMSIS when V5 was making some significant and valuable changes (especially in the CMSIS-RTOS2 API). Unfortunately, that has been delayed and pushed back several times, and only reached final release state a few days ago. Since not all of the underpinning tech was ready I've been spending time looking at the AOT side of things. One of the biggest concerns with just shifting all efforts to Llilium is that it is a very complex code base where it would be very hard to ramp up on. Other concerns include that it is very slow to compile and the overall footprint isn't as low as most of us would like/hope for. So I took everything I learned for using LLVM.NET in Llium and the early test project I had created before we moved to build Llilum out of an older project. I'm not claiming that doing Llilum based on the older Zelig code was a wrong or even bad choice - as it gave us something that mostly works and shows there is value to be had in the area of AOT compilation of .NET for micros. However it isn't something that's really a long term solution either. It was designed based on an original idea that everything could be done in C# without any need to interop with existing C code. 10 years ago when it was originally designed that was a plausible, albeit quite bold, story - today - not a chance. So, while we managed to bolt LLVM onto the bottom to replace the old ARMv4 code generation there was a lot where the original Zelig tools and LLVM tool essentially get in each others way. Given all of that, and my own desire to see a viable AOT solution. I dusted off some early prototyping code with a deliberately naïve plan - do as little as possible in the transform tool and let LLVM do as much as possible. The results of my experiments in this direction are actually pretty surprising. I generated a simple blinky app with no objects/GC (only static data) and an assembly generated from a CMSIS-SVD file to provide direct type safe access to hardware registers. Conversion of the main app, a secondary assembly implementing the equivalent to the mbed HAL GPIO API and the assembly generated from the SVD into a single fully optimized object file takes a little under 8 seconds, and when linked to the standard startup code from CMSIS the entire app comes in a little under 3K. Granted that is without any GC or allocated classes with vtables to take up space. But it shows we can be pretty close to C++ without a GC. The Boehm GC currently in Llilum wouldn't add a huge amount on top of that. Furthermore, the approach I've taken is to generate LLVM bit code modules for each assembly individually, and save them along side the original assembly. Thus a minor change in one assembly means generating only the bit code for that one assembly and all the others are re-used as is. Thus on a minor tweak to the main exe the time to generate the obj file a second time comes down to under 2 seconds The type safe hardware access portion of that relies on a new C#7 feature - byref returns, while the capability for that has always been in the .NET runtime (C++/CLI uses it) C# has never exposed it until now. This feature along with a special interface and some attributes known to the code generator allows generation of type safe hardware access that ultimately resolves to dereferencing constant pointers as you would get in C. example: /// <summary>Demonstrates a pure C# implementation of the equivalent mBed HAL APIs</summary>
/// <remarks>
/// DeviceMap is a static type in an assembly generated from the CMSIS-SVD description
/// of the on-chip hardware register space. It provides high performance type safe direct
/// access to the hardware registers. (Code size and perf is equivalent to C code as the
/// registers are mostly accessed via constant pointers and property methods aggressively inlined).
/// </remarks>
public static class GpioApi
{
/// <summary>Configures a GPIO pin as an input</summary>
/// <param name="pinData">GPIO pin definition data structure to hold initialized data for the pin</param>
/// <param name="pin">Pin id</param>
/// <param name="mode">Mode for the pin</param>
public static void ConfigureAsInput( ref PinInfo pinData, PinName pin, PinMode mode = PinMode.PullDefault )
{
pinData.Port = ( ( int )pin ) / 32;
pinData.Mask = 1u << ( ( int )pin ) % 32;
DeviceMap.GPIO.PORT[pinData.Port].DIR.Value &= ~pinData.Mask;
}
/// <summary>Configures a GPIO pin ans an input</summary>
/// <param name="pinData">GPIO pin definition data structure to hold initialied data for the pin</param>
/// <param name="pin">Pin id</param>
/// <param name="value">Initial state of the pin (default == 0)</param>
public static void ConfigureAsOutput( ref PinInfo pinData, PinName pin, int value = 0 )
{
pinData.Port = ( ( int )pin ) / 32;
pinData.Mask = 1u << ( ( int )pin ) % 32;
DeviceMap.GPIO.PORT[ pinData.Port ].DIR.Value |= pinData.Mask;
Write( ref pinData, value );
}
/// <summary>Writes a new state for a GPIO pin</summary>
/// <param name="pinData">Data structure identifying the pin</param>
/// <param name="value">new value for the pin</param>
public static void Write( ref PinInfo pinData, int value )
{
if (value == 0)
DeviceMap.GPIO.PORT[pinData.Port].CLR.Value = pinData.Mask;
else
DeviceMap.GPIO.PORT[pinData.Port].SET.Value = pinData.Mask;
}sample main: using NXP.LPC1768;
namespace Blinky
{
/// <summary>Simple test app to validate behavior of code generation</summary>
public static class Program
{
// GPIO pin info for the LEDs on the dev board
static PinInfo Led1;
static PinInfo Led2;
static PinInfo Led3;
static PinInfo Led4;
static PinInfo LedRed;
static PinInfo LedGreen;
static PinInfo LedBlue;
const int WaitTimeoutMs = 75;
/// <summary>Main entry point for the app</summary>
/// <remarks>
/// The runtime startup code will call this after initializing. Technically,
/// the ExportAsAttribute here is redundant as the codegen tool can infer
/// the name from the assemblies EntryPoint as "main" and effectively apply
/// the attribute dynamically.
/// </remarks>
//[ExportAs( "main")]
public static void Main( )
{
GpioApi.ConfigureAsOutput( ref Led1, PinName.LED1, 1 );
GpioApi.ConfigureAsOutput( ref Led2, PinName.LED2, 1 );
GpioApi.ConfigureAsOutput( ref Led3, PinName.LED3, 1 );
GpioApi.ConfigureAsOutput( ref Led4, PinName.LED4, 1 );
GpioApi.ConfigureAsOutput( ref LedRed, PinName.p23, 1 );
GpioApi.ConfigureAsOutput( ref LedGreen, PinName.p24, 1 );
GpioApi.ConfigureAsOutput( ref LedBlue, PinName.p25, 1 );
var state = true;
while( true )
{
var activeHigh = state ? 1 : 0;
var activeLow = state ? 0 : 1;
GpioApi.Write( ref Led1, activeHigh );
WaitApi.Wait( WaitTimeoutMs );
GpioApi.Write( ref Led2, activeHigh );
WaitApi.Wait( WaitTimeoutMs );
GpioApi.Write( ref Led3, activeHigh );
WaitApi.Wait( WaitTimeoutMs );
GpioApi.Write( ref Led4, activeHigh );
WaitApi.Wait( WaitTimeoutMs );
// RGB leds are active low
GpioApi.Write( ref LedRed, activeLow );
WaitApi.Wait( WaitTimeoutMs );
GpioApi.Write( ref LedGreen, activeLow );
WaitApi.Wait( WaitTimeoutMs );
GpioApi.Write( ref LedBlue, activeLow );
WaitApi.Wait( WaitTimeoutMs );
state = !state;
}
// Never returns...
}
}
}I'm hoping I can get the basic structure of the experiment cleaned up to a bit more useful state (and maybe even some docs!) by the end of the holiday season. There's still a long way to go with this. Some of which can be adapted from Llilium and or the CoreCLR work. The HAL layer and interop still needs to be defined, which was the biggest focus of my proposals for NETMF v5, not to mention the problem of the "soft dependencies". Although, in a surprise addition to CMSIS-5 there is generalized "pseudo-project" file that lists the files and headers in a source package but also lists dependencies on packages and interfaces that could serve well in that area. Whether this has any value to anyone or not is another story, How much functionality is required before it can be considered useful is an open question. |
Perhaps it would help to let the contributors know the reason why the PRs are getting stalled. I guess a simple comment/message "No PRs until #508 is fixed" will be sufficient and appreciated. Regarding the NETMF v5 - does it mean it will be AOT? Also, are you aware of CoreRT project? |
|
I've already made such a statement once but I can add it to all of the ones that have come on since then.
Maybe, maybe not - depends on the people who are willing and able to step up really. Ultimately there are a few questions there:
These aren't new questions but as of yet don't have a clear answer. There's been a lot of interest in an AOT solution but concerns over the time to a minimally viable product have people a bit on edge, leading to talk about tweaking the current NETMF a bit in the interim.
I am aware of the CoreRT project, in the past we've actually had a few conversations with the team driving that work. Unfortunately , the goals and requirements for CoreRT don't align well with those of a micro controller focused solution. While there is likely some things we can learn/leverage from there, it isn't useable as-is and trying to make it meet our needs has a high probability of getting in the way of CoreRT meeting its own goals. One of the bigger challenges CoreRT needs to deal with doesn't exist for the small micro space. (e.g. CoreRT has to contend with dynamic loading of modules and that those modules might have open generic types that other modules need to instantiate. ) On the flip side for the NETMF AOT space we have extreme memory constraints to contend with that CoreRT doesn't. Common concepts (and possibly code) like how types are laid out, how vtables are defined and used, etc... may also be useful to share. Ultimately, all such sharing in either direction needs to consider the impact to the first order goals of each project. |
|
My votes and opinions count for very little, but here are some of the priorities I see:
Personally, I would give up threading, the GC, and reflection in a heartbeat to have a high-performance close-to-the-metal AOT NETMF. This may be a controversial position, but it is mine. |
|
Again, Steve claims that "it's a lot more than you see", but perception is reality. And what I perceive is maybe one small commit a week — for more than half a year now. If there really is something "a lot more", then maybe it's just about time to show it? Steve still dreams of a very abstract and cool platform, that works on everything and has everything, but, judging on the opinions on this thread and in #491, it doesn't look like he has lots of supporters believing he can actually do it. V5 spec was a short spark of hope... that ended in 22 commits over 4 months. Now that the last big commercial passenger (==GHI) left the train, I see no reason why @josesimoes couldn't simply fork NETMF and go his own route, trying to bring in folks that don't like the way NETMF is currently going along. If he said "alright, let's just ditch MDK and KEIL support, GC and other 80% of code, let's just make it work, work fast, compile in 1 minute, and run on a 8$ Nucleo board out-of-the-box", that would certainly attract my attention. Surely, I'm not the center of the world, but that's my opinion anyway. |
|
@miloush While Llilum is interesting, Steve's proof-of-concept there is much more elegant. |
|
@Stabilitronas says:
I am making no such claims - The only thing I've said that I can think of that might have been interpreted that way was the point that the issues being discussed aren't as one sided as some claimed.
While the "Ditch" part of that is an open question here, the rest is essentially what I am saying I have a good basis for in the AOT work I've been doing. It isn't quite up to where its useful to publish (There's still some significant refactoring churn to complete) but I am hoping to get that done by the end of the holidays. It doesn't support all the base instructions and pretty much none of the object model ones yet, but one I get the refactoring done that is work that could be done by others. (A few major architectural choices regarding types and the GC need to be defined, but again multiple people can participate there especially in reviewing what CoreRT, Llilum, LLilc and any other similar platforms do to weigh the various tradeoffs etc...) Llilium has more features and functionality by a large factor than I currently do and may be a way to get some projects started despite its issues with the compilation time (and memory usage during compilation). I can't, and wouldn't if I could, dictate what happens next for the current NETMF code base. If there are enough people who see value in making the effort to V5 or some other proposed design - great let's go. Otherwise I'm going to continue my focus on the AOT tooling. If others want to join me on that, great. |
|
@smaillet-ms I don't want to dishonour what you did to NETMF, but again, three months later, you only tell that "there more you don't see it yet". Sorry man, that somewhat doesn't align with my understanding of what an open-source project is. You may be just be telling tales to us — there's simply no way to tell for sure. GHI, meanwhile, may be having 2-3 developers working on their own NETMF flavor full-time. Although I do not think of GHI as a real software company, but that may just make original NETMF simply inferior to GHI TinyCLR OS (or whatever it is) in a matter of months; moreover, they actually make hardware to run stuff on. So, here comes a very uncomfortable question: what will NETMF actually be able to offer then? If it doesn't run on a stock Nucleo board, where will it? "If others want to join me on that, great", you say. But this very thread, rematerializing every now and then in one form or another, simply tells that people just do not know how to do that exactly. So maybe you are doing something wrong, given that we all — including you and me — want NETMF to make progress and move on. |
|
@Stabilitronas GHI is almost exclusively a software company, from the NETMF perspective. They produce some simple ARM microcontroller breakout boards, but they're essentially just that; simple breakout boards. Nearly all of the value is in the software. @smaillet-ms I would say that the current state of NETMF and the OEMs that market(ed) could be seen as evidence that it's time for a new approach. A smaller, lighter, higher-performance (even if more feature-limited) competitor to Arduino and the like could be just what the community needs to get excited. |
@Stabilitronas I have no idea where you are inferring that's what I've said from anything I've said here or at any other time. But insulting my integrity by suggesting that I am in effect lying to you about what I've done for some undisclosed purpose is just uncalled for here. Let's keep this a civil discourse please. If you are referring to the AOT work then you are grossly misreading that. So let me re-iterate. On my own time I took it upon myself to look into whether a different approach to an AOT tool would work. I had some rather specific goals - in particular I wanted to be able to compile the code I showed here previously with no more overhead than C or C++. I've had that goal in mind for a few years and haven't really changed the test code I shoed here much since I first wrote it up. I've made a few attempts at this over the last few years and none of them really did the job so they've never been published. There are a bunch of subtle problems to solve in getting that to work and it wasn't until I was looking into C#7 where I started to see a really viable solution. I then decided to build a test app to determine if the idea could actually work. It actually didn't initially as there were still other problems to solve but it was close enough that I kept playing with my tests to get it to where I knew it could work. With that I expanded the test app to a bit more like a tool and actually try to compile the test app code in it's entirety. I got that to work but the code was ugly and relied on some code that is essentially considered deprecated. So I've updated it to use a different library and am still in the process of refactoring it to a point where it is useful enough, clean enough and documented enough to support OSS development. This is normal of any OSS project - Projects don't normally start out with a blank repo and an idea. They start out with a working implementation of the basic principles and then go from there.
Not uncomfortable to me - that's essentially the question being asked here. There does seems to be a good deal of hostility toward me for not providing a definitive answer, however so let me speak to that directly. I don't work for a business that has a financial stake in the future of NETMF. I, personally have no stake beyond my own history with NETMF and a love of the Idea of C# for micro controllers. I'm not so arrogant to think that what I might want is the one right true way. I've made some proposals and by and large that's not gone anywhere. The point I keep trying to make here is that I am not the only one who can make proposals or decide on what the future of this particular code base should be. That's ultimately up to the people who have real stakes here. Even if I tried to dictate every aspect of everything but nobody liked that - anyone could still fork and go on with their own variant. To a certain extent that's the path that GHI has chosen, while that is disappointing, I can't fault them for it - they have a business to run and have to make decisions based on that. As with any project, Open or closed really, in the end everyone with a real stake needs to asses for themselves whether active contribution, private forks or switching tech outright is appropriate. I won't make that choice for you. |
|
@smaillet-ms Keep moving forward Steve. Looking forward to what comes of your efforts. Hopefully any hostility is the result of communication breakdowns. |
|
@smaillet-ms Steve, you have always had my support, right back to the Embedded Solutions days. Thanks for introducing me to MF and for all the work you have done, especially the saving of MF after 4.3. |
|
Ok, now I am starting to understand...
I am sorry to say that, but the point you seem not to be getting is that technically you are the only one: a sole maintainer of the repo, who decide what changes make it in and what do not, thus affecting the future of the code base.
I guess the problem is that (perhaps for historical reasons) they are perceived as something you are already working on. In many cases, you have rejected a proposal with some kind of 'my plan is...' reason, which made us believe there is already something being actively developed, for example
I have postponed many things from my NETMF ToDo list waiting for the plans to materialize, and I think I was not the only one. Meanwhile, you have been waiting for people to step up... |
|
@godefroi software companies do not have assembly lines, 1M$ worth PnP machines and concrete polished, anti-stacic coated 2000 sq.m. manufacturing floors. Yes, value is at software side, but chips is what they sell, not software. As a software company, GHI is totally under-employed. |
|
@smaillet-ms I'm sorry Steve. I didn't mean any hostility. I used to love NETMF, but as a commercial entity, I'm pretty much much forced to drop NETMF support in all high-tech product I'm in charge of. Because, you know, whatever you say, one tiny commit per week is just not enough to call a project "active" these days IMO. And seems like I'm not the only one, given this very thread. As a software manager, when multiple people tell me "you're going the wrong way", I always re-evaluate my decisions, because it's a very good indicator that, in fact, I'm going the wrong way. In such cases, instead of furiously defending myself, I always ask as myself: maybe I'm wrong? Maybe I should shange my course? Maybe they're right? So please, take my criticism in a constructive way. But, again, you're the sole master of NETMF. I'm not questioning that. Other guys, like @josesimoes, will either fork it or let it go your way. Your way is one commit per week. Sorry man, that's just not enough to make NETMF great again these days. |
It’s been 3 months + 1 day since #491 was published. Along with #493, a reasonable amount of discussion was generated and some very interesting points were made. We even got a new branch on the repo for a “v5” where some high level documentation was stored.
Now let me ask you this: what has really changed since then?
Where is .NETMF today compared to where it was 3 months, 6 months, 12 months ago?
Has it moved forward?
Do we have more community engagement?
Were any of the main issues that plague .NETMF solved or at least addressed?
Do we have a plan? A roadmap?
Is there a team working on any of this?
I’ll dare to say that the .NETMF project hasn’t – if we want to be totally honest about it – moved an inch.
It’s about time that we stop beating around the bush and get to the point: .NETMF is not getting anywhere and it won’t go anywhere with what is there today!! This should have become quite obvious by now...
@smaillet-ms, because you are .NETMF project main contact let me ask you this: what else is needed for you to let community people in? What else is needed for you to relinquish total control of .NETMF?
If nothing changes (and I mean drastically and really fast) the moribund .NETMF will be completely dead. (Except on a few repos here and there were it will live forever that belong to people that really love it and will keep working on it no matter what)
Note: this will be my last take on .NETMF future and governance. I’ve made a few attempts to shake people’s minds and have them contribute and step in. If I’m the only one seeing things this way, then I’m the one who is wrong and I have to shut the f#$% up.
The text was updated successfully, but these errors were encountered: