Moved winetricks to submodule

[Root-Core]

It feels like a logical step, are there any problems that could arise?

The rationale is that it's easier to verify the origin of winetricks.

I added this when I implemented the verb check, because I used some files first... but they are outdated and I don't need them anymore. It is still a change I wanted to discuss.

EDIT: This could also cleanup the statistics of used languages.. as we have 84.1% Shell code.

[R1kaB3rN]

Have to check in about this -- whether we want to always use the latest winetricks. I think we do though. However, I think it would be better if we fork winetricks instead. That way, we wouldn't be fully dependent on them when things break, especially when they are relatively trivial to fix (e.g., digest mismatches).

Once we create our own fork, we'd also have to update the Makefile so it will be added to the dist directory.

[Root-Core]

Winetricks is moving pretty slow and the releases are getting quite stale. We want to use the latest, I think.

The submodules are actually fixed on a specific commit though, so we don't need to use the latest. There's really no difference from the current solution.

---

I'm working on a reimplementation of Winetricks in Python. I can't give an ETA at the moment, but I have some groundwork and I'm working on an importer that converts the verbs into a maintainable file format.

Winetricks is huge though... and slow. It has a lot of functionality, so it will take some time to implement all of it.

I hope to finish work on it, but don't hold your breath. I'm not sure if I'm motivated enough to finish this little side project.

[Root-Core]

We can actually override verbs for small fixes by adding them to the local verbs. So we don't need to fork.

[R1kaB3rN]

I'm working on a reimplementation of Winetricks in Python. I can't give an ETA at the moment, but I have some groundwork and I'm working on an importer that converts the verbs into a maintainable file format.

Winetricks is huge though... and slow. It has a lot of functionality, so it will take some time to implement all of it.

I hope to finish work on it, but don't hold your breath. I'm not sure if I'm motivated enough to finish this little side project.

This is great. Yes, I have been deliberating about a winetricks rewrite for a bit and had kinda planned for this, actually. Like the rewrite would be tailored to Proton, only containing the most useful verbs and metadata of them would be separated from the main program to separate files (e.g., toml, json, etc.). And while I imagine a rewrite in Python would speed up its runtime execution, be simpler, and better designed, I believe only part of its execution would be sped up unless we do not create subprocesses to wine binaries like the original script or perhaps if the registration of DLLs is possible to do in a thread-safe manner (which I haven't really explored yet).

[Root-Core]

I think it would be modules that can be imported with a CLI that imports the modules as well. Since the verbs would be in individual files, it doesn't really matter if there are only the most useful verbs or all of them. I will implement the most used functions of the verbs first, which should port most of the verbs pretty fast.

I believe only part of its execution would be sped up unless we do not create subprocesses to wine binaries like the original script or perhaps if the registration of DLLs is possible to do in a thread-safe manner (which I haven't really explored yet).

This is an issue, but I think some verbs are overkill.. like installing a complete software suite for having a dll.. but to refactor and test those verbs, we would need way more manpower. Not realistic atm.. but registering dlls or changing the registry might be viable.

[Heus-Sueh]

Sorry to get involved in the conversation here, but have you ever thought about forking the Wine dependencies repository from Bottles? I think it would be interesting to move away from Winetricks completely. The only problem would be the different names of the dependencies in the yaml files, but then you could create aliases or rename the ones that are different from Winetricks.

[Root-Core]

I've never looked at Bottles because it never worked that well for me.

The repo looks interesting, but it's only the "database" part, and it only has the most important stuff yet.
From a quick look, I think the actual code is in the Bottles backend. So it might take some work to get it working.

Thanks for the input, I didn't find this repo in my research. With the addition of our own yaml files, that could do the trick.

I'm still interested in a independent implementation of something like Winetricks, maybe also featuring a usable GUI.

[Root-Core]

A quick check what's missing in their database:

# Used in umu-protontricks
a = set(['allfonts', 'amstream', 'arial', 'cinepak', 'cnc_ddraw', 'cncnet_ra2', 'corefonts',
        'courier', 'crypt32', 'd3dcompiler_42', 'd3dcompiler_43', 'd3dcompiler_46', 'd3dcompiler_47',
        'd3dx10', 'd3dx11_42', 'd3dx11_43', 'd3dx9', 'd3dx9_41', 'd3dx9_42', 'd3dx9_43', 'd3dxof',
        'devenum', 'dgvoodoo2', 'dinput8', 'directmusic', 'directplay', 'directshow', 'dmband',
        'dmime', 'dmloader', 'dmstyle', 'dmsynth', 'dmusic', 'dotnet35', 'dotnet35sp1', 'dotnet40',
        'dotnet45', 'dotnet452', 'dotnet462', 'dsound', 'dswave', 'gdiplus', 'gui',
        'hidewineexports=enable', 'icodecs', 'klite', 'l3codecx', 'lavfilters', 'lucida', 'mdx',
        'mfc140', 'mfc42', 'mfc90', 'msxml3', 'ole32', 'oleaut32', 'openal', 'physx', 'powershell',
        'qasf', 'quartz', 'quartz_feb2010', 'rsx3d', 'segoe_script', 'sound=alsa', 'tahoma', 'trebuchet',
        'urlmon', 'vcrun2010', 'vcrun2017', 'vcrun2019', 'vcrun2022', 'vd=1280x720', 'verdana', 'win10',
        'win7', 'wininet', 'winxp', 'wmp11', 'wmp9', 'xact', 'xact_x64', 'xaudio29', 'xliveless'])

# Bottles
b = set(['aairruntime', 'allfonts', 'amstream', 'andale32', 'arial32', 'arialb32',
        'art2k7min', 'art2kmin', 'atmlib', 'cjkfonts', 'cnc-ddraw', 'comic32', 'consolas',
        'corefonts', 'courie32', 'd3dcompiler_42', 'd3dcompiler_43', 'd3dcompiler_46',
        'd3dcompiler_47', 'd3dx11', 'd3dx9', 'devenum', 'dirac', 'directmusic',
        'directplay', 'directshow', 'dmband', 'dmcompos', 'dmime', 'dmloader',
        'dmscript', 'dmstyle', 'dmsynth', 'dmusic', 'dmusic32', 'dotnet20', 'dotnet20sp1',
        'dotnet35', 'dotnet35sp1', 'dotnet40', 'dotnet45', 'dotnet452', 'dotnet46', 'dotnet461',
        'dotnet462', 'dotnet472', 'dotnet48', 'dotnetcore3', 'dotnetcoredesktop3', 'dotnetcoredesktop6',
        'dotnetcoredesktop7', 'dotnetcoredesktop8', 'dsdmo', 'dsound', 'dswave', 'dx8vb', 'ffdshow',
        'gdiplus', 'gecko', 'georgi32', 'gfw', 'gmdls', 'ie8_kb2936068', 'iertutil', 'impact32', 'jet40',
        'l3codecx', 'lavfilters702', 'lavfilters741', 'lucon', 'mdac28', 'mediafoundation', 'mfc40', 'mfc42',
        'mono', 'msasn1', 'msftedit', 'msls31', 'mspatcha', 'msxml3', 'msxml4', 'msxml6', 'physx', 'powershell',
        'powershell_core', 'qasf', 'qcap', 'qdvd', 'qedit', 'quartz', 'quicktime72', 'riched20', 'tahoma32',
        'times32', 'trebuc32', 'unifont', 'urlmon', 'vbrun6', 'vcredist2005', 'vcredist2008', 'vcredist2010',
        'vcredist2012', 'vcredist2013', 'vcredist2015', 'vcredist2019', 'vcredist2022', 'vcredist6',
        'vcredist6sp6', 'verdan32', 'webdin32', 'webview2', 'winhttp', 'wininet', 'wsh57', 'xact',
        'xact_x64', 'xinput', 'xna31', 'xna40'])

# Exclude
x = set(['gui', 'win10', 'win7', 'winxp', 'sound=alsa', 'vd=1280x720', 'hidewineexports=enable'])

# Mapping winetricks -> bottles
d = {
    # Fonts
    'arial': 'arial32',
    'courier': 'courie32',
    'tahoma': 'tahoma32',
    'trebuchet': 'trebuc32',
    'verdana': 'verdan32',

    # Renamed
    'cnc_ddraw': 'cnc-ddraw',
    'lavfilters': 'lavfilters741',
    'vcrun2010': 'vcredist2010',
    'vcrun2019': 'vcredist2019',
    'vcrun2022': 'vcredist2022',

    # Confident (part of meta package)
    'd3dx11_42': 'd3dx11',
    'd3dx11_43': 'd3dx11',
    'd3dx9_41': 'd3dx9',
    'd3dx9_42': 'd3dx9',
    'd3dx9_43': 'd3dx9',
    'mfc140': 'vcredist2015',

    # Speculative
    'oleaut32': 'vbrun6',
}

# Used without found in bottles, excluded and mapped
print(sorted(a - b - x - d.keys()))

Result:

['cinepak', 'cncnet_ra2', 'crypt32', 'd3dx10', 'd3dxof', 'dgvoodoo2', 'dinput8',
'icodecs', 'klite', 'lucida', 'mdx', 'mfc90', 'ole32', 'openal', 'quartz_feb2010',
'rsx3d', 'segoe_script', 'vcrun2017', 'wmp11', 'wmp9', 'xaudio29', 'xliveless']

Where these are local verbs:

['cncnet_ra2', 'dgvoodoo2', 'klite', 'rsx3d', 'segoe_script', 'xliveless']

Remaining:

['cinepak', 'crypt32', 'd3dx10', 'd3dxof', 'dinput8', 'icodecs', 'lucida', 'mdx',
'mfc90', 'ole32', 'openal', 'quartz_feb2010', 'vcrun2017', 'wmp11', 'wmp9', 'xaudio29']

[Heus-Sueh]

seeing the horrible state of winetricks where all dependencies are in giant bash file, I think it's worth the effort to try to complete this bottles database, I'm willing to help in any way possible

[R1kaB3rN]

Yeah, I'm up for utilizing the Bottles database in the rewrite. Certainly, I think a rewrite will need to separate the metadata away from the main program. Doing so would allow for better maintenance, project contribution, and automation (e.g., automating integrity of archives). Bottles has the right idea there, but I think it should probably include more metadata.

There's libwine from Bottles which may be useful here, but we don't necessarily need to limit ourselves to Python if one of the primary goals of the rewrite is to be fast. Winetricks is a package manager, so Python may not even be the right tool for the job to meet that goal. If another programming language would be more suitable give its inherent properties or available libraries (e.g., Rust) I think we should use that. No need to worry about OS compatibility either, as we'll be compiling the project within a predictable environment (Steam Runtime). Though, like I mentioned above, I suspect the main bottleneck to be the sequential registration of DLLs and subprocess calls to wine binaries.

As for preferences of other languages, I would be fine with Rust or Go. Though, more people in OWC are familiar with the former.

[R1kaB3rN]

In one of Bottle's blogs, they even touch on this issue a bit and their solution for it in Bottles Next is winebridge.

[Root-Core]

Bottles has the right idea there, but I think it should probably include more metadata.

Yeah.. Bottles seems to have done most of the stuff I was trying to achieve, and they have the funding.
This is a dent in my motivation, but the parsed verbs might still be useful. The format could certainly be improved, and there should be more than two categories. But this is also true for Winetricks.

However, creating compatible entries for the database for each missing verb could be done by hand. No need for my parser, they should be straightforward. Some probably won't be needed anymore.

Though, like I mentioned above, I suspect the main bottleneck to be the sequential registration of DLLs and subprocess calls to wine binaries.

Bottles Next sounds a bit overkill, but if it works out for them, I'm all for it.
I'm not yet convinced, that they will gain that much execution speed - that couldn't be achieved in a simpler way. You still have to run the agent in the wine prefix, which is slower than modifying the prefix files externally (if possible) or creating a batch script with all the necessary steps and running wine just once.

Installing heavy fixes - like wmp - will still take about the same amount of time. The only way around this would be to use some kind of snapshot/diff... but that would probably be "legally complex".

but we don't necessarily need to limit ourselves to Python if one of the primary goals of the rewrite is to be fast.

As you correctly pointed out, Python is not the bottleneck here.
But you're still right, we don't have to limit ourselves to Python.
Rust is quite interesting, but the parsers are sometimes not well supported. On the other hand, Go is used by Bottles Next, which might be useful in the future. I don't know much about Go, so I can't say how it fits our needs. It looks easy enough.

Everyone knows Python, even if they don't. For the scope of protonfixes, it just does the job.
I'm not sure what would be the best language for a Winetricks replacement, though.

[Heus-Sueh]

I think it's worth creating a separate issue or discussion for this, and asking other members for their opinions regarding this rewrite. I believe that heroic and lutris also depend on winetricks and would benefit from it.

[R1kaB3rN]

I think Heroic would be pretty interested in a rewrite, as they allow macOS users to use winetricks. cc @imLinguin.
So we may want the rewrite to be used outside the scope of protonfixes as well.

Go was designed with concurrency as its primary goal, and it makes achieving it trivial and cheap via channels and goroutines. Like Python, it also features a simple and concise syntax. If we want to do things concurrently, I believe this is the most appropriate tool.

[Root-Core]

Yes, the separate issue is a good thing.

Back to the topic: This PR is still a "better" solution than the copied Winetricks file, imho.
So do we want to merge it, or should we close it if there is no chance for a merge?

[R1kaB3rN]

Looks good to me. No need to fork either, like you mentioned.

Moving forward, any discussion or thoughts on the rewrite should be moved to the issue tracker as this discussion was going outside this PR's scope.

And in case upstream fails to push a fix for a problem (e.g., digest mismatches) a custom verb should be made at ./verbs to override it.