Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Figure out why MAX_PATH is still a problem in Cabal #10191

Closed
jasagredo opened this issue Jul 9, 2024 · 2 comments · Fixed by #10366
Closed

Figure out why MAX_PATH is still a problem in Cabal #10191

jasagredo opened this issue Jul 9, 2024 · 2 comments · Fixed by #10366

Comments

@jasagredo
Copy link
Collaborator

Base should be fixed once WinIO was merged, but there are still tests which even in newer GHCs still fail with MAX_PATH issues:

 C:\Users\runneradmin\AppData\Local\Temp\cabal-testsuite-26120\cabal-external.dist\work\.\dist\build\x86_64-windows\ghc-9.10.1\indef-0.1.0.0\indef-0.1.0.0-inplace+3YxWX6XJM8s5QouYtC2nTB\build\indef-0.1.0.0-inplace+3YxWX6XJM8s5QouYtC2nTB\autogen\Data\: openBinaryTempFileWithDefaultPermissions: invalid argument (CallStack (from HasCallStack):

  withMetadata, called at src\Distribution\Simple\Utils.hs:459:7 in Cabal-3.13.0.0-inplace:Distribution.Simple.Utils

Mistuke mentioned that Cabal uses its own logic for temp files, but this one looks like it uses the base one. This is worth investigating in any case, because Backpack results in very long paths always...

System information

  • Operating system Windows
  • cabal, ghc versions: 3.12.1.0 and 9.0.2-9.10.1
@jasagredo
Copy link
Collaborator Author

jasagredo commented Jul 15, 2024

Tests that are currently marked as failing because of this:

  • PackageTests\Backpack\Includes2\cabal-internal.test.hs
  • PackageTests\Backpack\Includes3\cabal-external.test.hs passing with GHC < 9.10
  • PackageTests\Backpack\Includes3\cabal-internal.test.hs
  • PackageTests\Backpack\T6385\cabal.test.hs

jasagredo added a commit to jasagredo/cabal that referenced this issue Sep 16, 2024
This helps with Windows not being capable of creating temp files in
long directories, like the ones that result from Backpack.

See how GetTempFileNameW specifies:

> The string cannot be longer than MAX_PATH–14 characters or GetTempFileName
will fail.

And actually there is a TODO in `Win32Utils.c` in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

Closes haskell#10191.
@jasagredo
Copy link
Collaborator Author

jasagredo commented Sep 18, 2024

Cabal uses temporary files for writing files atomically. It then renames the file to the right name after writing it. However this cannot deal with long paths on Windows.

See how GetTempFileNameW specifies:

The string cannot be longer than MAX_PATH–14 characters or GetTempFileName will fail.

And actually there is a TODO in Win32Utils.c in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

So my suggestion and the PR I'm preparing is to create temp files in the global temp directory instead of in the final destination. This is somewhat of a breaking change, but I don't think it will be too noticeable.

However this is kind of blocked on haskell/directory#189 to make this transparent for cabal, as otherwise users with multiple drives might have a hard time.

jasagredo added a commit to jasagredo/cabal that referenced this issue Sep 25, 2024
This change ensures all temporal files are created in the system temp directory
which usually is in a short path. This helps with Windows not being capable of
creating temp files in long directories, like the ones that result from
Backpack.

See how GetTempFileNameW specifies:

> The string cannot be longer than `MAX_PATH–14` characters or `GetTempFileName`
will fail.

And actually there is a TODO in `Win32Utils.c` in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

Closes haskell#10191.
jasagredo added a commit to jasagredo/cabal that referenced this issue Sep 30, 2024
This change ensures all temporal files are created in the system temp directory
which usually is in a short path. This helps with Windows not being capable of
creating temp files in long directories, like the ones that result from
Backpack.

See how GetTempFileNameW specifies:

> The string cannot be longer than `MAX_PATH–14` characters or `GetTempFileName`
will fail.

And actually there is a TODO in `Win32Utils.c` in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

Closes haskell#10191.
jasagredo added a commit to jasagredo/cabal that referenced this issue Sep 30, 2024
This change ensures all temporal files are created in the system temp directory
which usually is in a short path. This helps with Windows not being capable of
creating temp files in long directories, like the ones that result from
Backpack.

See how GetTempFileNameW specifies:

> The string cannot be longer than `MAX_PATH–14` characters or `GetTempFileName`
will fail.

And actually there is a TODO in `Win32Utils.c` in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

Closes haskell#10191.
jasagredo added a commit to jasagredo/cabal that referenced this issue Sep 30, 2024
This change ensures all temporal files are created in the system temp directory
which usually is in a short path. This helps with Windows not being capable of
creating temp files in long directories, like the ones that result from
Backpack.

See how GetTempFileNameW specifies:

> The string cannot be longer than `MAX_PATH–14` characters or `GetTempFileName`
will fail.

And actually there is a TODO in `Win32Utils.c` in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

Closes haskell#10191.
jasagredo added a commit to jasagredo/cabal that referenced this issue Oct 1, 2024
This change ensures all temporal files are created in the system temp directory
which usually is in a short path. This helps with Windows not being capable of
creating temp files in long directories, like the ones that result from
Backpack.

See how GetTempFileNameW specifies:

> The string cannot be longer than `MAX_PATH–14` characters or `GetTempFileName`
will fail.

And actually there is a TODO in `Win32Utils.c` in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

Closes haskell#10191.
Mikolaj pushed a commit to jasagredo/cabal that referenced this issue Oct 3, 2024
This change ensures all temporal files are created in the system temp directory
which usually is in a short path. This helps with Windows not being capable of
creating temp files in long directories, like the ones that result from
Backpack.

See how GetTempFileNameW specifies:

> The string cannot be longer than `MAX_PATH–14` characters or `GetTempFileName`
will fail.

And actually there is a TODO in `Win32Utils.c` in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

Closes haskell#10191.
@mergify mergify bot closed this as completed in #10366 Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant