[Bug]: Issues regarding the Fat32 and Exfat file systems

[psvajaz]

bit7z version

4.0.x

Compilation options

BIT7Z_AUTO_FORMAT, BIT7Z_USE_NATIVE_STRING

7-zip version

v23.01

7-zip shared library used

7z.dll / 7z.so

Compilers

MSVC

Compiler versions

MSVC2017

Architecture

x86_64, x86

Operating system

Windows

Operating system versions

Windows 7 or last

Bug description

There are two questions:

1. If compressed memory variables are used for decompression, the directory where the program is located will be opened, causing an exception in the fs object
2. If decompressing compressed packages in the fat32 or exfat file system can also cause fs object exceptions
   Our project's development environment is VS2017 and C++17, which may encounter the above issues. After our testing, using VS2019 or later versions and C++17 will not cause such problems. However, the project relies on many static libraries, all developed using VS2017, which makes it impossible to upgrade to the VS2019 version.

Steps to reproduce

No response

Expected behavior

No response

Relevant compilation output

No response

Code of Conduct

• By submitting this issue, I agree to follow bit7z's Code of Conduct

[rikyoz]

Hi!
Could you provide some sample code to replicate the issues you described?

If compressed memory variables are used for decompression, the directory where the program is located will be opened, causing an exception in the fs object

If I understand correctly, are you extracting an archive from a memory buffer to the filesystem?

If decompressing compressed packages in the fat32 or exfat file system can also cause fs object exceptions

Just to make sure I understood correctly: you're extracting archives to fat32 and exFAT filesystems, right? What archive format are you trying to extract?

Our project's development environment is VS2017 and C++17, which may encounter the above issues. After our testing, using VS2019 or later versions and C++17 will not cause such problems.

This makes me think that it might be some bug in the std::filesystem implementation of VS2017, as bit7z internally uses it for things like paths.
But I'll need more information to understand the cause of the issue.
Thank you for any further info you can provide!

[psvajaz]

1. When decompressing from stream to stream, fs will be used to open the directory where the current executing program is located (i.e.‘./’). If the current executing program is stored in the FAT32 or exFAT file system, an exception will occur

2. If the compressed file to be decompressed exists in the FAT32 or exFAT file system, or if it needs to be decompressed to FAT32 or exFAT, an fs exception will occur

The above error triggering condition is VS2017 C++17. If the development changes the environment to VS2019 or higher, this issue will not occur

[rikyoz]

Thanks for the further details!
I was able to replicate the issue; I'm working on a fix.
Implementing it might take some time, as it will likely require a bit of refactoring, but I'll release it as soon as possible!

[rikyoz]

Hi @psvajaz! Yesterday, I pushed a commit to the hotfix/v4.0.5 branch that should fix the issue.
If you can test it, it would really be helpful!
Thank you in advance!

[psvajaz]

@rikyoz
After testing:

1. Compressed packages saved in FAT32 or exFAT file systems can be decompressed to NTFS, FAT32, and exFAT file systems as usual.

2. Can compress and output regular files in NTFS to FAT32 and exFAT file systems

3. The compressed file cannot be loaded properly in the file system of FAT32 or exFAT, causing the compression process to fail. The specific compression error location is shown in the following figure:
   fsitem.cpp
   

[rikyoz]

Ok, so the fix was only partial. Thanks for the further tests!

The compressed file cannot be loaded properly in the file system of FAT32 or exFAT, causing the compression process to fail. The specific compression error location is shown in the following figure:
fsitem.cpp

Unfortunately, it seems to be a bug in old versions of MSVC (microsoft/STL#233).

I'm working on finding a possible workaround.

[rikyoz]

@psvajaz
I just pushed a commit to the branch hotfix/v4.0.5 with a workaround for the bug.

Now, you can pass the BIT7Z_DISABLE_USE_STD_FILESYSTEM option when configuring bit7z.
This will force the library to always use the ghc::filesystem library instead of the std::filesystem provided by the compiler.

From my tests, this library doesn't suffer from the same bug as MSVC's std::filesystem and seems to fix the issue.

Please let me know if this works for you, too!

[psvajaz]

After extensive testing, the bug has been fixed. thanks