Data error during extraction

[BiatuAutMiahn]

Error when running example:

Traceback (most recent call last):
  File "E:\Python\7ztest.py", line 38, in <module>
    sevenZfile.extractall('.')
  File "E:\Python\7ztest.py", line 33, in extractall
    outfile.write(self.archive.getmember(name).read())
  File "E:\Python\py7zlib.py", line 632, in read
    data = getattr(self, decoder)(coder, data, level, num_coders)
  File "E:\Python\py7zlib.py", line 717, in _read_lzma2
    return self._read_from_decompressor(coder, dec, input, level, num_coders, with_cache=True)
  File "E:\Python\py7zlib.py", line 688, in _read_from_decompressor
    data = decompressor.decompress(input)
ValueError: data error during decompression

Source

import py7zlib
import os

class SevenZFile(object):
    @classmethod
    def is_7zfile(cls, filepath):
        '''
        Class method: determine if file path points to a valid 7z archive.
        '''
        is7z = False
        fp = None
        try:
            fp = open(filepath, 'rb')
            archive = py7zlib.Archive7z(fp)
            n = len(archive.getnames())
            is7z = True
        finally:
            if fp:
                fp.close()
        return is7z

    def __init__(self, filepath):
        fp = open(filepath, 'rb')
        self.archive = py7zlib.Archive7z(fp)

    def extractall(self, path):
        for name in self.archive.getnames():
            outfilename = os.path.join(path, name)
            outdir = os.path.dirname(outfilename)
            if not os.path.exists(outdir):
                os.makedirs(outdir)
            outfile = open(outfilename, 'wb')
            outfile.write(self.archive.getmember(name).read())
            outfile.close()
			
if SevenZFile.is_7zfile('DP_LAN_Realtek-XP_18000.7z'):
    sevenZfile = SevenZFile('DP_LAN_Realtek-XP_18000.7z')
    sevenZfile.extractall('.')

[fancycode]

Could you please provide a (small) sample file that shows the error? Also which version of pylzma are you running?.

[Jimmy-Jon]

Hello fancycode, thank you for responding to the above comment because I am having the same problem as the original poster. Here is my extractall function that I use on 7z archives:

 def extractall(path):
    with open(item, 'rb') as fp:
      archive = py7zlib.Archive7z(fp)
      for name in archive.getnames():
        outfilename = os.path.join(path, name)
        outdir = os.path.dirname(outfilename)
        if not os.path.exists(outdir):
          os.makedirs(outdir)
        with open(outfilename, 'wb') as outfile:
          acv = archive.getmember(name)
          outfile.write(acv.read())
       

     extractall(path=os,curdir)

I've narrowed down where the data errors come from, most notably from executable files (.exe on windows), which when I extract create the proper executable file name, but will be a zero-byte file. It also appears .dll files will completely fail to extract and return the data error. However, this function works perfectly with a folder full of JPEG wallpapers or XML files, dll and exe files are the only files I noticed to return errors. I really would like to use this function since it's recursive and can do folders/subfolders. If you need anything else to help towards narrowing out this problem just ask and I will see what I can provide.

[fancycode]

Master now supports various BCJ filter, could you please check if this solves the issues you were having?

[Jimmy-Jon]

This new addition works, but only up to 128 kilobytes, after it hits that limit it stops reading and creates zero-byte files again. I verified a 70 kilobyte file with Md5Checker and the hash was exact, it just seems there is this data limit imposed consistently, which applies to .dll and executables.

[fancycode]

Should be fixed with the latest change, could you please test again?

[Jimmy-Jon]

Works perfectly! A pythonic way to extract 7z archives is amazing. All .dll files and .exe file hashes match. If I uncover anything else I can submit a new error report, thank you!

[fancycode]

Great, thanks for reporting & testing!

[embray]

Incidentally I just had the same problem and this fixed it. Any chance of getting a new release on PyPI with this fix? Thanks!

[BiatuAutMiahn]

Thanks @fancycode fir the fix, sorry for MIA catching up on subscriptions

[BiatuAutMiahn]

Successfully installed pylzma-v0.5.0-17-gccb0dev

Traceback (most recent call last):
  File "C:\System\Users\Biatu\Dev\Python\DriverMgr\dev_drivermgr.py", line 38, in <module>
    sevenZfile.extractall('.')
  File "C:\System\Users\Biatu\Dev\Python\DriverMgr\dev_drivermgr.py", line 33, in extractall
    outfile.write(self.archive.getmember(name).read())
  File "C:\ProgramData\Miniconda3\lib\site-packages\py7zlib.py", line 650, in read
    data = getattr(self, decoder)(coder, data, level, num_coders)
  File "C:\ProgramData\Miniconda3\lib\site-packages\py7zlib.py", line 750, in _read_lzma2
    return self._read_from_decompressor(coder, dec, input, level, num_coders, with_cache=True)
  File "C:\ProgramData\Miniconda3\lib\site-packages\py7zlib.py", line 717, in _read_from_decompressor
    data = decompressor.decompress(input, self._start+total_decompressed)
ValueError: data error during decompression

Process returned 1 (0x1)        execution time : 1.242 s
Press any key to continue . . .

Sources+7z file: https://drive.google.com/file/d/1YUgE15Tt2OS07X6Yzc_afc98UjRsfbQs/view?usp=sharing

[fancycode]

So this still happens on master for you 😞 Could you please provide a (small) file I can use for testing?

[BiatuAutMiahn]

Check my edit, included sources