Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running Denoiser... (Stuck) #14

Open
justme7678876788 opened this issue May 31, 2023 · 4 comments
Open

Running Denoiser... (Stuck) #14

justme7678876788 opened this issue May 31, 2023 · 4 comments

Comments

@justme7678876788
Copy link

Hi . after testing this on windows and not working i went to linux . after i installed it in linux i had some problems and they were all about the requirements versions which i found the right version and everything seems to go fine until the last step when it does this :

abder@abder-X75A1:~/Downloads$ /home/abder/.local/bin/pcleaner clean index.jpeg
Found 1 image.
Running text detection AI model...
Using device for text detection model: cpu
Using 1 processes for text detection.
100%|█████████████████████████████████████████████| 1/1 [00:13<00:00, 13.91s/it]


Running box data Pre-Processor...
2023-05-31 15:51:41.961 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
2023-05-31 15:51:53.992 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
2023-05-31 15:51:57.097 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
100%|█████████████████████████████████████████████| 1/1 [00:03<00:00,  3.72s/it]

OCR Analytics
-------------
Number of boxes: 2 | Number of small boxes: 2 (100%)
Number of removed boxes: 0 (0% total, 0% of small boxes)

Small box sizes:
   0- 500:  0 / 0
 500-1000:  0 / 0
1000-1500:  0 / 0
1500-2000:  0 / 0
2000-2500: ███████████████████████████████████████████████████████ 0 / 2
2500-3000:  0 / 0

█ Small boxes | █ Removed boxes

Removed bubbles:


Running Masker...
100%|█████████████████████████████████████████████| 1/1 [00:01<00:00,  1.64s/it]

Mask Fitment Analytics
----------------------
Total boxes: 2 | Masks succeeded: 0 (0%) | Masks failed: 2
Perfect masks: 0 (N/A) | Average border deviation: N/A

Mask usage by mask size (smallest to largest):
Box mask:  0 / 0

█ Perfect | █ Total

Pages with failures / total:
index.png: 2 / 2


Running Denoiser...
  0%|                                                     | 0/1 [00:00<?, ?it/s]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/denoiser.py", line 29, in denoise_page
    cleaned_image.paste(mask_image, (0, 0), mask_image)
  File "/usr/lib/python3/dist-packages/PIL/Image.py", line 1557, in paste
    self.im.paste(im, box, mask.im)
ValueError: bad transparency mask
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/abder/.local/bin/pcleaner", line 8, in <module>
    sys.exit(main())
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/main.py", line 219, in main
    run_cleaner(
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/main.py", line 418, in run_cleaner
    for analytic in tqdm(pool.imap(dn.denoise_page, data), total=len(data)):
  File "/home/abder/.local/lib/python3.10/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
ValueError: bad transparency mask

and here it is with another image :

abder@abder-X75A1:/media/abder/Stuffs/Manga/Blade of the Immortal (1997-2015) (Digital) (danke-Empire)/Blade of the Immortal v01 - Blood of a Thousand (1997) (Digital) (danke-Empire)$ /home/abder/.local/bin/pcleaner clean im7.jpg
Found 1 image.
Running text detection AI model...
Using device for text detection model: cpu
Using 1 processes for text detection.
100%|█████████████████████████████████████████████| 1/1 [00:12<00:00, 12.52s/it]


Running box data Pre-Processor...
2023-05-31 15:54:55.353 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
2023-05-31 15:55:00.636 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
2023-05-31 15:55:02.815 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 478.36it/s]

OCR Analytics
-------------
Number of boxes: 2 | Number of small boxes: 0 (0%)
Number of removed boxes: 0 (0% total, N/A of small boxes)
No not-removed small boxes found.


Removed bubbles:


Running Masker...
100%|█████████████████████████████████████████████| 1/1 [00:02<00:00,  2.02s/it]

Mask Fitment Analytics
----------------------
Total boxes: 2 | Masks succeeded: 2 (100%) | Masks failed: 0
Perfect masks: 1 (50%) | Average border deviation: 0.06

Mask usage by mask size (smallest to largest):
Mask 0  :  0 / 0
Mask 1  :  0 / 0
Mask 2  :  0 / 0
Mask 3  :  0 / 0
Mask 4  :  0 / 0
Mask 5  :  0 / 0
Mask 6  :  0 / 0
Mask 7  :  0 / 0
Mask 8  :  0 / 0
Mask 9  :  0 / 0
Mask 10 : ████████████████████████████████████████████████████████ 1 / 1
Box mask: ████████████████████████████████████████████████████████ 0 / 1

█ Perfect | █ Total

Pages with failures / total:


Running Denoiser...
  0%|                                                     | 0/1 [00:00<?, ?it/s]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/denoiser.py", line 29, in denoise_page
    cleaned_image.paste(mask_image, (0, 0), mask_image)
  File "/usr/lib/python3/dist-packages/PIL/Image.py", line 1557, in paste
    self.im.paste(im, box, mask.im)
ValueError: bad transparency mask
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/abder/.local/bin/pcleaner", line 8, in <module>
    sys.exit(main())
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/main.py", line 219, in main
    run_cleaner(
  File "/home/abder/.local/lib/python3.10/site-packages/pcleaner/main.py", line 418, in run_cleaner
    for analytic in tqdm(pool.imap(dn.denoise_page, data), total=len(data)):
  File "/home/abder/.local/lib/python3.10/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
    raise value
ValueError: bad transparency mask
@justme7678876788
Copy link
Author

Btw . here's my problem in the windows version :

E:\Manga\Blade of the Immortal (1997-2015) (Digital) (danke-Empire)\Blade of the Immortal v01 - Blood of a Thousand (1997) (Digital) (danke-Empire)>pcleaner clean im6.jpg
Found 1 image.
Running text detection AI model...
Using device for text detection model: cpu
Using 1 processes for text detection.
100%|████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:18<00:00, 18.77s/it]


Running box data Pre-Processor...
2023-05-31 19:33:32.080 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
2023-05-31 19:33:40.137 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
2023-05-31 19:33:44.564 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\Scripts\pcleaner.exe\__main__.py", line 7, in <module>
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\main.py", line 221, in main
    run_cleaner(
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\main.py", line 350, in run_cleaner
    ocr_analytic = pp.prep_json_file(
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\pre_processor.py", line 105, in prep_json_file
    page_data.grow_boxes(scale_len(pre_processor_conf.box_padding_initial), st.BoxType.BOX)
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\structures.py", line 105, in grow_boxes
    image_width, image_height = self.image_size
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\structures.py", line 78, in image_size
    metadata = magic.from_file(self.image_path)
AttributeError: module 'magic' has no attribute 'from_file'. Did you mean: '_image_size'?

it looks like a mistake in the code it self

@justme7678876788
Copy link
Author

justme7678876788 commented May 31, 2023 •
edited
Loading

after downloading manually the libmagic it seems like the error changed

E:\Manga\Blade of the Immortal (1997-2015) (Digital) (danke-Empire)\Blade of the Immortal v01 - Blood of a Thousand (1997) (Digital) (danke-Empire)>pcleaner clean im6.jpg
Found 1 image.
Running text detection AI model...
Using device for text detection model: cpu
Using 1 processes for text detection.
100%|████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:17<00:00, 17.89s/it]


Running box data Pre-Processor...
2023-05-31 20:28:42.812 | INFO     | manga_ocr.ocr:__init__:13 - Loading OCR model from kha-white/manga-ocr-base
2023-05-31 20:28:52.518 | INFO     | manga_ocr.ocr:__init__:22 - Using CPU
2023-05-31 20:28:56.608 | INFO     | manga_ocr.ocr:__init__:29 - OCR ready
  0%|                                                                                            | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\Scripts\pcleaner.exe\__main__.py", line 7, in <module>
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\main.py", line 221, in main
    run_cleaner(
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\main.py", line 350, in run_cleaner
    ocr_analytic = pp.prep_json_file(
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\pre_processor.py", line 105, in prep_json_file
    page_data.grow_boxes(scale_len(pre_processor_conf.box_padding_initial), st.BoxType.BOX)
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\structures.py", line 105, in grow_boxes
    image_width, image_height = self.image_size
  File "C:\Users\Harrou abderrahmane\AppData\Local\Programs\Python\Python310\lib\site-packages\pcleaner\structures.py", line 79, in image_size
    size_str = re.search(r"(\d+) x (\d+)", metadata).groups()
AttributeError: 'NoneType' object has no attribute 'groups'. Did you mean: '_image_size'?

i followed this : hey-red/Mime#9

@VoxelCubes
Copy link
Owner

Yeah, I haven't had the time to redo my windows vm and trying to get a build working there, so thanks for the tips on how I might be able to do that.

As for the denoiser, I personally have never run into that issue and wasn't able to reproduce it, but I can try catching whatever error that is and moving on, though I can't know for sure if it's fixed. Are you able to share one or more images with this problem (in a way that preserves the original file, so that it hopefully works the same)?

In the meantime, add the -D flag (or --skip-denoising) to your command or alternatively use a profile where denoising is explicitly disabled.

Thanks!

@VoxelCubes
Copy link
Owner

Unfortunately I still haven't been able to reproduce this issue, so I don't know to fix it. If you have images that cause issues, please email them to me at [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@VoxelCubes @justme7678876788