Can't use WD14 tagger in sd3-flux.1 branch

[zadokov]

I have tried several combinations, but while I can train Lora on this branch, I can't create captions.
I need to activate a separate 'master' branch process for tagging.

I get the below when trying to create captions on sd3-flux.1 branch.

Any idea why? Is that going to be fixed once it is merged to the master branch?

14:26:15-400371 INFO Captioning files in F:\SD\workspace\images\training_images\img_AVK - AVKFLX - flx3...
14:26:15-402391 INFO Executing command: C:\SD\kohya_ss_flux.1\venv\Scripts\accelerate.EXE launch C:/SD/kohya_ss_flux.1/sd-scripts/finetune/tag_images_by_wd14_tagger.py --batch_size 1 --caption_extension .txt --caption_separator ,
--character_threshold 0.2 --debug --frequency_tags --general_threshold 0.25 --max_data_loader_n_workers 2 --onnx --remove_underscore --repo_id SmilingWolf/wd-convnext-tagger-v3 --thresh 0.25
F:\SD\workspace\images\training_images\img_AVK - AVKFLX - flx3
The following values were not passed to accelerate launch and had defaults used instead:
--num_processes was set to a value of 1
--num_machines was set to a value of 1
--mixed_precision was set to a value of 'no'
--dynamo_backend was set to a value of 'no'
To avoid this warning pass in values for each of the problematic parameters or run accelerate config.
C:\SD\kohya_ss_flux.1\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
C:\SD\kohya_ss_flux.1\venv\lib\site-packages\diffusers\utils\outputs.py:63: FutureWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
torch.utils._pytree._register_pytree_node(
2024-10-28 14:26:22 INFO using existing wd14 tagger model tag_images_by_wd14_tagger.py:113
INFO Running wd14 tagger with onnx tag_images_by_wd14_tagger.py:121
INFO loading onnx model: wd14_tagger_model\SmilingWolf_wd-convnext-tagger-v3/model.onnx tag_images_by_wd14_tagger.py:122
2024-10-28 14:26:22.3626443 [E:onnxruntime:Default, provider_bridge_ort.cc:1548 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

*************** EP Error ***************
EP Error D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.
when using ['CUDAExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.

---

2024-10-28 14:26:22.4872174 [E:onnxruntime:Default, provider_bridge_ort.cc:1548 onnxruntime::TryGetProviderInfo_CUDA] D:\a_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"

Traceback (most recent call last):
File "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\SD\kohya_ss_flux.1\sd-scripts\finetune\tag_images_by_wd14_tagger.py", line 515, in
main(args)
File "C:\SD\kohya_ss_flux.1\sd-scripts\finetune\tag_images_by_wd14_tagger.py", line 155, in main
ort_sess = ort.InferenceSession(
File "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 432, in init
raise fallback_error from e
File "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 427, in init
self._create_inference_session(self._fallback_providers, None)
File "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:857 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

Traceback (most recent call last):
File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\SD\kohya_ss_flux.1\venv\Scripts\accelerate.EXE_main.py", line 7, in
File "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\accelerate\commands\launch.py", line 1106, in launch_command
simple_launcher(args)
File "C:\SD\kohya_ss_flux.1\venv\lib\site-packages\accelerate\commands\launch.py", line 704, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['C:\SD\kohya_ss_flux.1\venv\Scripts\python.exe', 'C:/SD/kohya_ss_flux.1/sd-scripts/finetune/tag_images_by_wd14_tagger.py', '--batch_size', '1', '--caption_extension', '.txt', '--caption_separator', ', ', '--character_threshold', '0.2', '--debug', '--frequency_tags', '--general_threshold', '0.25', '--max_data_loader_n_workers', '2', '--onnx', '--remove_underscore', '--repo_id', 'SmilingWolf/wd-convnext-tagger-v3', '--thresh', '0.25', 'F:\SD\workspace\images\training_images\img_AVK - AVKFLX - flx3']' returned non-zero exit status 1.
14:26:23-442294 INFO ...captioning done

[b-fission]

The sd3-flux.1 branch is using cuda12 because of pytorch, but also keeps an older version of onnx that expects cuda11. On my machine, WD14 and onnx were able to use CPU as a fallback instead of failing completely.

Try upgrading onnx to the latest version which runs on cuda12. Open the file called requirements_windows.txt and change the line that says onnxruntime-gpu==1.17.1 to onnxruntime-gpu==1.19.2

[zadokov]

The sd3-flux.1 branch is using cuda12 because of pytorch, but also keeps an older version of onnx that expects cuda11. On my machine, WD14 and onnx were able to use CPU as a fallback instead of failing completely.

Try upgrading onnx to the latest version which runs on cuda12. Open the file called requirements_windows.txt and change the line that says onnxruntime-gpu==1.17.1 to onnxruntime-gpu==1.19.2

Worked!!!! Thanks!
@bmaltais/@b-fission - Please update the requirements files accordingly

[bmaltais]

Updated the version in the latest commit. Should hopefully fix the issue.