taskInfo bytecode string incompatibility in py3

[eberrippe]

Hi @timurhai ,

We have updated our renders/servers to run with Python 3.10 instead of Python 2.7. Since this update, we've encountered an issue in the taskInfo within events/services/parsers where certain strings are being parsed as byte strings (e.g., b"hello world") instead of standard strings. This discrepancy is causing several issues in the system.

Could you modify these entries upstream to ensure they are correctly interpreted as regular strings?

Here are examples of entries that are currently being parsed as byte objects:

{
   "wdir":"/tmp",
   "parser":"b""nuke",
   "command_block":"b""nuke -F @#@-@#@/1 -V 2.nk\"",
   "command_task":"b""",
   "capacity":4,
   "files_block":[
      "b""/.@####@.exr"
   ],
   "files_task":[
      
   ],
   "file_size_min":-1,
   "file_size_max":-1,
   "hosts":[
      
   ],
   "parsed_files":[
      
   ],
   "environment":{
      "MAINC000",
      "MAI"scenes",
      "PROJECT":"abcd"
   },
   "tickets":{
      "dfa":1,
      "iod":1,
      "df":5,
      "nauke":1,
      "renyferder_usage":5,
      "fddt":4
   },
   "frame_start":1009,
   "frame_finish":1013,
   "frame_inc":1,
   "frames_num":5,
   "task_id":0,
   "task_name":"b""frame 1009-1013",
   "task_custom_data":"b"",
   "block_id":0,
   "block_name":"b""07dfadfeoer",
   "block_flags":33,
   "block_custom_data":"b""",
   "job_id":84,
   "job_name":"b""071aeerrlihoh",
   "job_flags":3471202581171300206,
   "job_custom_data":"b""{\"aere"}",
   "user_name":"b""dafd",
   "user_flags":7431513519383457836,
   "user_custom_data":"b""",
   "render_flags":8245929725132764528,
   "render_custom_data":"b""",
   "store_dir":"b""/vadfeeawre_1",
   "command":"nuke -F 1009-1013.nk\"",
   "files":[
      "/madfanticSeg-car/071adfadfadf9.exr",
   ]
}

Thanks a lot!

Jan

[timurhai]

Hello! Sorry, I did not got were, why and what you are parsing?
Most others (and me) are using Python 3 for many years (>10).
And there is not problems with Python3 and switching from 2 to 3.
(some types of vfx software switched from 2 to 3 recently)

[eberrippe]

Hi @timurhai,
thanks for your response.

Looking at:

cgru/afanasy/python/services/events.py

Line 104 in 91c93d4

Just imagine, we would add a print(self.taskInfo["user_name"])

This would result in b"timurhai"

We ecounter the same issue in the parser.

cgru/afanasy/python/parsers/parser.py

Line 64 in 91c93d4

Also if we would add a print(self.taskInfo["job_name"])

b"myTestJob"

We have this byte String interpretation since we use a python3x build with our renders.

Or is it only us?

Thanks
Jan

[lithorus]

1. How was the job submitted?
2. Which version of CGRU?
3. Which OS?

[eberrippe]

We are still at CentOS linux 7 but will jump to Alamalinux
We use AF version 3.3.0
It was submitted via our submitter using afanasy/python/af.py

Does it mean you dont have the same thing happening to you @lithorus ? Maybe it could be build related.

[sebastianelsner]

Let me rewind and rephrase what @eberrippe originally posted:

Before we compiled Afanasy 3.3.0 with Python 2.7 on CentOS 7. We had "normal" strings in the self.taskInfo in events.py

{
   "wdir":"/tmp",
   "parser":"nuke",
   "command_block":"nuke -F @#@-@#@/1 -V 2.nk\",
   "command_task":"",
   "capacity":4,
   ...

Now we started to compile afanasy 3.3.0 with Python 3.10 on CentOS 7. And when printing the self.taskInfo in events.py we get weird byte strings like:

{
   "wdir":"/tmp",
   "parser":"b""nuke",
   "command_block":"b""nuke -F @#@-@#@/1 -V 2.nk\"",
   "command_task":"b""",
   "capacity":4,
   ...

What Python versions are you guys using? I suppose you are not seeing this?

[lithorus]

I was more interested in seeing some actual sample code. Is is submitted through within Nuke (and which version)?

We submit from within the DCC's with our own tools, which means it's a wide range of python versions. Also af.py is pure python and doesn't depend on any specific version.

Try to describe exactly which steps the submission go through (and with code examples). To me it looks like you're generating something in python, but does the actual submission through a command line using eg. os.system(). That might explain the double quoting.

[lithorus]

Here is a very basic example you can try :

import af

job = af.Job("test job")
job.setProject("project")
job.setDepartment("department")
block = af.Block("test block")
block.setCommand("echo dummy job")
block.setNumeric(1, 1, 1)
job.blocks.append(block)
print(job.data)
job.send()

This will also print what is sent to the server.

[lithorus]

Also, compiling it with python3 doesn't make it more compatible with DCCs running python3. It's only python3 in afrender's parsers and services, etc.