REST API missing file

[ctrl-z-9000-times]

Hi @dkeeney,

I tried making a new release, but I got the following error messages, which prevented the python packages from being built & uploaded to the test server. All of these test failures appear to have the same cause, a missing file for the REST API.

The missing file is: /project/build/Release/bin/rest_server

IIRC: when you build a python package to distribute it, it only includes python files (*.py) be default. Some where in the setup & install scripts there are options to include other files.

  =================================== FAILURES ===================================
  ___________________ HtmRestApiTest.testNetworkRESTBaseDelete ___________________
  
  self = <htm_rest_api_test.HtmRestApiTest testMethod=testNetworkRESTBaseDelete>
  
      def setUp(self):
  >     self._process = subprocess.Popen([REST_SERVER, '8050', '127.0.0.1'])
  
  /project/py/tests/rest/htm_rest_api_test.py:28: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:800: in __init__
      restore_signals, start_new_session)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <subprocess.Popen object at 0x7f48d00443d0>
  args = ['/project/build/Release/bin/rest_server', '8050', '127.0.0.1']
  executable = b'/project/build/Release/bin/rest_server', preexec_fn = None
  close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
  creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
  c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True
  start_new_session = False
  
      def _execute_child(self, args, executable, preexec_fn, close_fds,
                         pass_fds, cwd, env,
                         startupinfo, creationflags, shell,
                         p2cread, p2cwrite,
                         c2pread, c2pwrite,
                         errread, errwrite,
                         restore_signals, start_new_session):
          """Execute program (POSIX version)"""
      
          if isinstance(args, (str, bytes)):
              args = [args]
          else:
              args = list(args)
      
          if shell:
              # On Android the default shell is at '/system/bin/sh'.
              unix_shell = ('/system/bin/sh' if
                        hasattr(sys, 'getandroidapilevel') else '/bin/sh')
              args = [unix_shell, "-c"] + args
              if executable:
                  args[0] = executable
      
          if executable is None:
              executable = args[0]
          orig_executable = executable
      
          # For transferring possible exec failure from child to parent.
          # Data format: "exception name:hex errno:description"
          # Pickle is not used; it is complex and involves memory allocation.
          errpipe_read, errpipe_write = os.pipe()
          # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
          low_fds_to_close = []
          while errpipe_write < 3:
              low_fds_to_close.append(errpipe_write)
              errpipe_write = os.dup(errpipe_write)
          for low_fd in low_fds_to_close:
              os.close(low_fd)
          try:
              try:
                  # We must avoid complex work that could involve
                  # malloc or free in the child process to avoid
                  # potential deadlocks, thus we do all this here.
                  # and pass it to fork_exec()
      
                  if env is not None:
                      env_list = []
                      for k, v in env.items():
                          k = os.fsencode(k)
                          if b'=' in k:
                              raise ValueError("illegal environment variable name")
                          env_list.append(k + b'=' + os.fsencode(v))
                  else:
                      env_list = None  # Use execv instead of execve.
                  executable = os.fsencode(executable)
                  if os.path.dirname(executable):
                      executable_list = (executable,)
                  else:
                      # This matches the behavior of os._execvpe().
                      executable_list = tuple(
                          os.path.join(os.fsencode(dir), executable)
                          for dir in os.get_exec_path(env))
                  fds_to_keep = set(pass_fds)
                  fds_to_keep.add(errpipe_write)
                  self.pid = _posixsubprocess.fork_exec(
                          args, executable_list,
                          close_fds, tuple(sorted(map(int, fds_to_keep))),
                          cwd, env_list,
                          p2cread, p2cwrite, c2pread, c2pwrite,
                          errread, errwrite,
                          errpipe_read, errpipe_write,
                          restore_signals, start_new_session, preexec_fn)
                  self._child_created = True
              finally:
                  # be sure the FD is closed no matter what
                  os.close(errpipe_write)
      
              # self._devnull is not always defined.
              devnull_fd = getattr(self, '_devnull', None)
              if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                  os.close(p2cread)
              if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                  os.close(c2pwrite)
              if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                  os.close(errwrite)
              if devnull_fd is not None:
                  os.close(devnull_fd)
              # Prevent a double close of these fds from __init__ on error.
              self._closed_child_pipe_fds = True
      
              # Wait for exec to fail or succeed; possibly raising an
              # exception (limited in size)
              errpipe_data = bytearray()
              while True:
                  part = os.read(errpipe_read, 50000)
                  errpipe_data += part
                  if not part or len(errpipe_data) > 50000:
                      break
          finally:
              # be sure the FD is closed no matter what
              os.close(errpipe_read)
      
          if errpipe_data:
              try:
                  pid, sts = os.waitpid(self.pid, 0)
                  if pid == self.pid:
                      self._handle_exitstatus(sts)
                  else:
                      self.returncode = sys.maxsize
              except ChildProcessError:
                  pass
      
              try:
                  exception_name, hex_errno, err_msg = (
                          errpipe_data.split(b':', 2))
                  # The encoding here should match the encoding
                  # written in by the subprocess implementations
                  # like _posixsubprocess
                  err_msg = err_msg.decode()
              except ValueError:
                  exception_name = b'SubprocessError'
                  hex_errno = b'0'
                  err_msg = 'Bad exception data from child: {!r}'.format(
                                bytes(errpipe_data))
              child_exception_type = getattr(
                      builtins, exception_name.decode('ascii'),
                      SubprocessError)
              if issubclass(child_exception_type, OSError) and hex_errno:
                  errno_num = int(hex_errno, 16)
                  child_exec_never_called = (err_msg == "noexec")
                  if child_exec_never_called:
                      err_msg = ""
                      # The error must be from chdir(cwd).
                      err_filename = cwd
                  else:
                      err_filename = orig_executable
                  if errno_num != 0:
                      err_msg = os.strerror(errno_num)
                      if errno_num == errno.ENOENT:
                          err_msg += ': ' + repr(err_filename)
  >               raise child_exception_type(errno_num, err_msg, err_filename)
  E               FileNotFoundError: [Errno 2] No such file or directory: '/project/build/Release/bin/rest_server': '/project/build/Release/bin/rest_server'
  
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:1551: FileNotFoundError
  __________________ HtmRestApiTest.testNetworkRESTBaseExample ___________________
  
  self = <htm_rest_api_test.HtmRestApiTest testMethod=testNetworkRESTBaseExample>
  
      def setUp(self):
  >     self._process = subprocess.Popen([REST_SERVER, '8050', '127.0.0.1'])
  
  /project/py/tests/rest/htm_rest_api_test.py:28: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:800: in __init__
      restore_signals, start_new_session)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <subprocess.Popen object at 0x7f48cfdb7610>
  args = ['/project/build/Release/bin/rest_server', '8050', '127.0.0.1']
  executable = b'/project/build/Release/bin/rest_server', preexec_fn = None
  close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
  creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
  c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True
  start_new_session = False
  
      def _execute_child(self, args, executable, preexec_fn, close_fds,
                         pass_fds, cwd, env,
                         startupinfo, creationflags, shell,
                         p2cread, p2cwrite,
                         c2pread, c2pwrite,
                         errread, errwrite,
                         restore_signals, start_new_session):
          """Execute program (POSIX version)"""
      
          if isinstance(args, (str, bytes)):
              args = [args]
          else:
              args = list(args)
      
          if shell:
              # On Android the default shell is at '/system/bin/sh'.
              unix_shell = ('/system/bin/sh' if
                        hasattr(sys, 'getandroidapilevel') else '/bin/sh')
              args = [unix_shell, "-c"] + args
              if executable:
                  args[0] = executable
      
          if executable is None:
              executable = args[0]
          orig_executable = executable
      
          # For transferring possible exec failure from child to parent.
          # Data format: "exception name:hex errno:description"
          # Pickle is not used; it is complex and involves memory allocation.
          errpipe_read, errpipe_write = os.pipe()
          # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
          low_fds_to_close = []
          while errpipe_write < 3:
              low_fds_to_close.append(errpipe_write)
              errpipe_write = os.dup(errpipe_write)
          for low_fd in low_fds_to_close:
              os.close(low_fd)
          try:
              try:
                  # We must avoid complex work that could involve
                  # malloc or free in the child process to avoid
                  # potential deadlocks, thus we do all this here.
                  # and pass it to fork_exec()
      
                  if env is not None:
                      env_list = []
                      for k, v in env.items():
                          k = os.fsencode(k)
                          if b'=' in k:
                              raise ValueError("illegal environment variable name")
                          env_list.append(k + b'=' + os.fsencode(v))
                  else:
                      env_list = None  # Use execv instead of execve.
                  executable = os.fsencode(executable)
                  if os.path.dirname(executable):
                      executable_list = (executable,)
                  else:
                      # This matches the behavior of os._execvpe().
                      executable_list = tuple(
                          os.path.join(os.fsencode(dir), executable)
                          for dir in os.get_exec_path(env))
                  fds_to_keep = set(pass_fds)
                  fds_to_keep.add(errpipe_write)
                  self.pid = _posixsubprocess.fork_exec(
                          args, executable_list,
                          close_fds, tuple(sorted(map(int, fds_to_keep))),
                          cwd, env_list,
                          p2cread, p2cwrite, c2pread, c2pwrite,
                          errread, errwrite,
                          errpipe_read, errpipe_write,
                          restore_signals, start_new_session, preexec_fn)
                  self._child_created = True
              finally:
                  # be sure the FD is closed no matter what
                  os.close(errpipe_write)
      
              # self._devnull is not always defined.
              devnull_fd = getattr(self, '_devnull', None)
              if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                  os.close(p2cread)
              if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                  os.close(c2pwrite)
              if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                  os.close(errwrite)
              if devnull_fd is not None:
                  os.close(devnull_fd)
              # Prevent a double close of these fds from __init__ on error.
              self._closed_child_pipe_fds = True
      
              # Wait for exec to fail or succeed; possibly raising an
              # exception (limited in size)
              errpipe_data = bytearray()
              while True:
                  part = os.read(errpipe_read, 50000)
                  errpipe_data += part
                  if not part or len(errpipe_data) > 50000:
                      break
          finally:
              # be sure the FD is closed no matter what
              os.close(errpipe_read)
      
          if errpipe_data:
              try:
                  pid, sts = os.waitpid(self.pid, 0)
                  if pid == self.pid:
                      self._handle_exitstatus(sts)
                  else:
                      self.returncode = sys.maxsize
              except ChildProcessError:
                  pass
      
              try:
                  exception_name, hex_errno, err_msg = (
                          errpipe_data.split(b':', 2))
                  # The encoding here should match the encoding
                  # written in by the subprocess implementations
                  # like _posixsubprocess
                  err_msg = err_msg.decode()
              except ValueError:
                  exception_name = b'SubprocessError'
                  hex_errno = b'0'
                  err_msg = 'Bad exception data from child: {!r}'.format(
                                bytes(errpipe_data))
              child_exception_type = getattr(
                      builtins, exception_name.decode('ascii'),
                      SubprocessError)
              if issubclass(child_exception_type, OSError) and hex_errno:
                  errno_num = int(hex_errno, 16)
                  child_exec_never_called = (err_msg == "noexec")
                  if child_exec_never_called:
                      err_msg = ""
                      # The error must be from chdir(cwd).
                      err_filename = cwd
                  else:
                      err_filename = orig_executable
                  if errno_num != 0:
                      err_msg = os.strerror(errno_num)
                      if errno_num == errno.ENOENT:
                          err_msg += ': ' + repr(err_filename)
  >               raise child_exception_type(errno_num, err_msg, err_filename)
  E               FileNotFoundError: [Errno 2] No such file or directory: '/project/build/Release/bin/rest_server': '/project/build/Release/bin/rest_server'
  
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:1551: FileNotFoundError
  _____________________ HtmRestApiTest.testNetworkRESTDelete _____________________
  
  self = <htm_rest_api_test.HtmRestApiTest testMethod=testNetworkRESTDelete>
  
      def setUp(self):
  >     self._process = subprocess.Popen([REST_SERVER, '8050', '127.0.0.1'])
  
  /project/py/tests/rest/htm_rest_api_test.py:28: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:800: in __init__
      restore_signals, start_new_session)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <subprocess.Popen object at 0x7f48cfe83790>
  args = ['/project/build/Release/bin/rest_server', '8050', '127.0.0.1']
  executable = b'/project/build/Release/bin/rest_server', preexec_fn = None
  close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
  creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
  c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True
  start_new_session = False
  
      def _execute_child(self, args, executable, preexec_fn, close_fds,
                         pass_fds, cwd, env,
                         startupinfo, creationflags, shell,
                         p2cread, p2cwrite,
                         c2pread, c2pwrite,
                         errread, errwrite,
                         restore_signals, start_new_session):
          """Execute program (POSIX version)"""
      
          if isinstance(args, (str, bytes)):
              args = [args]
          else:
              args = list(args)
      
          if shell:
              # On Android the default shell is at '/system/bin/sh'.
              unix_shell = ('/system/bin/sh' if
                        hasattr(sys, 'getandroidapilevel') else '/bin/sh')
              args = [unix_shell, "-c"] + args
              if executable:
                  args[0] = executable
      
          if executable is None:
              executable = args[0]
          orig_executable = executable
      
          # For transferring possible exec failure from child to parent.
          # Data format: "exception name:hex errno:description"
          # Pickle is not used; it is complex and involves memory allocation.
          errpipe_read, errpipe_write = os.pipe()
          # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
          low_fds_to_close = []
          while errpipe_write < 3:
              low_fds_to_close.append(errpipe_write)
              errpipe_write = os.dup(errpipe_write)
          for low_fd in low_fds_to_close:
              os.close(low_fd)
          try:
              try:
                  # We must avoid complex work that could involve
                  # malloc or free in the child process to avoid
                  # potential deadlocks, thus we do all this here.
                  # and pass it to fork_exec()
      
                  if env is not None:
                      env_list = []
                      for k, v in env.items():
                          k = os.fsencode(k)
                          if b'=' in k:
                              raise ValueError("illegal environment variable name")
                          env_list.append(k + b'=' + os.fsencode(v))
                  else:
                      env_list = None  # Use execv instead of execve.
                  executable = os.fsencode(executable)
                  if os.path.dirname(executable):
                      executable_list = (executable,)
                  else:
                      # This matches the behavior of os._execvpe().
                      executable_list = tuple(
                          os.path.join(os.fsencode(dir), executable)
                          for dir in os.get_exec_path(env))
                  fds_to_keep = set(pass_fds)
                  fds_to_keep.add(errpipe_write)
                  self.pid = _posixsubprocess.fork_exec(
                          args, executable_list,
                          close_fds, tuple(sorted(map(int, fds_to_keep))),
                          cwd, env_list,
                          p2cread, p2cwrite, c2pread, c2pwrite,
                          errread, errwrite,
                          errpipe_read, errpipe_write,
                          restore_signals, start_new_session, preexec_fn)
                  self._child_created = True
              finally:
                  # be sure the FD is closed no matter what
                  os.close(errpipe_write)
      
              # self._devnull is not always defined.
              devnull_fd = getattr(self, '_devnull', None)
              if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                  os.close(p2cread)
              if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                  os.close(c2pwrite)
              if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                  os.close(errwrite)
              if devnull_fd is not None:
                  os.close(devnull_fd)
              # Prevent a double close of these fds from __init__ on error.
              self._closed_child_pipe_fds = True
      
              # Wait for exec to fail or succeed; possibly raising an
              # exception (limited in size)
              errpipe_data = bytearray()
              while True:
                  part = os.read(errpipe_read, 50000)
                  errpipe_data += part
                  if not part or len(errpipe_data) > 50000:
                      break
          finally:
              # be sure the FD is closed no matter what
              os.close(errpipe_read)
      
          if errpipe_data:
              try:
                  pid, sts = os.waitpid(self.pid, 0)
                  if pid == self.pid:
                      self._handle_exitstatus(sts)
                  else:
                      self.returncode = sys.maxsize
              except ChildProcessError:
                  pass
      
              try:
                  exception_name, hex_errno, err_msg = (
                          errpipe_data.split(b':', 2))
                  # The encoding here should match the encoding
                  # written in by the subprocess implementations
                  # like _posixsubprocess
                  err_msg = err_msg.decode()
              except ValueError:
                  exception_name = b'SubprocessError'
                  hex_errno = b'0'
                  err_msg = 'Bad exception data from child: {!r}'.format(
                                bytes(errpipe_data))
              child_exception_type = getattr(
                      builtins, exception_name.decode('ascii'),
                      SubprocessError)
              if issubclass(child_exception_type, OSError) and hex_errno:
                  errno_num = int(hex_errno, 16)
                  child_exec_never_called = (err_msg == "noexec")
                  if child_exec_never_called:
                      err_msg = ""
                      # The error must be from chdir(cwd).
                      err_filename = cwd
                  else:
                      err_filename = orig_executable
                  if errno_num != 0:
                      err_msg = os.strerror(errno_num)
                      if errno_num == errno.ENOENT:
                          err_msg += ': ' + repr(err_filename)
  >               raise child_exception_type(errno_num, err_msg, err_filename)
  E               FileNotFoundError: [Errno 2] No such file or directory: '/project/build/Release/bin/rest_server': '/project/build/Release/bin/rest_server'
  
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:1551: FileNotFoundError
  ____________________ HtmRestApiTest.testNetworkRESTExample _____________________
  
  self = <htm_rest_api_test.HtmRestApiTest testMethod=testNetworkRESTExample>
  
      def setUp(self):
  >     self._process = subprocess.Popen([REST_SERVER, '8050', '127.0.0.1'])
  
  /project/py/tests/rest/htm_rest_api_test.py:28: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:800: in __init__
      restore_signals, start_new_session)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <subprocess.Popen object at 0x7f48cfd45610>
  args = ['/project/build/Release/bin/rest_server', '8050', '127.0.0.1']
  executable = b'/project/build/Release/bin/rest_server', preexec_fn = None
  close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
  creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
  c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True
  start_new_session = False
  
      def _execute_child(self, args, executable, preexec_fn, close_fds,
                         pass_fds, cwd, env,
                         startupinfo, creationflags, shell,
                         p2cread, p2cwrite,
                         c2pread, c2pwrite,
                         errread, errwrite,
                         restore_signals, start_new_session):
          """Execute program (POSIX version)"""
      
          if isinstance(args, (str, bytes)):
              args = [args]
          else:
              args = list(args)
      
          if shell:
              # On Android the default shell is at '/system/bin/sh'.
              unix_shell = ('/system/bin/sh' if
                        hasattr(sys, 'getandroidapilevel') else '/bin/sh')
              args = [unix_shell, "-c"] + args
              if executable:
                  args[0] = executable
      
          if executable is None:
              executable = args[0]
          orig_executable = executable
      
          # For transferring possible exec failure from child to parent.
          # Data format: "exception name:hex errno:description"
          # Pickle is not used; it is complex and involves memory allocation.
          errpipe_read, errpipe_write = os.pipe()
          # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
          low_fds_to_close = []
          while errpipe_write < 3:
              low_fds_to_close.append(errpipe_write)
              errpipe_write = os.dup(errpipe_write)
          for low_fd in low_fds_to_close:
              os.close(low_fd)
          try:
              try:
                  # We must avoid complex work that could involve
                  # malloc or free in the child process to avoid
                  # potential deadlocks, thus we do all this here.
                  # and pass it to fork_exec()
      
                  if env is not None:
                      env_list = []
                      for k, v in env.items():
                          k = os.fsencode(k)
                          if b'=' in k:
                              raise ValueError("illegal environment variable name")
                          env_list.append(k + b'=' + os.fsencode(v))
                  else:
                      env_list = None  # Use execv instead of execve.
                  executable = os.fsencode(executable)
                  if os.path.dirname(executable):
                      executable_list = (executable,)
                  else:
                      # This matches the behavior of os._execvpe().
                      executable_list = tuple(
                          os.path.join(os.fsencode(dir), executable)
                          for dir in os.get_exec_path(env))
                  fds_to_keep = set(pass_fds)
                  fds_to_keep.add(errpipe_write)
                  self.pid = _posixsubprocess.fork_exec(
                          args, executable_list,
                          close_fds, tuple(sorted(map(int, fds_to_keep))),
                          cwd, env_list,
                          p2cread, p2cwrite, c2pread, c2pwrite,
                          errread, errwrite,
                          errpipe_read, errpipe_write,
                          restore_signals, start_new_session, preexec_fn)
                  self._child_created = True
              finally:
                  # be sure the FD is closed no matter what
                  os.close(errpipe_write)
      
              # self._devnull is not always defined.
              devnull_fd = getattr(self, '_devnull', None)
              if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                  os.close(p2cread)
              if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                  os.close(c2pwrite)
              if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                  os.close(errwrite)
              if devnull_fd is not None:
                  os.close(devnull_fd)
              # Prevent a double close of these fds from __init__ on error.
              self._closed_child_pipe_fds = True
      
              # Wait for exec to fail or succeed; possibly raising an
              # exception (limited in size)
              errpipe_data = bytearray()
              while True:
                  part = os.read(errpipe_read, 50000)
                  errpipe_data += part
                  if not part or len(errpipe_data) > 50000:
                      break
          finally:
              # be sure the FD is closed no matter what
              os.close(errpipe_read)
      
          if errpipe_data:
              try:
                  pid, sts = os.waitpid(self.pid, 0)
                  if pid == self.pid:
                      self._handle_exitstatus(sts)
                  else:
                      self.returncode = sys.maxsize
              except ChildProcessError:
                  pass
      
              try:
                  exception_name, hex_errno, err_msg = (
                          errpipe_data.split(b':', 2))
                  # The encoding here should match the encoding
                  # written in by the subprocess implementations
                  # like _posixsubprocess
                  err_msg = err_msg.decode()
              except ValueError:
                  exception_name = b'SubprocessError'
                  hex_errno = b'0'
                  err_msg = 'Bad exception data from child: {!r}'.format(
                                bytes(errpipe_data))
              child_exception_type = getattr(
                      builtins, exception_name.decode('ascii'),
                      SubprocessError)
              if issubclass(child_exception_type, OSError) and hex_errno:
                  errno_num = int(hex_errno, 16)
                  child_exec_never_called = (err_msg == "noexec")
                  if child_exec_never_called:
                      err_msg = ""
                      # The error must be from chdir(cwd).
                      err_filename = cwd
                  else:
                      err_filename = orig_executable
                  if errno_num != 0:
                      err_msg = os.strerror(errno_num)
                      if errno_num == errno.ENOENT:
                          err_msg += ': ' + repr(err_filename)
  >               raise child_exception_type(errno_num, err_msg, err_filename)
  E               FileNotFoundError: [Errno 2] No such file or directory: '/project/build/Release/bin/rest_server': '/project/build/Release/bin/rest_server'
  
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:1551: FileNotFoundError
  ___________________ HtmRestApiTest.testNetworkRESTHelloWorld ___________________
  
  self = <htm_rest_api_test.HtmRestApiTest testMethod=testNetworkRESTHelloWorld>
  
      def setUp(self):
  >     self._process = subprocess.Popen([REST_SERVER, '8050', '127.0.0.1'])
  
  /project/py/tests/rest/htm_rest_api_test.py:28: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:800: in __init__
      restore_signals, start_new_session)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <subprocess.Popen object at 0x7f48cff0ab50>
  args = ['/project/build/Release/bin/rest_server', '8050', '127.0.0.1']
  executable = b'/project/build/Release/bin/rest_server', preexec_fn = None
  close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
  creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
  c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True
  start_new_session = False
  
      def _execute_child(self, args, executable, preexec_fn, close_fds,
                         pass_fds, cwd, env,
                         startupinfo, creationflags, shell,
                         p2cread, p2cwrite,
                         c2pread, c2pwrite,
                         errread, errwrite,
                         restore_signals, start_new_session):
          """Execute program (POSIX version)"""
      
          if isinstance(args, (str, bytes)):
              args = [args]
          else:
              args = list(args)
      
          if shell:
              # On Android the default shell is at '/system/bin/sh'.
              unix_shell = ('/system/bin/sh' if
                        hasattr(sys, 'getandroidapilevel') else '/bin/sh')
              args = [unix_shell, "-c"] + args
              if executable:
                  args[0] = executable
      
          if executable is None:
              executable = args[0]
          orig_executable = executable
      
          # For transferring possible exec failure from child to parent.
          # Data format: "exception name:hex errno:description"
          # Pickle is not used; it is complex and involves memory allocation.
          errpipe_read, errpipe_write = os.pipe()
          # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
          low_fds_to_close = []
          while errpipe_write < 3:
              low_fds_to_close.append(errpipe_write)
              errpipe_write = os.dup(errpipe_write)
          for low_fd in low_fds_to_close:
              os.close(low_fd)
          try:
              try:
                  # We must avoid complex work that could involve
                  # malloc or free in the child process to avoid
                  # potential deadlocks, thus we do all this here.
                  # and pass it to fork_exec()
      
                  if env is not None:
                      env_list = []
                      for k, v in env.items():
                          k = os.fsencode(k)
                          if b'=' in k:
                              raise ValueError("illegal environment variable name")
                          env_list.append(k + b'=' + os.fsencode(v))
                  else:
                      env_list = None  # Use execv instead of execve.
                  executable = os.fsencode(executable)
                  if os.path.dirname(executable):
                      executable_list = (executable,)
                  else:
                      # This matches the behavior of os._execvpe().
                      executable_list = tuple(
                          os.path.join(os.fsencode(dir), executable)
                          for dir in os.get_exec_path(env))
                  fds_to_keep = set(pass_fds)
                  fds_to_keep.add(errpipe_write)
                  self.pid = _posixsubprocess.fork_exec(
                          args, executable_list,
                          close_fds, tuple(sorted(map(int, fds_to_keep))),
                          cwd, env_list,
                          p2cread, p2cwrite, c2pread, c2pwrite,
                          errread, errwrite,
                          errpipe_read, errpipe_write,
                          restore_signals, start_new_session, preexec_fn)
                  self._child_created = True
              finally:
                  # be sure the FD is closed no matter what
                  os.close(errpipe_write)
      
              # self._devnull is not always defined.
              devnull_fd = getattr(self, '_devnull', None)
              if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                  os.close(p2cread)
              if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                  os.close(c2pwrite)
              if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                  os.close(errwrite)
              if devnull_fd is not None:
                  os.close(devnull_fd)
              # Prevent a double close of these fds from __init__ on error.
              self._closed_child_pipe_fds = True
      
              # Wait for exec to fail or succeed; possibly raising an
              # exception (limited in size)
              errpipe_data = bytearray()
              while True:
                  part = os.read(errpipe_read, 50000)
                  errpipe_data += part
                  if not part or len(errpipe_data) > 50000:
                      break
          finally:
              # be sure the FD is closed no matter what
              os.close(errpipe_read)
      
          if errpipe_data:
              try:
                  pid, sts = os.waitpid(self.pid, 0)
                  if pid == self.pid:
                      self._handle_exitstatus(sts)
                  else:
                      self.returncode = sys.maxsize
              except ChildProcessError:
                  pass
      
              try:
                  exception_name, hex_errno, err_msg = (
                          errpipe_data.split(b':', 2))
                  # The encoding here should match the encoding
                  # written in by the subprocess implementations
                  # like _posixsubprocess
                  err_msg = err_msg.decode()
              except ValueError:
                  exception_name = b'SubprocessError'
                  hex_errno = b'0'
                  err_msg = 'Bad exception data from child: {!r}'.format(
                                bytes(errpipe_data))
              child_exception_type = getattr(
                      builtins, exception_name.decode('ascii'),
                      SubprocessError)
              if issubclass(child_exception_type, OSError) and hex_errno:
                  errno_num = int(hex_errno, 16)
                  child_exec_never_called = (err_msg == "noexec")
                  if child_exec_never_called:
                      err_msg = ""
                      # The error must be from chdir(cwd).
                      err_filename = cwd
                  else:
                      err_filename = orig_executable
                  if errno_num != 0:
                      err_msg = os.strerror(errno_num)
                      if errno_num == errno.ENOENT:
                          err_msg += ': ' + repr(err_filename)
  >               raise child_exception_type(errno_num, err_msg, err_filename)
  E               FileNotFoundError: [Errno 2] No such file or directory: '/project/build/Release/bin/rest_server': '/project/build/Release/bin/rest_server'
  
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:1551: FileNotFoundError
  _________________ HtmRestApiTest.testNetworkRESTSetInputScalar _________________
  
  self = <htm_rest_api_test.HtmRestApiTest testMethod=testNetworkRESTSetInputScalar>
  
      def setUp(self):
  >     self._process = subprocess.Popen([REST_SERVER, '8050', '127.0.0.1'])
  
  /project/py/tests/rest/htm_rest_api_test.py:28: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:800: in __init__
      restore_signals, start_new_session)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <subprocess.Popen object at 0x7f48cfd70090>
  args = ['/project/build/Release/bin/rest_server', '8050', '127.0.0.1']
  executable = b'/project/build/Release/bin/rest_server', preexec_fn = None
  close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
  creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
  c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True
  start_new_session = False
  
      def _execute_child(self, args, executable, preexec_fn, close_fds,
                         pass_fds, cwd, env,
                         startupinfo, creationflags, shell,
                         p2cread, p2cwrite,
                         c2pread, c2pwrite,
                         errread, errwrite,
                         restore_signals, start_new_session):
          """Execute program (POSIX version)"""
      
          if isinstance(args, (str, bytes)):
              args = [args]
          else:
              args = list(args)
      
          if shell:
              # On Android the default shell is at '/system/bin/sh'.
              unix_shell = ('/system/bin/sh' if
                        hasattr(sys, 'getandroidapilevel') else '/bin/sh')
              args = [unix_shell, "-c"] + args
              if executable:
                  args[0] = executable
      
          if executable is None:
              executable = args[0]
          orig_executable = executable
      
          # For transferring possible exec failure from child to parent.
          # Data format: "exception name:hex errno:description"
          # Pickle is not used; it is complex and involves memory allocation.
          errpipe_read, errpipe_write = os.pipe()
          # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
          low_fds_to_close = []
          while errpipe_write < 3:
              low_fds_to_close.append(errpipe_write)
              errpipe_write = os.dup(errpipe_write)
          for low_fd in low_fds_to_close:
              os.close(low_fd)
          try:
              try:
                  # We must avoid complex work that could involve
                  # malloc or free in the child process to avoid
                  # potential deadlocks, thus we do all this here.
                  # and pass it to fork_exec()
      
                  if env is not None:
                      env_list = []
                      for k, v in env.items():
                          k = os.fsencode(k)
                          if b'=' in k:
                              raise ValueError("illegal environment variable name")
                          env_list.append(k + b'=' + os.fsencode(v))
                  else:
                      env_list = None  # Use execv instead of execve.
                  executable = os.fsencode(executable)
                  if os.path.dirname(executable):
                      executable_list = (executable,)
                  else:
                      # This matches the behavior of os._execvpe().
                      executable_list = tuple(
                          os.path.join(os.fsencode(dir), executable)
                          for dir in os.get_exec_path(env))
                  fds_to_keep = set(pass_fds)
                  fds_to_keep.add(errpipe_write)
                  self.pid = _posixsubprocess.fork_exec(
                          args, executable_list,
                          close_fds, tuple(sorted(map(int, fds_to_keep))),
                          cwd, env_list,
                          p2cread, p2cwrite, c2pread, c2pwrite,
                          errread, errwrite,
                          errpipe_read, errpipe_write,
                          restore_signals, start_new_session, preexec_fn)
                  self._child_created = True
              finally:
                  # be sure the FD is closed no matter what
                  os.close(errpipe_write)
      
              # self._devnull is not always defined.
              devnull_fd = getattr(self, '_devnull', None)
              if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                  os.close(p2cread)
              if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                  os.close(c2pwrite)
              if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                  os.close(errwrite)
              if devnull_fd is not None:
                  os.close(devnull_fd)
              # Prevent a double close of these fds from __init__ on error.
              self._closed_child_pipe_fds = True
      
              # Wait for exec to fail or succeed; possibly raising an
              # exception (limited in size)
              errpipe_data = bytearray()
              while True:
                  part = os.read(errpipe_read, 50000)
                  errpipe_data += part
                  if not part or len(errpipe_data) > 50000:
                      break
          finally:
              # be sure the FD is closed no matter what
              os.close(errpipe_read)
      
          if errpipe_data:
              try:
                  pid, sts = os.waitpid(self.pid, 0)
                  if pid == self.pid:
                      self._handle_exitstatus(sts)
                  else:
                      self.returncode = sys.maxsize
              except ChildProcessError:
                  pass
      
              try:
                  exception_name, hex_errno, err_msg = (
                          errpipe_data.split(b':', 2))
                  # The encoding here should match the encoding
                  # written in by the subprocess implementations
                  # like _posixsubprocess
                  err_msg = err_msg.decode()
              except ValueError:
                  exception_name = b'SubprocessError'
                  hex_errno = b'0'
                  err_msg = 'Bad exception data from child: {!r}'.format(
                                bytes(errpipe_data))
              child_exception_type = getattr(
                      builtins, exception_name.decode('ascii'),
                      SubprocessError)
              if issubclass(child_exception_type, OSError) and hex_errno:
                  errno_num = int(hex_errno, 16)
                  child_exec_never_called = (err_msg == "noexec")
                  if child_exec_never_called:
                      err_msg = ""
                      # The error must be from chdir(cwd).
                      err_filename = cwd
                  else:
                      err_filename = orig_executable
                  if errno_num != 0:
                      err_msg = os.strerror(errno_num)
                      if errno_num == errno.ENOENT:
                          err_msg += ': ' + repr(err_filename)
  >               raise child_exception_type(errno_num, err_msg, err_filename)
  E               FileNotFoundError: [Errno 2] No such file or directory: '/project/build/Release/bin/rest_server': '/project/build/Release/bin/rest_server'
  
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:1551: FileNotFoundError
  __________________ HtmRestApiTest.testNetworkRESTSetInputSdr ___________________
  
  self = <htm_rest_api_test.HtmRestApiTest testMethod=testNetworkRESTSetInputSdr>
  
      def setUp(self):
  >     self._process = subprocess.Popen([REST_SERVER, '8050', '127.0.0.1'])
  
  /project/py/tests/rest/htm_rest_api_test.py:28: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:800: in __init__
      restore_signals, start_new_session)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <subprocess.Popen object at 0x7f48cfd3fa50>
  args = ['/project/build/Release/bin/rest_server', '8050', '127.0.0.1']
  executable = b'/project/build/Release/bin/rest_server', preexec_fn = None
  close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
  creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
  c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True
  start_new_session = False
  
      def _execute_child(self, args, executable, preexec_fn, close_fds,
                         pass_fds, cwd, env,
                         startupinfo, creationflags, shell,
                         p2cread, p2cwrite,
                         c2pread, c2pwrite,
                         errread, errwrite,
                         restore_signals, start_new_session):
          """Execute program (POSIX version)"""
      
          if isinstance(args, (str, bytes)):
              args = [args]
          else:
              args = list(args)
      
          if shell:
              # On Android the default shell is at '/system/bin/sh'.
              unix_shell = ('/system/bin/sh' if
                        hasattr(sys, 'getandroidapilevel') else '/bin/sh')
              args = [unix_shell, "-c"] + args
              if executable:
                  args[0] = executable
      
          if executable is None:
              executable = args[0]
          orig_executable = executable
      
          # For transferring possible exec failure from child to parent.
          # Data format: "exception name:hex errno:description"
          # Pickle is not used; it is complex and involves memory allocation.
          errpipe_read, errpipe_write = os.pipe()
          # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
          low_fds_to_close = []
          while errpipe_write < 3:
              low_fds_to_close.append(errpipe_write)
              errpipe_write = os.dup(errpipe_write)
          for low_fd in low_fds_to_close:
              os.close(low_fd)
          try:
              try:
                  # We must avoid complex work that could involve
                  # malloc or free in the child process to avoid
                  # potential deadlocks, thus we do all this here.
                  # and pass it to fork_exec()
      
                  if env is not None:
                      env_list = []
                      for k, v in env.items():
                          k = os.fsencode(k)
                          if b'=' in k:
                              raise ValueError("illegal environment variable name")
                          env_list.append(k + b'=' + os.fsencode(v))
                  else:
                      env_list = None  # Use execv instead of execve.
                  executable = os.fsencode(executable)
                  if os.path.dirname(executable):
                      executable_list = (executable,)
                  else:
                      # This matches the behavior of os._execvpe().
                      executable_list = tuple(
                          os.path.join(os.fsencode(dir), executable)
                          for dir in os.get_exec_path(env))
                  fds_to_keep = set(pass_fds)
                  fds_to_keep.add(errpipe_write)
                  self.pid = _posixsubprocess.fork_exec(
                          args, executable_list,
                          close_fds, tuple(sorted(map(int, fds_to_keep))),
                          cwd, env_list,
                          p2cread, p2cwrite, c2pread, c2pwrite,
                          errread, errwrite,
                          errpipe_read, errpipe_write,
                          restore_signals, start_new_session, preexec_fn)
                  self._child_created = True
              finally:
                  # be sure the FD is closed no matter what
                  os.close(errpipe_write)
      
              # self._devnull is not always defined.
              devnull_fd = getattr(self, '_devnull', None)
              if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
                  os.close(p2cread)
              if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
                  os.close(c2pwrite)
              if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
                  os.close(errwrite)
              if devnull_fd is not None:
                  os.close(devnull_fd)
              # Prevent a double close of these fds from __init__ on error.
              self._closed_child_pipe_fds = True
      
              # Wait for exec to fail or succeed; possibly raising an
              # exception (limited in size)
              errpipe_data = bytearray()
              while True:
                  part = os.read(errpipe_read, 50000)
                  errpipe_data += part
                  if not part or len(errpipe_data) > 50000:
                      break
          finally:
              # be sure the FD is closed no matter what
              os.close(errpipe_read)
      
          if errpipe_data:
              try:
                  pid, sts = os.waitpid(self.pid, 0)
                  if pid == self.pid:
                      self._handle_exitstatus(sts)
                  else:
                      self.returncode = sys.maxsize
              except ChildProcessError:
                  pass
      
              try:
                  exception_name, hex_errno, err_msg = (
                          errpipe_data.split(b':', 2))
                  # The encoding here should match the encoding
                  # written in by the subprocess implementations
                  # like _posixsubprocess
                  err_msg = err_msg.decode()
              except ValueError:
                  exception_name = b'SubprocessError'
                  hex_errno = b'0'
                  err_msg = 'Bad exception data from child: {!r}'.format(
                                bytes(errpipe_data))
              child_exception_type = getattr(
                      builtins, exception_name.decode('ascii'),
                      SubprocessError)
              if issubclass(child_exception_type, OSError) and hex_errno:
                  errno_num = int(hex_errno, 16)
                  child_exec_never_called = (err_msg == "noexec")
                  if child_exec_never_called:
                      err_msg = ""
                      # The error must be from chdir(cwd).
                      err_filename = cwd
                  else:
                      err_filename = orig_executable
                  if errno_num != 0:
                      err_msg = os.strerror(errno_num)
                      if errno_num == errno.ENOENT:
                          err_msg += ': ' + repr(err_filename)
  >               raise child_exception_type(errno_num, err_msg, err_filename)
  E               FileNotFoundError: [Errno 2] No such file or directory: '/project/build/Release/bin/rest_server': '/project/build/Release/bin/rest_server'
  
  /opt/python/cp37-cp37m/lib/python3.7/subprocess.py:1551: FileNotFoundError
  =============================== warnings summary ===============================
  ../project/bindings/py/tests/sparse_link_test.py:28
    /project/bindings/py/tests/sparse_link_test.py:28: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
    Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
      TEST_DATA_DENSE = np.zeros(OUTPUT_WIDTH, dtype=np.bool)
  
  -- Docs: https://docs.pytest.org/en/stable/warnings.html
  Error: Command ['sh', '-c', 'pytest /project/py/tests /project/bindings/py/tests'] failed with code 1. 
  --------------- generated xml file: /root/junit-test-results.xml ---------------
  
  =========================== short test summary info ============================
  FAILED ../project/py/tests/rest/htm_rest_api_test.py::HtmRestApiTest::testNetworkRESTBaseDelete
  FAILED ../project/py/tests/rest/htm_rest_api_test.py::HtmRestApiTest::testNetworkRESTBaseExample
  FAILED ../project/py/tests/rest/htm_rest_api_test.py::HtmRestApiTest::testNetworkRESTDelete
  FAILED ../project/py/tests/rest/htm_rest_api_test.py::HtmRestApiTest::testNetworkRESTExample
  FAILED ../project/py/tests/rest/htm_rest_api_test.py::HtmRestApiTest::testNetworkRESTHelloWorld
  FAILED ../project/py/tests/rest/htm_rest_api_test.py::HtmRestApiTest::testNetworkRESTSetInputScalar
  FAILED ../project/py/tests/rest/htm_rest_api_test.py::HtmRestApiTest::testNetworkRESTSetInputSdr
  ============ 7 failed, 360 passed, 12 skipped, 1 warning in 55.57s =============
                                                             ✕ 69.15s
Error: Process completed with exit code 1.

[dkeeney]

The rest_server is a separate executable program. It is something that python can make a LAN connection to but not call directly. This server is written in C++ and must be compiled for each platform.

For unit tests the python code will start up the server, connect to it, exchange some messages, disconnect, and then shut down the server.

So, is there a way to include an executable program in the pypi package?

[ctrl-z-9000-times]

Okay, thanks for explaining what that is.

So, is there a way to include an executable program in the pypi package?

Yes, I've done it before but I don't remember how...
It should just be a few lines added to the "setup.py" script?
I will look into it when I get a chance.

[ctrl-z-9000-times]

MEMO: https://docs.python.org/3.8/distutils/setupscript.html#installing-package-data