Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lxc_ssh uses local unix username instead of ansible_user when connecting to host #37

Open
boucman opened this issue Jun 9, 2021 · 106 comments

Comments

@boucman
Copy link

boucman commented Jun 9, 2021

Hello everybody
this is an upstream version of debops/debops#1731

The problem was reproduced with raw "ansible" command so we are pretty sure this is not a debops problem.

I will copy/paste the other bug report here, but some discussion have already taken place on the other side.

Hello
I am in the process of upgrading from 2.2.1 to 2.2.2

Most of my machines are LXC containers which I reach through lxc_ssh

  • my username on my control machine is jerros
  • the host machine is called monstro (monstro.daviel.idf.intranet)
  • the container is called build-yocto

When I attempt any playbook on build-yocto, I get a permisssion denied.

Interesting extract of my trace below:

jerros@logrus:~/comite_expertise/adminsys-ecs$ debops -l build-yocto service/apt -vvv
Running Ansible playbooks:
/home/jerros/comite_expertise/adminsys-ecs/debops/ansible/playbooks/service/apt.yml
(...)

PLAY [Manage Advanced Package Manager] *****************************************************************************************************************************************************************************
XXX connect
<monstro.daviel> ESTABLISH SSH CONNECTION FOR USER: root
<monstro.daviel> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jerros/.ansible/cp/f16751a621 -tt monstro.daviel 'which lxc'
<monstro.daviel> (255, b'', b'[email protected]: Permission denied (publickey).\r\n')
<monstro.daviel> ssh_retry: attempt: 0, ssh return code is 255. cmd ([b'ssh', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/jerros/.ansible/cp/f16751a621', b'-tt', b'monstro.daviel', b'which lxc']...), pausing for 0 seconds
<monstro.daviel> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jerros/.ansible/cp/f16751a621 -tt monstro.daviel 'which lxc'
<monstro.daviel> (255, b'', b'[email protected]: Permission denied (publickey).\r\n')
<monstro.daviel> ssh_retry: attempt: 1, ssh return code is 255. cmd ([b'ssh', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/jerros/.ansible/cp/f16751a621', b'-tt', b'monstro.daviel', b'which lxc']...), pausing for 1 seconds
<monstro.daviel> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jerros/.ansible/cp/f16751a621 -tt monstro.daviel 'which lxc'
<monstro.daviel> (255, b'', b'[email protected]: Permission denied (publickey).\r\n')
<monstro.daviel> ssh_retry: attempt: 2, ssh return code is 255. cmd ([b'ssh', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/jerros/.ansible/cp/f16751a621', b'-tt', b'monstro.daviel', b'which lxc']...), pausing for 3 seconds
<monstro.daviel> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/jerros/.ansible/cp/f16751a621 -tt monstro.daviel 'which lxc'
<monstro.daviel> (255, b'', b'[email protected]: Permission denied (publickey).\r\n')

TASK [Gathering Facts] *********************************************************************************************************************************************************************************************
task path: /home/jerros/comite_expertise/adminsys-ecs/debops/ansible/playbooks/service/apt.yml:7
fatal: [build-yocto]: FAILED! => 
  msg: |-
    Failed to connect to the host via ssh: [email protected]: Permission denied (publickey).

We see that the proble is that lxc_ssh attempts to connect to the host (monstro) using my local unix username (jerros) instead of using root (I can connect directly as root on that machine, I tested separately)

This used to work in 2.2.1 and reverting to debops 2.2.1 solves the problem

Also note the line <monstro.daviel> ESTABLISH SSH CONNECTION FOR USER: root so debops thinks it is attempting to connect as root, but seems to be connecting without a username, so using the default, local, username.

as a complement, a cutdown output of ansible-inventory for build-yocto

{
    "ansible_connection": "lxc_ssh",
    "ansible_host": "monstro.daviel",
    "ansible_ssh_extra_args": "{{inventory_hostname}}",
    "ansible_user": "root",
    "netbase__domain": "daviel.idf.intranet",
    "netbase__host_ipv4_address": "127.0.1.1",
}

Thx a lot
Jérémy

@boucman
Copy link
Author

boucman commented Jun 9, 2021

After digging a little, this seems caused by

self.user = self.get_option("remote_user")

Which resets the previous (correct) value of self.user.
this line was added in #31
and contradicts
self.user = self._play_context.remote_user

Commenting that line fixes the problem for me, but I don't have enough understanding of how ansible works to know what is the correct fix.

Hopefully someone with more knowledge can tell me what's going on here

@andreasscherbaum
Copy link
Owner

@dmp1ce @stefangweichinger The PR #31 came from you. Can you have a look if this is an oversight, or breaks something? Thanks!

@stefangweichinger
Copy link

#31 came from @dmp1ce ... my PRs came after that and hopefully did only formatting changes ;-)

@dmp1ce
Copy link

dmp1ce commented Jun 9, 2021

@boucman Are you sure we are getting the full picture from your setup? When I change ansible_ssh_user in my inventory, the playbook is run correctly with the user expected. Here is a copy of my inventory for example:

[docker_host]
poweredge

[lxd_host]
poweredge

[lxd_guest]
omd2 lxc_host=omd ansible_host=192.168.1.111 ansible_connection=lxc_ssh ansible_ssh_user=dave

[miners]
MP-145 ansible_host=10.0.0.163
MP-146 ansible_host=10.0.0.177

root user is default I believe so maybe the user jerros is being set somewhere else in the playbook variables.

Either way, a test should be created to verify the plugin is working properly, IMHO.

@dmp1ce
Copy link

dmp1ce commented Jun 9, 2021

From my understanding line 517 gets the remote_user information from a number of different places. Make sure none of them are being used in your setup.

self.user = self._play_context.remote_user

Here is the documentation which mirrors the code in ssh.py from the official Ansible connection plugin library.

ansible-lxc-ssh/lxc_ssh.py

Lines 254 to 269 in d421553

remote_user:
description:
- User name with which to login to the remote server, normally
set by the remote_user keyword.
- If no user is supplied, Ansible will let the ssh client binary
choose the user as it normally
ini:
- section: defaults
key: remote_user
env:
- name: ANSIBLE_REMOTE_USER
vars:
- name: ansible_user
- name: ansible_ssh_user
cli:
- name: user

@andreasscherbaum
Copy link
Owner

If this is something which can be replicated and the plugin is at fault, I'm happy to create an additional test.
If that is coming from another variable, there is not much which can be done about it.

@AnvithLobo
Copy link

Can confirm this issue.
inventory:

type3:
      ansible_host: IP
      ansible_port: 22
      ansible_ssh_user: 'root'
      remote_user: root
      ansible_user: 'root'
      ansible_ssh_private_key_file : keys/playbook_keys
      lxc_host: 'playbook2'
      ansible_connection: lxc_ssh

i tried every possible combination from where the user variable can be taken.

but it still tries to connect using the current linux username

(venv) testuser@local:~/data/ansiblelxd/Python/ansible$ ansible-playbook -i inventory.yml playbook.yml -vv
ansible-playbook [core 2.11.1]
  config file = /home/testuser/data/ansiblelxd/Python/ansible/config.cfg
  configured module search path = ['/home/testuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/testuser/venv/lib/python3.9/site-packages/ansible
  ansible collection location = /home/testuser/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/testuser/venv/bin/ansible-playbook
  python version = 3.9.5 (default, May 11 2021, 08:20:37) [GCC 10.3.0]
  jinja version = 3.0.1
  libyaml = True
Using /home/testuser/data/ansiblelxd/Python/ansible/config.cfg as config file
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: playbook.yml ***********************************************************************************************************************************************
1 plays in playbook.yml

PLAY [Ping] **********************************************************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************************************************
task path: /home/testuser/data/ansiblelxd/Python/ansible/playbook.yml:2
<IP> ssh_retry: attempt: 0, ssh return code is 255. cmd ([b'ssh', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/61f24b0e12', b'-tt', b'IP', b'which lxc']...), pausing for 0 seconds
<IP> ssh_retry: attempt: 1, ssh return code is 255. cmd ([b'ssh', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/61f24b0e12', b'-tt', b'IP', b'which lxc']...), pausing for 1 seconds
<IP> ssh_retry: attempt: 2, ssh return code is 255. cmd ([b'ssh', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/61f24b0e12', b'-tt', b'IP', b'which lxc']...), pausing for 3 seconds
fatal: [type3]: FAILED! => {"msg": "Failed to connect to the host via ssh: testuser@IP: Permission denied (publickey,password).\r\n"}

PLAY RECAP ***********************************************************************************************************************************************************
type3                      : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

(venv) testuser@local:~/data/ansiblelxd/Python/ansible$

@dmp1ce
Copy link

dmp1ce commented Jun 9, 2021

I'm just guessing here but is ANSIBLE_REMOTE_USER environment variable set? I'm not sure if that take precedence over the inventory file.

Maybe try again with -vvvv, although I'm not sure if all that extra debug data will be useful.

What is the remote_user settings? If it is left blank then it uses the user executing the Ansible command by default. https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-remote-user

@AnvithLobo
Copy link

AnvithLobo commented Jun 9, 2021

No environment variables are set since most of my playbooks are called from ansible_runner python api.

I'm fairly new to ansible to there might be some mistakes here (sorry about that)

here's my inventory file

---
all:
  hosts: 
    type3:
      ansible_host: IP.IP.IP.IP
      ansible_port: 22
      ansible_ssh_user: root
      remote_user: root
      ansible_user: root
      ansible_ssh_private_key_file : /home/testuser/playbook_keys
      lxc_host: 'playbook2'
      ansible_connection: lxc_ssh

    type2:
      ansible_host: IP.IP.IP.IP
      ansible_port: 22
      ansible_ssh_user: root
      remote_user: root
      ansible_user: root
      ansible_ssh_private_key_file : /home/testuser/playbook_keys
      ansible_connection: ssh

#37 (comment)

commenting the line like @boucman said does sorta fix the problem at hand.

but now i get

"msg": "Failed to connect to the host via ssh: root@IP: Permission denied (publickey,password).

looks like this is due to it not using the key mentioned in ansible_ssh_private_key_file variable unlike the ansible.builtin.ssh connector




running the same inventory by commenting lxc_host: 'playbook2' and changing ansible_connection to ssh i can login perfectly without issues.

here's the full debug log for ansible-playbook run

(venv) testuser@local:~/data/ansiblelxd/Python/ansible$ ansible-playbook -i inventory.yml playbook.yml -vvvv
ansible-playbook [core 2.11.1]
  config file = /home/testuser/data/ansiblelxd/Python/ansible/config.cfg
  configured module search path = ['/home/testuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/testuser/venv/lib/python3.9/site-packages/ansible
  ansible collection location = /home/testuser/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/testuser/venv/bin/ansible-playbook
  python version = 3.9.5 (default, May 11 2021, 08:20:37) [GCC 10.3.0]
  jinja version = 3.0.1
  libyaml = True
Using /home/testuser/data/ansiblelxd/Python/ansible/config.cfg as config file
setting up inventory plugins
host_list declined parsing /home/testuser/data/ansiblelxd/Python/ansible/inventory.yml as it did not pass its verify_file() method
Parsed /home/testuser/data/ansiblelxd/Python/ansible/inventory.yml inventory source with yaml plugin
Loading callback plugin default of type stdout, v2.0 from /home/testuser/venv/lib/python3.9/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: playbook.yml ***********************************************************************************************************************************************Positional arguments: playbook.yml
verbosity: 4
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/home/testuser/data/ansiblelxd/Python/ansible/inventory.yml',)
forks: 5
1 plays in playbook.yml

PLAY [Ping] **********************************************************************************************************************************************************
TASK [Gathering Facts] ***********************************************************************************************************************************************task path: /home/testuser/data/ansiblelxd/Python/ansible/playbook.yml:2
XXX connect
<IP.IP.IP.IP> ESTABLISH SSH CONNECTION FOR USER: root
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/61f24b0e12 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (255, b'', b'OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/home/testuser/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/home/testuser/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/testuser/.ansible/cp/61f24b0e12" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9748 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as \'testuser\'\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host \'IP.IP.IP.IP\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n')
<IP.IP.IP.IP> ssh_retry: attempt: 0, ssh return code is 255. cmd ([b'ssh', b'-vvv', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/61f24b0e12', b'-tt', b'IP.IP.IP.IP', b'which lxc']...), pausing for 0 seconds
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/61f24b0e12 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (255, b'', b'OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/home/testuser/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/home/testuser/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/testuser/.ansible/cp/61f24b0e12" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9742 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as \'testuser\'\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host \'IP.IP.IP.IP\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n')
<IP.IP.IP.IP> ssh_retry: attempt: 1, ssh return code is 255. cmd ([b'ssh', b'-vvv', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/61f24b0e12', b'-tt', b'IP.IP.IP.IP', b'which lxc']...), pausing for 1 seconds
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/61f24b0e12 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (255, b'', b'OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/home/testuser/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/home/testuser/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/testuser/.ansible/cp/61f24b0e12" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9736 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as \'testuser\'\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host \'IP.IP.IP.IP\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n')
<IP.IP.IP.IP> ssh_retry: attempt: 2, ssh return code is 255. cmd ([b'ssh', b'-vvv', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/61f24b0e12', b'-tt', b'IP.IP.IP.IP', b'which lxc']...), pausing for 3 seconds
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/61f24b0e12 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (255, b'', b'OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/home/testuser/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/home/testuser/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/testuser/.ansible/cp/61f24b0e12" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9746 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as \'testuser\'\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host \'IP.IP.IP.IP\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n')
fatal: [type3]: FAILED! => {
    "msg": "Failed to connect to the host via ssh: OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/testuser/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/testuser/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket \"/home/testuser/.ansible/cp/61f24b0e12\" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9746 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as 'testuser'\r\ndebug3: hostkeys_foreach: reading file \"/home/testuser/.ssh/known_hosts\"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file \"/home/testuser/.ssh/known_hosts\"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host 'IP.IP.IP.IP' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n"








Logs after commenting line self.user = self.get_option("remote_user")

(venv) testuser@local:~/data/ansiblelxd/Python/ansible$ ansible-playbook -i inventory.yml playbook.yml -vvvv
ansible-playbook [core 2.11.1]
  config file = /home/testuser/data/ansiblelxd/Python/ansible/config.cfg
  configured module search path = ['/home/testuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/testuser/venv/lib/python3.9/site-packages/ansible
  ansible collection location = /home/testuser/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/testuser/venv/bin/ansible-playbook
  python version = 3.9.5 (default, May 11 2021, 08:20:37) [GCC 10.3.0]
  jinja version = 3.0.1
  libyaml = True
Using /home/testuser/data/ansiblelxd/Python/ansible/config.cfg as config file
setting up inventory plugins
host_list declined parsing /home/testuser/data/ansiblelxd/Python/ansible/inventory.yml as it did not pass its verify_file() method
Parsed /home/testuser/data/ansiblelxd/Python/ansible/inventory.yml inventory source with yaml plugin
Loading callback plugin default of type stdout, v2.0 from /home/testuser/venv/lib/python3.9/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: playbook.yml ***********************************************************************************************************************************
Positional arguments: playbook.yml
verbosity: 4
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/home/testuser/data/ansiblelxd/Python/ansible/inventory.yml',)
forks: 5
1 plays in playbook.yml

PLAY [Ping] **********************************************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************************************
task path: /home/testuser/data/ansiblelxd/Python/ansible/playbook.yml:2
XXX connect
<IP.IP.IP.IP> ESTABLISH SSH CONNECTION FOR USER: root
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (255, b'', b'OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/home/testuser/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/home/testuser/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/testuser/.ansible/cp/96d0c51d80" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9745 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as \'root\'\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host \'IP.IP.IP.IP\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n')
<IP.IP.IP.IP> ssh_retry: attempt: 0, ssh return code is 255. cmd ([b'ssh', b'-vvv', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'User="root"', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/96d0c51d80', b'-tt', b'IP.IP.IP.IP', b'which lxc']...), pausing for 0 seconds
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (255, b'', b'OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/home/testuser/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/home/testuser/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/testuser/.ansible/cp/96d0c51d80" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9744 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as \'root\'\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host \'IP.IP.IP.IP\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n')
<IP.IP.IP.IP> ssh_retry: attempt: 1, ssh return code is 255. cmd ([b'ssh', b'-vvv', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'User="root"', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/96d0c51d80', b'-tt', b'IP.IP.IP.IP', b'which lxc']...), pausing for 1 seconds
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (255, b'', b'OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/home/testuser/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/home/testuser/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/testuser/.ansible/cp/96d0c51d80" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9737 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as \'root\'\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host \'IP.IP.IP.IP\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n')
<IP.IP.IP.IP> ssh_retry: attempt: 2, ssh return code is 255. cmd ([b'ssh', b'-vvv', b'-C', b'-o', b'ControlMaster=auto', b'-o', b'ControlPersist=60s', b'-o', b'StrictHostKeyChecking=no', b'-o', b'KbdInteractiveAuthentication=no', b'-o', b'PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased', b'-o', b'PasswordAuthentication=no', b'-o', b'User="root"', b'-o', b'ConnectTimeout=10', b'-o', b'ControlPath=/home/testuser/.ansible/cp/96d0c51d80', b'-tt', b'IP.IP.IP.IP', b'which lxc']...), pausing for 3 seconds
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (255, b'', b'OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/home/testuser/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/home/testuser/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/home/testuser/.ansible/cp/96d0c51d80" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9744 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as \'root\'\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file "/home/testuser/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host \'IP.IP.IP.IP\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n')
fatal: [type3]: FAILED! => {
    "msg": "Failed to connect to the host via ssh: OpenSSH_8.4p1 Ubuntu-5ubuntu1, OpenSSL 1.1.1j  16 Feb 2021\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/testuser/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/testuser/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket \"/home/testuser/.ansible/cp/96d0c51d80\" does not exist\r\ndebug2: ssh_connect_direct\r\ndebug1: Connecting to IP.IP.IP.IP [IP.IP.IP.IP] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9744 ms remain after connect\r\ndebug1: identity file /home/testuser/.ssh/id_rsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_rsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_dsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ecdsa_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519 type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk type -1\r\ndebug1: identity file /home/testuser/.ssh/id_ed25519_sk-cert type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss type -1\r\ndebug1: identity file /home/testuser/.ssh/id_xmss-cert type -1\r\ndebug1: Local version string SSH-2.0-OpenSSH_8.4p1 Ubuntu-5ubuntu1\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.2\r\ndebug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.2 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to IP.IP.IP.IP:22 as 'root'\r\ndebug3: hostkeys_foreach: reading file \"/home/testuser/.ssh/known_hosts\"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug3: order_hostkeyalgs: have matching best-preference key type [email protected], using HostkeyAlgorithms verbatim\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c\r\ndebug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],ssh-ed25519,[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: [email protected],zlib,none\r\ndebug2: compression stoc: [email protected],zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256\r\ndebug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]\r\ndebug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,[email protected]\r\ndebug2: compression stoc: none,[email protected]\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: [email protected]\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:mEdxLC7nX8V8Hvss1Nikd2zGuG07cLAjYZZ6rAAgVz0\r\ndebug3: hostkeys_foreach: reading file \"/home/testuser/.ssh/known_hosts\"\r\ndebug3: record_hostkey: found key type ECDSA in file /home/testuser/.ssh/known_hosts:1\r\ndebug3: load_hostkeys: loaded 1 keys from IP.IP.IP.IP\r\ndebug1: Host 'IP.IP.IP.IP' is known and matches the ECDSA host key.\r\ndebug1: Found key in /home/testuser/.ssh/known_hosts:1\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey out after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey in after 134217728 blocks\r\ndebug1: Will attempt key: /home/testuser/.ssh/id_rsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_dsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ecdsa_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519 \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_ed25519_sk \r\ndebug1: Will attempt key: /home/testuser/.ssh/id_xmss \r\ndebug2: pubkey_prepare: done\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected]>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,password\r\ndebug3: start over, passed a different list publickey,password\r\ndebug3: preferred publickey,gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: gssapi-with-mic,gssapi-keyex,hostbased\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Trying private key: /home/testuser/.ssh/id_rsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_rsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_dsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_dsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ecdsa_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ecdsa_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_ed25519_sk\r\ndebug3: no such identity: /home/testuser/.ssh/id_ed25519_sk: No such file or directory\r\ndebug1: Trying private key: /home/testuser/.ssh/id_xmss\r\ndebug3: no such identity: /home/testuser/.ssh/id_xmss: No such file or directory\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\[email protected]: Permission denied (publickey,password).\r\n"
}

PLAY RECAP ***********************************************************************************************************************************************
type3                      : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

@dmp1ce
Copy link

dmp1ce commented Jun 9, 2021

Do you have any vars.yml or host_vars? I'm assuming inventory vars take precedence anyways though.

@AnvithLobo
Copy link

Do you have any vars.yml or host_vars? I'm assuming inventory vars take precedence anyways though.

No just the ones i've written above. Also its a brand new test environment with latest ansible installed. Updated the above comment to add more logs and observations.

@dmp1ce
Copy link

dmp1ce commented Jun 10, 2021

There is probably an issue with the plugin but I cannot find it because I cannot reproduce the issue.

Do you have steps to reproduce? Should I setup DebOps and try to reproduce that way?

@AnvithLobo
Copy link

AnvithLobo commented Jun 10, 2021

There is probably an issue with the plugin but I cannot find it because I cannot reproduce the issue.

Do you have steps to reproduce? Should I setup DebOps and try to reproduce that way?

No a simple ansible playbook that pings the container using the latest ansible should do it

here's the playbook

---
- name: Ping
  hosts: type2

  tasks:
  - name: Ping Server 
    ping:

It should not be related to DebOps as i'm not using it.

using this playbook with the inventory above with command ansible-playbook -i inventory.yml playbook.yml should reproduce the issue.

here's some more info

lxc_ssh : latest 
ansible : latest (4.0.0) / 2.9.22
os : ubuntu 20.04 / ubuntu 21.04 / kali 2021.1
lxd : latest using ubuntu repo / snap on kali 
python 3.9.5

@stefangweichinger
Copy link

I tested your small ping-playbook with my test setup and it does NOT fail.

In my case the LXC-server (the variable lxc_host is a bit misleading, btw) is a Debian 11 VM, in there I have some containers also running Debian 11.

I have to add that all my hosts in this setup already have been bootstrapped with DebOps: that installs python stuff, sudo rights, ssh-keys etc etc. Without that bootstrap step the container doesn't have python and is not reachable via ansible.

Also that additional container pings OK after installing python into it.

So I also couldn't yet reproduce the issue.

@AnvithLobo mentions lxd, I use lxc, might that be the relevant difference?

@AnvithLobo
Copy link

AnvithLobo commented Jun 10, 2021

@AnvithLobo mentions lxd, I use lxc, might that be the relevant difference?

it fails even before it reaches lxc part

Also tried it with py 3.6 same issues can someone provide me their inventory file so i can compare it with mine ?

also it it looks like the remote user is being fetched from config.cfg when set to root / ubuntu for remote_user it picks up the username from there. but when its commented out it defaults to local linux username ignoring the variables set in inventory.

@stefangweichinger
Copy link

@AnvithLobo my inventory is in debops/debops#1731 (comment) (the related issue in the debops repo)

@AnvithLobo
Copy link

@AnvithLobo my inventory is in debops/debops#1731 (comment) (the related issue in the debops repo)

yeah but i don't see a remote_user / ansible_user variable in your inventory. So i'm guessing that your ansible local system has the same username as remote system running LXC or you've set the remote_user in ansible.cfg file.

The problem with me is username on all my LXC servers are different same for the SSH keys too. Similar to OP where his local username is different than remote username

The ansible.builtin.ssh accepts ansible_user and ansible_ssh_private_key_file to solve this.

@AnvithLobo
Copy link

more debugging adding both

remote_user = root
private_key_file = /home/testuser/data/ansiblelxd/Python/ansible/keys/playbook_keys

in ansible.cfg solves the problem at hand.

So its definitely ignoring the vars set in inventory.yml

@stefangweichinger
Copy link

As my project dir is a debops-project there's a .debops.cfg that is interpreted by the debops wrapper and generates the local ansible.cfg. In both files there is no user defined. As far as I understand this means my local non-root-user sgw logs into the target machine as root@container. Right?

Interesting that "None" is shown here:

<app1.my.tld> ESTABLISH SSH CONNECTION FOR USER: None
<app1.my.tld> SSH: EXEC ssh -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/sgw/.ansible/cp/e88dbb5549 -tt app1.my.tld 'which lxc'
<app1.my.tld> (1, b'', b'Shared connection to 172.32.99.110 closed.\r\n')

Adding remote_user = root also works here, as you mentioned right now.

@AnvithLobo
Copy link

yeah it tries to use the local username / config.cfg username and not the one set in inventory to connect to the remote server.

if you do not have remote_user and private_key_file in config.cfg it uses defaults from ansible ie your local username and default ssh key without considering the vars set in inventory file unlike the default ssh module

@stefangweichinger
Copy link

Adding

# ansible/inventory/group_vars/all/inventory.yml
ansible_user: 'breakit'

leads to the playbook failing. So it's read here.

@AnvithLobo
Copy link

AnvithLobo commented Jun 10, 2021

what command are you using to run the playbook?
ansible-playbook -i path/to/inventory.yml path/to/playbook.yml ?

also whats the error
if its saying

FAILED! => {"msg": "Failed to connect to the host via ssh: breakit@IP: Permission denied

then yes its being read if not then its probably not.

@stefangweichinger
Copy link

I use both debops and ansible.
If I edit .debops.cfg I run debops, that generates ansible.cfg.

Both read the default inventory in ansible/inventory/hosts.

ansible-playbook playbook.yml
debops playbook.yml

debops-docs tell me: https://docs.debops.org/en/stable-2.2/introduction/getting-started.html#ansible-user

I googled the difference between ansible_user and remote_user:
https://stackoverflow.com/questions/36668756/ansible-remote-user-vs-ansible-user

Besides, ansible_user is used when we want to specifiy default SSH user in ansible hosts file where as remote_user is used in playbook context.

@stefangweichinger
Copy link

if its saying

FAILED! => {"msg": "Failed to connect to the host via ssh: breakit@IP: Permission denied

then yes its being read if not then its probably not.

Yes, that's the message.

@AnvithLobo
Copy link

AnvithLobo commented Jun 10, 2021

does your ansible.cfg (or generated config.cfg from debops) contain the the var ansible_user / remote_user when you ran this ?

@stefangweichinger
Copy link

does your ansible.cfg contain the the var ansible_user / remote_user when you ran this ?

No.

@AnvithLobo
Copy link

could you run it without debops with just ansible-playbook and see if it works ?

I tried several different machines with several different version of python and ansible without setting these in config.cfg i can't get it to work while the ansible.builtin.ssh does work

@stefangweichinger
Copy link

I did that above already. And it also fails with "[email protected]: Permission denied (publickey)."

@AnvithLobo
Copy link

here's a log with remote_user and private_key_file set in config.cfg works without any issues

❯ ansible-playbook -i inventory.yml playbook.yml -vvvv
ansible-playbook [core 2.11.1]
  config file = /home/testuser/data/ansiblelxd/Python/ansible/config.cfg
  configured module search path = ['/home/testuser/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/testuser/venv/lib/python3.8/site-packages/ansible
  ansible collection location = /home/testuser/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/testuser/venv/bin/ansible-playbook
  python version = 3.8.5 (default, May 27 2021, 13:30:53) [GCC 9.3.0]
  jinja version = 3.0.1
  libyaml = True
Using /home/testuser/data/ansiblelxd/Python/ansible/config.cfg as config file
setting up inventory plugins
host_list declined parsing /home/testuser/data/ansiblelxd/Python/ansible/inventory.yml as it did not pass its verify_file() method
Parsed /home/testuser/data/ansiblelxd/Python/ansible/inventory.yml inventory source with yaml plugin
Loading callback plugin default of type stdout, v2.0 from /home/testuser/venv/lib/python3.8/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: playbook.yml ***********************************************************************************************************************************Positional arguments: playbook.yml
verbosity: 4
private_key_file: /home/testuser/playbook_keys
remote_user: root
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/home/testuser/data/ansiblelxd/Python/ansible/inventory.yml',)
forks: 2
1 plays in playbook.yml

PLAY [Ping] **********************************************************************************************************************************************
TASK [Gathering Facts] ***********************************************************************************************************************************task path: /home/testuser/data/ansiblelxd/Python/ansible/playbook.yml:2
XXX connect
<IP.IP.IP.IP> ESTABLISH SSH CONNECTION FOR USER: root
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (0, b'/usr/bin/lxc\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX connect
<IP.IP.IP.IP> ESTABLISH SSH CONNECTION FOR USER: root
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'which lxc-info'
<IP.IP.IP.IP> (1, b'', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to IP.IP.IP.IP closed.\r\n')
LXC v2
XXX exec_command: /bin/sh -c 'echo ~root && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'echo ~root && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'/root\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX exec_command: /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727 `" && echo ansible-tmp-1623313473.107566-15069-6205902310727="` echo /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727 `" ) && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727 `" && echo ansible-tmp-1623313473.107566-15069-6205902310727="` echo /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727 `" ) && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'ansible-tmp-1623313473.107566-15069-6205902310727=/root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
<type3> Attempting python interpreter discovery
XXX exec_command: /bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'/usr/bin/python'"'"'; command -v '"'"'python3.9'"'"'; command -v '"'"'python3.8'"'"'; command -v '"'"'python3.7'"'"'; command -v '"'"'python3.6'"'"'; command -v '"'"'python3.5'"'"'; command -v '"'"'python2.7'"'"'; command -v '"'"'python2.6'"'"'; command -v '"'"'/usr/libexec/platform-python'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'python'"'"'; echo ENDFOUND && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'echo PLATFORM; uname; echo FOUND; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'/usr/bin/python'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'python3.9'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'python3.8'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'python3.7'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'python3.6'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'python3.5'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'python2.7'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'python2.6'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'/usr/libexec/platform-python'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'/usr/bin/python3'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'python'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'; echo ENDFOUND && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'PLATFORM\r\nLinux\r\nFOUND\r\n/usr/bin/python3.8\r\n/usr/bin/python3\r\nENDFOUND\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
<type3> Python interpreter discovery fallback (pipelining support required for extended interpreter discovery)
Using module file /home/testuser/venv/lib/python3.8/site-packages/ansible/modules/setup.py
XXX connect
<IP.IP.IP.IP> PUT /home/testuser/.ansible/tmp/ansible-local-150637aeax772/tmpyhe_ji5r TO /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/AnsiballZ_setup.py
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 IP.IP.IP.IP 'lxc exec playbook2
                                                                                                                                                         2 --mode=non-interactive -- /bin/sh -c '"'"'cat > /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/AnsiballZ_setup.py; echo -n done'"'"''
<IP.IP.IP.IP> (0, b'done', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
XXX exec_command: /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/ /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/AnsiballZ_setup.py && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/ /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/AnsiballZ_setup.py && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX exec_command: /bin/sh -c '/usr/bin/python3.8 /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/AnsiballZ_setup.py && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'/usr/bin/python3.8 /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/AnsiballZ_setup.py && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'\r\n{"ansible_facts": {"ansible_selinux_python_present": true, "ansible_selinux": {"status": "disabled"}, "ansible_system": "Linux", "ansible_kernel": "5.4.0-73-generic", "ansible_kernel_version": "#82-Ubuntu SMP Wed Apr 14 17:39:42 UTC 2021", "ansible_machine": "x86_64", "ansible_python_version": "3.8.5", "ansible_fqdn": "playbook2.lxd", "ansible_hostname": "playbook2", "ansible_nodename": "playbook2", "ansible_domain": "lxd", "ansible_userspace_bits": "64", "ansible_architecture": "x86_64", "ansible_userspace_architecture": "x86_64", "ansible_machine_id": "adac233692a0476796bb6c4755a2f222", "ansible_is_chroot": false, "ansible_fips": false, "ansible_ssh_host_key_dsa_public": "AAAAB3NzaC1kc3MAAACBAJf8WJ+iuzW8bThcydlegkATGoTwtOmJ34s+cUPNTYQzLKfAf71Ag/f+dLTMm30HBzKv0nU5IYdJyMm9iNDmnzTwMZUWj4n4jRCjeahgTZZ6ZwEhFv3oHkuRQSXRHPEe+ZAW5r1bnJ0eAHjEohjKTsp0yKni9qi/dZuYXZVbd+eRAAAAFQCmGnhnU2IJ5ZNYDDk8eVlhoXaR/QAAAIBqW7iE5a/lbyITjMatiwtMGcnANhca1mOgDJ8olVhDXPCF4utkhUKn7idCZ2u+VLQlG9lZklDb3OCC8eR5Hw4DX9+/eAaGNmhavmvFE5Jb1z+NpNFBE3jB377UtARkyy7iQCi31XNzrXJI1yBP6IxskqHuPbgJ0n5UJeE/MhWEYQAAAIEAlwDAS80bnzhDGRN6LB4NdBrqO9AQb49rNk8ktNI5U3oh/HEfjurJ1HoVlqvmSfzRIF900/5C3GbyVHLLsMJjKA/xEgNAyQ4OznR1cov5/fIVu4IVuzOfUAo08GM8iujr9QsWBBxU7hCI1HilqOoQzOb5o+BNLDYEH5Sts36rffU=", "ansible_ssh_host_key_dsa_public_keytype": "ssh-dss", "ansible_ssh_host_key_rsa_public": "AAAAB3NzaC1yc2EAAAADAQABAAABgQDOyYfTG9OJ/0gbvcJF0QwbkQ5j1MFm4wZmPUZOS6W/5cXPSU5f136FakfmUmQkNyOSCbQlubg4lOZHzNzks13lZuiL7R+d8cpkRn2YDaSPBjxIbqyD7HRWnYL8Z7bJp2VKgsu30ol5+TlZP6OyZ9gqncEPzqCJx6QQiicGjbt8+8pd4aw1ZUttF6NqkVV41HL34vtaiNioaWslTYpT5Ut8klWtZ4Vt9BHKdYYhVQv0kKUKHSyaSQ+C6Zu/bX0c/c3/J7KxuS+B09PCN5AXt5aD2ww2jbC5VuxBgYdWBqkm4kDNLsz3DPUXZ9cE/An4QzZrieGwgOWsfT1HobYLZmxNeuhERFVCSdGhDFxXRQ9cHYas2YtEegSaKL8LBJ73OoVCDhAmNda/KnyRhbyK7pQh36hUYOhBud9hh+Zcx0TuMi7kfcK3pV5l1TrHnU5hcqHAQm2hXdJUcHE2dOYq7RTulsOn3N4bPnmthTC8hM5GqmXkqUjXg49NyyQIV4Wy+Us=", "ansible_ssh_host_key_rsa_public_keytype": "ssh-rsa", "ansible_ssh_host_key_ecdsa_public": "AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdg0AFrB+fBblTUbTlj9cx9pRgRQ8A2S464UADyYrssyMhdNY44Quzl6OfynM82vqGPzUFIi2RNCDbf1ABwOc4=", "ansible_ssh_host_key_ecdsa_public_keytype": "ecdsa-sha2-nistp256", "ansible_ssh_host_key_ed25519_public": "AAAAC3NzaC1lZDI1NTE5AAAAINLABJkqcm3ffL2LpY4jAokpSmRSp/ZNNBhqy5OWLhdQ", "ansible_ssh_host_key_ed25519_public_keytype": "ssh-ed25519", "ansible_user_id": "root", "ansible_user_uid": 0, "ansible_user_gid": 0, "ansible_user_gecos": "root", "ansible_user_dir": "/root", "ansible_user_shell": "/bin/bash", "ansible_real_user_id": 0, "ansible_effective_user_id": 0, "ansible_real_group_id": 0, "ansible_effective_group_id": 0, "ansible_distribution": "Ubuntu", "ansible_distribution_release": "focal", "ansible_distribution_version": "20.04", "ansible_distribution_major_version": "20", "ansible_distribution_file_path": "/etc/os-release", "ansible_distribution_file_variety": "Debian", "ansible_distribution_file_parsed": true, "ansible_os_family": "Debian", "ansible_virtualization_type": "lxc", "ansible_virtualization_role": "guest", "ansible_virtualization_tech_guest": ["lxc", "container", "kvm"], "ansible_virtualization_tech_host": [], "ansible_lsb": {"id": "Ubuntu", "description": "Ubuntu 20.04.2 LTS", "release": "20.04", "codename": "focal", "major_release": "20"}, "ansible_hostnqn": "", "ansible_system_capabilities_enforced": "False", "ansible_system_capabilities": [], "ansible_iscsi_iqn": "", "ansible_cmdline": {"BOOT_IMAGE": "/boot/vmlinuz-5.4.0-73-generic", "root": "UUID=fd515699-109d-421d-866a-0142999e20a4", "ro": true}, "ansible_proc_cmdline": {"BOOT_IMAGE": "/boot/vmlinuz-5.4.0-73-generic", "root": "UUID=fd515699-109d-421d-866a-0142999e20a4", "ro": true}, "ansible_local": {}, "ansible_service_mgr": "systemd", "ansible_processor": ["0", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "1", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "2", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "3", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "4", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "5", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "6", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "7", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "8", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "9", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "10", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz", "11", "GenuineIntel", "Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz"], "ansible_processor_count": 12, "ansible_processor_cores": 1, "ansible_processor_threads_per_core": 1, "ansible_processor_vcpus": 12, "ansible_processor_nproc": 12, "ansible_memtotal_mb": 48195, "ansible_memfree_mb": 47943, "ansible_swaptotal_mb": 0, "ansible_swapfree_mb": 0, "ansible_memory_mb": {"real": {"total": 48195, "used": 252, "free": 47943}, "nocache": {"free": 48041, "used": 154}, "swap": {"total": 0, "free": 0, "used": 0, "cached": 0}}, "ansible_bios_date": "04/01/2014", "ansible_bios_vendor": "SeaBIOS", "ansible_bios_version": "1.13.0-2.module_el8.3.0+555+a55c8938", "ansible_board_asset_tag": "NA", "ansible_board_name": "RHEL-AV", "ansible_board_serial": "NA", "ansible_board_vendor": "Red Hat", "ansible_board_version": "RHEL-8.2.0 PC (Q35 + ICH9, 2009)", "ansible_chassis_asset_tag": "NA", "ansible_chassis_serial": "NA", "ansible_chassis_vendor": "Red Hat", "ansible_chassis_version": "RHEL-8.2.0 PC (Q35 + ICH9, 2009)", "ansible_form_factor": "Other", "ansible_product_name": "KVM", "ansible_product_serial": "NA", "ansible_product_uuid": "NA", "ansible_product_version": "RHEL-8.2.0 PC (Q35 + ICH9, 2009)", "ansible_system_vendor": "Red Hat", "ansible_devices": {"loop1": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": null, "model": null, "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "113504", "sectorsize": "512", "size": "55.42 MB", "host": "", "holders": []}, "loop6": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": null, "model": null, "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "65744", "sectorsize": "512", "size": "32.10 MB", "host": "", "holders": []}, "loop4": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": null, "model": null, "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "144136", "sectorsize": "512", "size": "70.38 MB", "host": "", "holders": []}, "loop2": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": null, "model": null, "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "138376", "sectorsize": "512", "size": "67.57 MB", "host": "", "holders": []}, "loop0": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": null, "model": null, "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "113560", "sectorsize": "512", "size": "55.45 MB", "host": "", "holders": []}, "loop7": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": null, "model": null, "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "host": "", "holders": []}, "sda": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": "QEMU", "model": "QEMU HARDDISK", "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {"sda2": {"links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "start": "4096", "sectors": "1509942912", "sectorsize": 512, "size": "720.00 GB", "uuid": null, "holders": []}, "sda1": {"links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "start": "2048", "sectors": "2048", "sectorsize": 512, "size": "1.00 MB", "uuid": null, "holders": []}}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "1509949440", "sectorsize": "512", "size": "720.00 GB", "host": "SCSI storage controller: Red Hat, Inc. Virtio SCSI (rev 01)", "holders": []}, "loop5": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": null, "model": null, "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "65712", "sectorsize": "512", "size": "32.09 MB", "host": "", "holders": []}, "loop3": {"virtual": 1, "links": {"ids": [], "uuids": [], "labels": [], "masters": []}, "vendor": null, "model": null, "sas_address": null, "sas_device_handle": null, "removable": "0", "support_discard": "4096", "partitions": {}, "rotational": "1", "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "host": "", "holders": []}}, "ansible_device_links": {"ids": {}, "uuids": {}, "labels": {}, "masters": {}}, "ansible_uptime_seconds": 820298, "ansible_lvm": {"lvs": {}, "vgs": {}, "pvs": {}}, "ansible_mounts": [], "ansible_dns": {"nameservers": ["127.0.0.53"], "options": {"edns0": true, "trust-ad": true}, "search": ["lxd"]}, "ansible_pkg_mgr": "apt", "ansible_fibre_channel_wwn": [], "ansible_interfaces": ["eth0", "lo"], "ansible_lo": {"device": "lo", "mtu": 65536, "active": true, "type": "loopback", "promisc": false, "ipv4": {"address": "127.0.0.1", "broadcast": "", "netmask": "255.0.0.0", "network": "127.0.0.0"}, "ipv6": [{"address": "::1", "prefix": "128", "scope": "host"}], "features": {"rx_checksumming": "on [fixed]", "tx_checksumming": "on", "tx_checksum_ipv4": "off [fixed]", "tx_checksum_ip_generic": "on [fixed]", "tx_checksum_ipv6": "off [fixed]", "tx_checksum_fcoe_crc": "off [fixed]", "tx_checksum_sctp": "on [fixed]", "scatter_gather": "on", "tx_scatter_gather": "on [fixed]", "tx_scatter_gather_fraglist": "on [fixed]", "tcp_segmentation_offload": "on", "tx_tcp_segmentation": "on", "tx_tcp_ecn_segmentation": "on", "tx_tcp_mangleid_segmentation": "on", "tx_tcp6_segmentation": "on", "generic_segmentation_offload": "on", "generic_receive_offload": "on", "large_receive_offload": "off [fixed]", "rx_vlan_offload": "off [fixed]", "tx_vlan_offload": "off [fixed]", "ntuple_filters": "off [fixed]", "receive_hashing": "off [fixed]", "highdma": "on [fixed]", "rx_vlan_filter": "off [fixed]", "vlan_challenged": "on [fixed]", "tx_lockless": "on [fixed]", "netns_local": "on [fixed]", "tx_gso_robust": "off [fixed]", "tx_fcoe_segmentation": "off [fixed]", "tx_gre_segmentation": "off [fixed]", "tx_gre_csum_segmentation": "off [fixed]", "tx_ipxip4_segmentation": "off [fixed]", "tx_ipxip6_segmentation": "off [fixed]", "tx_udp_tnl_segmentation": "off [fixed]", "tx_udp_tnl_csum_segmentation": "off [fixed]", "tx_gso_partial": "off [fixed]", "tx_sctp_segmentation": "on", "tx_esp_segmentation": "off [fixed]", "tx_udp_segmentation": "off [fixed]", "fcoe_mtu": "off [fixed]", "tx_nocache_copy": "off [fixed]", "loopback": "on [fixed]", "rx_fcs": "off [fixed]", "rx_all": "off [fixed]", "tx_vlan_stag_hw_insert": "off [fixed]", "rx_vlan_stag_hw_parse": "off [fixed]", "rx_vlan_stag_filter": "off [fixed]", "l2_fwd_offload": "off [fixed]", "hw_tc_offload": "off [fixed]", "esp_hw_offload": "off [fixed]", "esp_tx_csum_hw_offload": "off [fixed]", "rx_udp_tunnel_port_offload": "off [fixed]", "tls_hw_tx_offload": "off [fixed]", "tls_hw_rx_offload": "off [fixed]", "rx_gro_hw": "off [fixed]", "tls_hw_record": "off [fixed]"}, "timestamping": ["tx_software", "rx_software", "software"], "hw_timestamp_filters": []}, "ansible_eth0": {"device": "eth0", "macaddress": "00:16:3e:5e:f7:32", "mtu": 1500, "active": true, "type": "ether", "speed": 10000, "promisc": false, "ipv4": {"address": "10.77.25.134", "broadcast": "10.77.25.255", "netmask": "255.255.255.0", "network": "10.77.25.0"}, "ipv6": [{"address": "fd42:40f4:bbc1:54e9:216:3eff:fe5e:f732", "prefix": "64", "scope": "global"}, {"address": "fe80::216:3eff:fe5e:f732", "prefix": "64", "scope": "link"}], "features": {"rx_checksumming": "on", "tx_checksumming": "on", "tx_checksum_ipv4": "off [fixed]", "tx_checksum_ip_generic": "on", "tx_checksum_ipv6": "off [fixed]", "tx_checksum_fcoe_crc": "off [fixed]", "tx_checksum_sctp": "on", "scatter_gather": "on", "tx_scatter_gather": "on", "tx_scatter_gather_fraglist": "on", "tcp_segmentation_offload": "on", "tx_tcp_segmentation": "on", "tx_tcp_ecn_segmentation": "on", "tx_tcp_mangleid_segmentation": "on", "tx_tcp6_segmentation": "on", "generic_segmentation_offload": "on", "generic_receive_offload": "on", "large_receive_offload": "off [fixed]", "rx_vlan_offload": "on", "tx_vlan_offload": "on", "ntuple_filters": "off [fixed]", "receive_hashing": "off [fixed]", "highdma": "on", "rx_vlan_filter": "off [fixed]", "vlan_challenged": "off [fixed]", "tx_lockless": "on [fixed]", "netns_local": "off [fixed]", "tx_gso_robust": "off [fixed]", "tx_fcoe_segmentation": "off [fixed]", "tx_gre_segmentation": "on", "tx_gre_csum_segmentation": "on", "tx_ipxip4_segmentation": "on", "tx_ipxip6_segmentation": "on", "tx_udp_tnl_segmentation": "on", "tx_udp_tnl_csum_segmentation": "on", "tx_gso_partial": "off [fixed]", "tx_sctp_segmentation": "on", "tx_esp_segmentation": "off [fixed]", "tx_udp_segmentation": "off [fixed]", "fcoe_mtu": "off [fixed]", "tx_nocache_copy": "off", "loopback": "off [fixed]", "rx_fcs": "off [fixed]", "rx_all": "off [fixed]", "tx_vlan_stag_hw_insert": "on", "rx_vlan_stag_hw_parse": "on", "rx_vlan_stag_filter": "off [fixed]", "l2_fwd_offload": "off [fixed]", "hw_tc_offload": "off [fixed]", "esp_hw_offload": "off [fixed]", "esp_tx_csum_hw_offload": "off [fixed]", "rx_udp_tunnel_port_offload": "off [fixed]", "tls_hw_tx_offload": "off [fixed]", "tls_hw_rx_offload": "off [fixed]", "rx_gro_hw": "off [fixed]", "tls_hw_record": "off [fixed]"}, "timestamping": ["tx_software", "rx_software", "software"], "hw_timestamp_filters": []}, "ansible_default_ipv4": {"gateway": "10.77.25.1", "interface": "eth0", "address": "10.77.25.134", "broadcast": "10.77.25.255", "netmask": "255.255.255.0", "network": "10.77.25.0", "macaddress": "00:16:3e:5e:f7:32", "mtu": 1500, "type": "ether", "alias": "eth0"}, "ansible_default_ipv6": {"gateway": "fe80::216:3eff:fe9a:5639", "interface": "eth0", "address": "fd42:40f4:bbc1:54e9:216:3eff:fe5e:f732", "prefix": "64", "scope": "global", "macaddress": "00:16:3e:5e:f7:32", "mtu": 1500, "type": "ether"}, "ansible_all_ipv4_addresses": ["10.77.25.134"], "ansible_all_ipv6_addresses": ["fd42:40f4:bbc1:54e9:216:3eff:fe5e:f732", "fe80::216:3eff:fe5e:f732"], "ansible_env": {"USER": "root", "HOME": "/root", "container": "lxc", "TERM": "xterm-256color", "PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin", "LANG": "C.UTF-8", "PWD": "/root"}, "ansible_date_time": {"year": "2021", "month": "06", "weekday": "Thursday", "weekday_number": "4", "weeknumber": "23", "day": "10", "hour": "08", "minute": "24", "second": "39", "epoch": "1623313479", "date": "2021-06-10", "time": "08:24:39", "iso8601_micro": "2021-06-10T08:24:39.283780Z", "iso8601": "2021-06-10T08:24:39Z", "iso8601_basic": "20210610T082439283780", "iso8601_basic_short": "20210610T082439", "tz": "UTC", "tz_dst": "UTC", "tz_offset": "+0000"}, "ansible_apparmor": {"status": "enabled"}, "ansible_python": {"version": {"major": 3, "minor": 8, "micro": 5, "releaselevel": "final", "serial": 0}, "version_info": [3, 8, 5, "final", 0], "executable": "/usr/bin/python3.8", "has_sslcontext": true, "type": "cpython"}, "gather_subset": ["all"], "module_setup": true}, "invocation": {"module_args": {"gather_subset": ["all"], "gather_timeout": 10, "filter": [], "fact_path": "/etc/ansible/facts.d"}}}\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX exec_command: /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/ > /dev/null 2>&1 && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1623313473.107566-15069-6205902310727/ > /dev/null 2>&1 && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX close
[WARNING]: Platform linux on host type3 is using the discovered Python interpreter at /usr/bin/python3.8, but future installation of another Python
interpreter could change the meaning of that path. See https://docs.ansible.com/ansible/2.11/reference_appendices/interpreter_discovery.html for more
information.
ok: [type3]
META: ran handlers

TASK [Ping Server] ***************************************************************************************************************************************
task path: /home/testuser/data/ansiblelxd/Python/ansible/playbook.yml:6
XXX connect
<IP.IP.IP.IP> ESTABLISH SSH CONNECTION FOR USER: root
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'which lxc'
<IP.IP.IP.IP> (0, b'/usr/bin/lxc\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX connect
<IP.IP.IP.IP> ESTABLISH SSH CONNECTION FOR USER: root
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'which lxc-info'
<IP.IP.IP.IP> (1, b'', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to IP.IP.IP.IP closed.\r\n')
LXC v2
XXX exec_command: /bin/sh -c 'echo ~root && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'echo ~root && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'/root\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX exec_command: /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250 `" && echo ansible-tmp-1623313482.4964542-15080-73430535004250="` echo /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250 `" ) && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250 `" && echo ansible-tmp-1623313482.4964542-15080-73430535004250="` echo /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250 `" ) && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'ansible-tmp-1623313482.4964542-15080-73430535004250=/root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
Using module file /home/testuser/venv/lib/python3.8/site-packages/ansible/modules/ping.py
XXX connect
<IP.IP.IP.IP> PUT /home/testuser/.ansible/tmp/ansible-local-150637aeax772/tmpqd8l10fu TO /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/AnsiballZ_ping.py
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'cat > /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/AnsiballZ_ping.py; echo -n done'"'"''
<IP.IP.IP.IP> (0, b'done', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
XXX exec_command: /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/ /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/AnsiballZ_ping.py && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/ /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/AnsiballZ_ping.py && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX exec_command: /bin/sh -c '/usr/bin/python3.8 /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/AnsiballZ_ping.py && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'/usr/bin/python3.8 /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/AnsiballZ_ping.py && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'\r\n{"ping": "pong", "invocation": {"module_args": {"data": "pong"}}}\r\n', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX exec_command: /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/ > /dev/null 2>&1 && sleep 0'
XXX connect
<IP.IP.IP.IP> SSH: EXEC ssh -vvv -o ControlMaster=auto -o ControlPersist=900s -o PreferredAuthentications=publickey -o StrictHostKeyChecking=no -o 'IdentityFile="/home/testuser/playbook_keys"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey,gssapi-with-mic,gssapi-keyex,hostbased -o PasswordAuthentication=no -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/home/testuser/.ansible/cp/96d0c51d80 -tt IP.IP.IP.IP 'lxc exec playbook2 --mode=non-interactive -- /bin/sh -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'rm -f -r /root/.ansible/tmp/ansible-tmp-1623313482.4964542-15080-73430535004250/ > /dev/null 2>&1 && sleep 0'"'"'"'"'"'"'"'"''"'"''
<IP.IP.IP.IP> (0, b'', b'OpenSSH_8.2p1 Ubuntu-4ubuntu0.2, OpenSSL 1.1.1f  31 Mar 2020\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files\r\ndebug1: /etc/ssh/ssh_config line 21: Applying options for *\r\ndebug2: resolve_canonicalize: hostname IP.IP.IP.IP is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 14512\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to IP.IP.IP.IP closed.\r\n')
XXX close
ok: [type3] => {
    "changed": false,
    "invocation": {
        "module_args": {
            "data": "pong"
        }
    },
    "ping": "pong"
}
META: ran handlers
META: ran handlers

PLAY RECAP ***********************************************************************************************************************************************
type3                      : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

@AnvithLobo
Copy link

managed to fix it.

changed the line from

self.user = self.get_option("remote_user")

to

self.user = self._play_context.remote_user

and

key = self.get_option("private_key_file")

changed to

key = self._play_context.private_key_file

@stefangweichinger
Copy link

my local test setup also works OK: running my debops-setup against a Debian11-LXC-host-VM with several LXC-containers inside.

But I don't have the special case with the separate ansible_user. Do your new tests test that case as well?
Otherwise we wait for the other participants to have a test run.

@andreasscherbaum
Copy link
Owner

Given all the different open PRs and comments: may I suggest a brief call and see what is required, in which order, what's still open and what can be merged?

@stefangweichinger
Copy link

stefangweichinger commented Jun 21, 2021

I agree! ;-)

Quick look: #41 could be dropped now, IMO.
#39 and #42 seem both to add lxc_container, maybe #42 is more complete (without having checked that)?

EDIT: #42 also takes care of the removed version check ... I think it should be reviewed if we keep the version check in as suggested in faa7f94

@dmp1ce
Copy link

dmp1ce commented Jun 22, 2021

#40 adds tests which actually test this plugin. I highly recommend looking over that PR and merging it before changing anything else.

@andreasscherbaum
Copy link
Owner

#40 is failing in the current state, but the overall approach looks good. Once the self-contained tests are working I can merge that PR.

@andreasscherbaum
Copy link
Owner

@stefangweichinger agreed to have a call about all of the open PRs. I'm in CEST timezone, but can accommodate most times of the day.

@stefangweichinger
Copy link

@stefangweichinger agreed to have a call about all of the open PRs. I'm in CEST timezone, but can accommodate most times of the day.

Oh, you really mean a literal call, I understood your suggestion like "we should all discuss and decide now" ... like a "call to action" ;-)

Does #38 now solve the original issue for @boucman and @AnvithLobo ?

I am a bit biased: I would drop #41, merge #38, somehow decide what to take out of #39 and #42 (maybe only focus on #42) ...

Maybe we should do it in steps: work on all this in a separate development branch until all has been resolved.

@andreasscherbaum
Copy link
Owner

@stefangweichinger Yes. We have 3 Issues and 6 PRs going on, and everyone is discussing everywhere, making suggestions which PR can be dropped and merged in other PRs. At least for me this is confusing, sorry. I'd like to solve this situation. We can continue discussing this in more comments.

@stefangweichinger
Copy link

@andreasscherbaum I agree, it's confusing. And I agree we should get on with it.

I don't know yet how to see #40: is it OK and the tested script is not, or is the test not OK yet? Can't tell right now.

IMO @dmp1ce and @AnvithLobo could review and improve the PRs #40, #41, #42 and maybe distill it into one working PR. This PR might then be merged and #38 rebased on top of that. What do you think?

@stefangweichinger
Copy link

@stefangweichinger agreed to have a call about all of the open PRs. I'm in CEST timezone, but can accommodate most times of the day.

If you prefer a call, why not? I assume we both speak german ;-) / I think of suggesting a PR containing all the stuff ...

@stefangweichinger
Copy link

@andreasscherbaum forgive me if this confuses even more ;-)

I did some ugly cherry-picking into a separate branch https://github.com/stefangweichinger/ansible-lxc-ssh/commits/alltogether
It's not beautifully done and the history looks bad, I am sure I could redo it more elegant.

Basically it contains #38, #39, #40, #42 and skips the parts where the version-checks were removed (yes, LXCv1 might not be relevant anymore ... otherwise: why not keep it in there if it doesn't do harm ... and the new way of doing the checks prepares for other future checks).

If I haven't missed or screwed up something, this might be approximately what we want.

The tests succeeded here: https://github.com/stefangweichinger/ansible-lxc-ssh/actions/runs/961476719

I do not yet file another PR for that .... please have a look, folks

@dmp1ce @boucman @AnvithLobo

@AnvithLobo
Copy link

@stefangweichinger
Copy link

I don't understand yet. @AnvithLobo : you point at a test that exists. What do you mean?

@AnvithLobo
Copy link

AnvithLobo commented Jun 24, 2021

yeah no we added a nested lxc container we need the same checks in the first container to check for LXD version like it does for the host machine. Right now it assumes that the first level of container has lxd v4.

@stefangweichinger
Copy link

I think I understand but I am not sure if I can add that. Maybe you are faster than me here? ;-)

@stefangweichinger
Copy link

yeah no we added a nested lxc container we need the same checks in the first container to check for LXD version like it does for the host machine. Right now it assumes that the first level of container has lxd v4.

I am trying to come up with some patches, but I lose track here: why do we need to install (and check the version of) LXC/LXD inside of nested_container ?

Trying to store the version at first via a new tests/test-lxd-version.yml now.

@AnvithLobo
Copy link

lxd init preseed is a bit different for V3 and V4.
But LXD usually in all my cases were pre installed on ubuntu container images. But it looks like these containers for some reason didn't have them installed. Though there is a good change that apt install LXD would install LXD v4 inside the container too it'd be best to check to make sure. And if and when v5 is released and if preseed changes again we will know the reason and can add checks quicker.

@dmp1ce
Copy link

dmp1ce commented Jun 24, 2021

I don't know yet how to see #40: is it OK and the tested script is not, or is the test not OK yet? Can't tell right now.

#40 is a good test to the best I can tell. I copied work from @AnvithLobo mostly. The #40 PR is causing a FAIL, which is actually a good sign, because the ssh_lxc plugin is actually broken, as many have pointed out. #41 caused the test to pass as you can see in the PR results.

@dmp1ce
Copy link

dmp1ce commented Jun 24, 2021

I would suggest merge #40 and then new PRs can be more easily verified by checking to see if the tests pass.

@andreasscherbaum
Copy link
Owner

2 days later and we are still discussing in comments what is good and should be merged ;-)

@dmp1ce
Copy link

dmp1ce commented Jun 24, 2021

The current tests which are running are worse than nothing, because they are giving false positives.

Well, they gave a good starting point to build from, but the results they are returning aren't useful.

@andreasscherbaum
Copy link
Owner

I understand this is complex, and quite a few people have stakes here, and want to get this improved and PRs merged.
My proposal is to discuss this, early next week (or maybe this weekend).
@dmp1ce is Eastern Time, @stefangweichinger is CEST, so Stefan and my afternoon or evening would be day over for David. I don't know where @AnvithLobo is located.
Opinions?

@dmp1ce
Copy link

dmp1ce commented Jun 25, 2021

I'm not going to hold a hard stance on anything. I'm only trying to contribute code and keep the plugin working for people.

I'm happy to discuss over chat, but I doubt I'll be able to schedule a real-time chat.

@moqmar
Copy link

moqmar commented Jun 28, 2021

Does #38 now solve the original issue for @boucman and @AnvithLobo ?

I can confirm that the PR solves this issue, which by the way also applies to the SSH port (ansible_ssh_port or hostname:1234 in inventory).

@stefangweichinger
Copy link

Phew, even more PRs now, quite some activity ;-) Maybe this repo needs one or more devel-branches, in which we can do merges and develop things (people would need to create PRs against devel, not against main/master ... ).

Right now it looks confusing, at least to me. The last PRs might or might not be already fixed by other older open PRs, right?

@andreasscherbaum
Copy link
Owner

It is confusing, indeed.

@AnvithLobo
Copy link

the ansible.builtin.ssh plugin in itself has some sort of issue with ssh_pass that i haven't been able to debug yet. It fails at random times and works for a while after you make ansible login with ssh key.

I'm not sure if lxc_ssh also carries over this problem and if above PR solves it.

I currently stick to a workflow where paramiko_ssh sets up ssh key on container host machine and sets up most host side of things and then have lxc_ssh use that ssh key for working on containers.

@stefangweichinger
Copy link

Well, aren't we running in circles here?

I tried to come up with a combined step in #37 (comment) ...

#42 is in there. And I am not sure if "the results they are returning aren't useful" is still true für #38 and #42.

If yes, why "I would suggest merge #40 and then new PRs can be more easily verified by checking to see if the tests pass". If the tests aren't functional, why merge them and go on based on false positives?

We need to agree on the next steps and their order. Discussion is good, but what is the next problem we can actually fix to proceed?

@dmp1ce
Copy link

dmp1ce commented Jun 29, 2021

Well, my vote is for fixing the tests with #40, then applying a PR which makes the tests green such as #41.

@andreasscherbaum
Copy link
Owner

@boucman Can you please see if #38 fixes your problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants