ansible@ansible:~$ ansible all -i debian, -m ping -u ansible -k -b

The problem is that it took the debian vm too long to switch to root with sudo. This happens due to an incorrectly configured dns of its own hostname.

I added the hostname to the local /etc/hosts file.