Solvedansible modules core apt update_cache gives error on ubuntu for Ansible 2.0.0.2

ISSUE TYPE

Bug Report

COMPONENT NAME

apt module

ANSIBLE VERSION

N/A

SUMMARY

The simple following command:

- name: Update apt cache.
  apt: update_cache=true
  become: true

Throws an error on AWS Ubuntu instances.

fatal: [xx.xx.xxx.x]: FAILED! => {"changed": false, "failed": true, "module_stderr": "", "module_stdout": "Traceback (most recent call last):\r\n File "/home/ubuntu/.ansible/tmp/ansible-tmp-1454536847.17-86172912102079/apt", line 2630, in \r\n main()\r\n File "/home/ubuntu/.ansible/tmp/ansible-tmp-1454536847.17-86172912102079/apt", line 603, in main\r\n cache = apt.Cache()\r\n File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 107, in init\r\n self.open(progress)\r\n File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 151, in open\r\n self._cache = apt_pkg.Cache(progress)\r\nSystemError: E:Encountered a section with no Package: header, E:Problem with MergeList /var/lib/apt/lists/us-west-2.ec2.archive.ubuntu.com_ubuntu_dists_trusty-updates_universe_i18n_Translation-en, E:The package lists or status file could not be parsed or opened.\r\n", "msg": "MODULE FAILURE", "parsed": false}

However, when I ssh into the instance, and do sudo apt-get update manually, there is no problem at all. And then, if I ran the task again, the problem will go away, I guess because I did the manual update on the instance.

Any ideas?

Thanks

+1
Ansible 1.9.4
Exactly the same error, doing 'apt-get update' command succeeded while always failed with 3 retries of apt module

Removing /var/lib/apt/lists/* could resolve this but as command 'sudo apt-get update' works without that cleanup so seems this is due to Ansible apt module.

Right now I have to add a retry with command in case apt module failed

- name: apt update
  apt: update_cache=yes cache_valid_time=3600
  register: apt_result
  until: apt_result|success
  retries: 3
  delay: 1
  sudo: yes
  ignore_errors: yes

- name: retry if needed using command apt-get update
  command: apt-get update
  sudo: yes
  when: apt_result|failed

+1
Running sudo rm -vf /var/lib/apt/lists/* gets around the problem.

@jctanner please reopen this. it is not fixed
with ansible 2.2.1.0 we get

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: SystemError: E:Datei /var/lib/apt/lists/ftp.hosteurope.de_mirror_archive.ubuntu.com_dists_trusty_universe_binary-i386_Packages konnte nicht geöffnet werden. - open (2: No such file or directory)
failed: [ipm_promotion_stage] (item=[u'postgresql-9.3', u'postgresql-client-9.3']) => {"failed": true, "item": ["postgresql-9.3", "postgresql-client-9.3"], "module_stderr": "Traceback (most recent call last):\n File \"/tmp/ansible_6GV6h7/ansible_module_apt.py\", line 903, in <module>\n main()\n File \"/tmp/ansible_6GV6h7/ansible_module_apt.py\", line
799, in main\n    cache = apt.Cache()\n  File \"/usr/lib/python2.7/dist-packages/apt/cache.py\", line 107, in __init__\n    self.open(progress)\n  File \"/usr/lib/python2.7/dist-packages/apt/cache.py\", line 153, in open\n    self._records = apt_pkg.PackageRecords(self._cache)\nSystemError: E:Datei /var/lib/apt/lists/ftp.hosteurope.de_mirror_archive.ubuntu.com_dists_trusty_universe_binary-i386_Packages konnte nicht geöffnet werden. - open (2: No such file or directory)\n", "module_stdout": "", "msg": "MODULE FAILURE"}

There is no problem when doing this with apt-get directly. The files are there.
We do apt-get clean and rm -rf /var/lib/apt/lists/* when the playbook starts manually but no luck.

This is not breaking on every playbook run but completely random during different tasks that use the apt module.

+1
I've stopped using the apt module entirely because of this. Running:

shell: apt-get update

is much more reliable, so it seems the issue stems from the python apt module like others have pointed out. That being said you will get annoying warnings like this:

[WARNING]: Consider using apt-get module rather than running apt-get

sudo apt-get clean 
sudo apt-get update

this fixed it for me, running it on target machine