Solvedansible json_query filter fails when using the functions "contains", "starts_with", others
✔️Accepted Answer
The problem is related to the fact that Ansible uses own types for strings: AnsibleUnicode
and AnsibleUnsafeText
.
And as long as jmespath library has very strict type-checking, it fails to accept this types as string literals.
There was a clever hint posted here: #20379 (comment)
It is used to alter jmespath's typemap to accept Ansible custom types.
I was going to file an issue, but pasting my repro-code here instead:
- hosts: localhost
gather_facts: no
vars:
qry1: "[?contains(foo, 'bar')]"
qry2: "[?contains(item, '1')].cmd"
json1:
- foo: bar
- foo: test
tasks:
- name: This fails because type(foo) = AnsibleUnicode
debug:
msg: "{{ json1 | json_query(qry1) }}"
ignore_errors: yes
- name: This works
debug:
msg: "{{ json1 | to_json | from_json | json_query(qry1) }}"
- name: This generates registered facts that gets SafeWrapped
command: echo {{ item }}
with_sequence: count=2
register: cmd_res
changed_when: no
- name: This fails because type(foo) = AnsibleUnsafeText
debug:
msg: "{{ cmd_res.results | json_query(qry2) }}"
ignore_errors: yes
- debug:
msg: "{{ cmd_res.results | to_json | from_json | json_query(qry2) }}"
Other Answers:
| to_json | from_json
workaround works for me with join
function. Thx @berlic
Just two years later, this issue is still present in ansible 2.8.1.
At least a note in documentation would be great, as it would save an hour of troubleshooting to every single person hitting this road block.
Is there anyone who can/know how to submit an improvement for ansible doc?
I think if noone's going to fix this, can someone at least put a note in the json_query documentation about using to_json | from_json
when using JMESPath functions/methods?
I have a good news and a bad news.
I have a fix/workaround but.. I don't know why it works
to fix, in json_query.py add import ast
then add data = ast.literal_eval(str(data))
in json_query()
It convert the original data, which is a "dict" and when printed, looks good, to a string and convert it back to dict.
I'm not understanding the fix but it works.. I'll not make a PR has I have no explanation and also because the fix may have to be done elsewhere.
At the end may be the issue come from Jinja. I assume json_query() is called from jinja, could it be that Jinja manipulate 'data' in a way who change it's internal representation for python and produce this ?
I attach my json_query.py patched version. @bdlamprecht @Imran-ibm you may want to try it
json_query.zip
ISSUE TYPE
COMPONENT NAME
json_query
ANSIBLE VERSION
CONFIGURATION
Standard
OS / ENVIRONMENT
N/A
SUMMARY
When using the
json_query
filter with the query[?interface.name=='irb.1111'].{interface: interface.name, address: address}
everything functions fine and I get the correct results back. When that filter is changed to[?contains(interface.name,'irb')].{interface: interface.name, address: address}
traceback errors withjmespath/functions.py
start occurring that complain aboutinvalid type for value
.A previous bug (#20379) along the same lines was opened back in January against Ansible v2.2.1, but hasn't been resolved yet.
STEPS TO REPRODUCE
Provided results.json
That works as expected with the following results:
However, when you change the line
with_items
in the PB above to usejq_new
instead ofjq_old
, this traceback occurs:EXPECTED RESULTS
Something along the lines of (from http://jmespath.org):
ACTUAL RESULTS
See above for the traceback that occured.