Solvedemissary Dev Portal attempts to find docs for every mapping by default

Describe the bug
Hi, I am using Ambassador 1.41. and Isito 1.51 on AWS. Istio is working file without mtls. and requests are working through mappings and here using I'm self-signed certificate and its working but additionally every 10, 15 seconds or on every hitting request its getting 404. don't know what is exactly needs output, hitting to .ambassador-internal/openapi-docs

Ambassador pod logs:

2020-04-20T07:08:47.820739878Z ACCESS [2020-04-20T07:08:40.299Z] "GET /test/ghost/.ambassador-internal/openapi-docs HTTP/1.1" 404 - 0 183 1 0 "10.191.243.124" "Go-http-client/1.1" "47376ecf-4352-9420-ba44-5fed720f64f0" "127.0.0.1:8443" "172.20.255.115:9091"
2020-04-20T07:08:47.820742511Z ACCESS [2020-04-20T07:08:40.303Z] "GET /test/ghost2/.ambassador-internal/openapi-docs HTTP/1.1" 404 - 0 181 1 0 "10.191.243.124" "Go-http-client/1.1" "2885effe-5f4d-92c4-a58b-0f39fdd465eb" "127.0.0.1:8443" "172.20.132.127:9092"

Istio proxy logs:

2020-04-20T06:32:41.295495213Z [2020-04-20T06:32:40.287Z] "GET /test/ghost/.ambassador-internal/openapi-docs HTTP/1.1" 404 - "-" "-" 0 181 0 0 "10.191.243.124" "Go-http-client/1.1" "81fd0683-505e-9191-8711-81c39ec074fa" "127.0.0.1:8443" "127.0.0.1:9091" inbound|9091|http|ghost-http-svc.namespace.svc.cluster.local 127.0.0.1:50018 10.191.243.90:9091 10.191.243.124:0 - default
2020-04-20T06:33:41.295919366Z [2020-04-20T06:33:40.300Z] "GET /test/ghost2/.ambassador-internal/openapi-docs HTTP/1.1" 404 - "-" "-" 0 181 0 0 "10.191.243.124" "Go-http-client/1.1" "d5dd610f-67ff-9b0d-add3-a26aacca1faa" "127.0.0.1:8443" "127.0.0.1:9092" inbound|9092|http|ghost2-http-svc.namespace.svc.cluster.local 127.0.0.1:51286 10.191.243.90:9092 10.191.243.124:0 - default

TLS log from ambassador pod

2020-04-20T06:53:33.863263626Z status_dict {'Error check': {'status': True, 'specifics': [(True, 'No errors logged')]}, 'TLS': {'status': True, 'specifics': [(True, '1 TLSContext is active')]}, 'Mappings': {'status': True, 'specifics': [(True, '15 Mappings are active')]}}

Expected behavior
why this 404 getting
Versions (please complete the following information):

  • Ambassador: [1.4.1]
  • Kubernetes environment [e.g. Minikube, bare metal, Google Kubernetes Engine]
  • Version [1.15]

more logs

2020-04-20T07:00:26.322180559Z [2020-04-20 07:00:26.322][262][debug][http] [source/common/http/conn_manager_impl.cc:708] [C184840][S16622113245629541650] request headers complete (end_stream=true):
2020-04-20T07:00:26.322185245Z ':authority', 'elb.us-east-1.elb.amazonaws.com'
2020-04-20T07:00:26.322188905Z ':path', '/test/ghost'
2020-04-20T07:00:26.322192904Z ':method', 'GET'
2020-04-20T07:00:26.322195883Z 'user-agent', 'curl/7.64.1'
2020-04-20T07:00:26.322198933Z 'accept', '/'
2020-04-20T07:00:26.32220149Z 'test', 'dev'
2020-04-20T07:00:26.322204555Z
2020-04-20T07:00:26.322207164Z [2020-04-20 07:00:26.322][262][debug][http] [source/common/http/conn_manager_impl.cc:1257] [C184840][S16622113245629541650] request end stream
2020-04-20T07:00:26.322331695Z [2020-04-20 07:00:26.322][262][debug][router] [source/common/router/router.cc:434] [C0][S17947907241811361392] cluster 'cluster_extauth_127_0_0_1_8500_ambassador' match for URL '/envoy.service.auth.v2alpha.Authorization/Check'
2020-04-20T07:00:26.322337215Z [2020-04-20 07:00:26.322][262][debug][router] [source/common/router/router.cc:549] [C0][S17947907241811361392] router decoding headers:
2020-04-20T07:00:26.3223396Z ':method', 'POST'
2020-04-20T07:00:26.322341612Z ':path', '/envoy.service.auth.v2alpha.Authorization/Check'
2020-04-20T07:00:26.322343627Z ':authority', 'cluster_extauth_127_0_0_1_8500_ambassador'
2020-04-20T07:00:26.322345752Z ':scheme', 'http'
2020-04-20T07:00:26.322347754Z 'te', 'trailers'
2020-04-20T07:00:26.322349516Z 'grpc-timeout', '5000m'
2020-04-20T07:00:26.322351319Z 'content-type', 'application/grpc'
2020-04-20T07:00:26.322353166Z 'x-ot-span-context', 'EhQJrypZ/X31wLQR+wYi4lTiyG4YAQ=='
2020-04-20T07:00:26.322355185Z 'x-envoy-internal', 'true'
2020-04-20T07:00:26.322370444Z 'x-forwarded-for', '10.191.243.124'
2020-04-20T07:00:26.322373357Z 'x-envoy-expected-rq-timeout-ms', '5000'
2020-04-20T07:00:26.322376241Z
2020-04-20T07:00:26.322379341Z [2020-04-20 07:00:26.322][262][debug][pool] [source/common/http/http2/conn_pool.cc:98] [C42] creating stream
2020-04-20T07:00:26.322401258Z [2020-04-20 07:00:26.322][262][debug][router] [source/common/router/router.cc:1618] [C0][S17947907241811361392] pool ready
2020-04-20T07:00:26.322754595Z time="2020-04-20 07:00:26" level=info msg="[gRPC] HTTP/1.1 GET elb.us-east-1.elb.amazonaws.com /test/ghost" MAIN=http REQUEST_ID=16622113245629541650 SUB=http-handler
2020-04-20T07:00:26.322771504Z time="2020-04-20 07:00:26" level=info msg="using default rule" MAIN=http REQUEST_ID=16622113245629541650 SUB=http-handler
2020-04-20T07:00:26.322774557Z time="2020-04-20 07:00:26" level=info msg="selected rule host="", path="", filters=[]" MAIN=http REQUEST_ID=16622113245629541650 SUB=http-handler
2020-04-20T07:00:26.322786238Z time="2020-04-20 07:00:26" level=info msg="[gRPC] *filterapi.HTTPRequestModification : 0 headers (66.904µs)" MAIN=http REQUEST_ID=16622113245629541650 SUB=http-handler
2020-04-20T07:00:26.3229688Z [2020-04-20 07:00:26.322][262][debug][router] [source/common/router/router.cc:1036] [C0][S17947907241811361392] upstream headers complete: end_stream=false
2020-04-20T07:00:26.322975446Z [2020-04-20 07:00:26.322][262][debug][http] [source/common/http/async_client_impl.cc:93] async http request response headers (end_stream=false):
2020-04-20T07:00:26.32297907Z ':status', '200'
2020-04-20T07:00:26.3229824Z 'content-type', 'application/grpc'
2020-04-20T07:00:26.322985791Z 'trailer', 'Grpc-Status'
2020-04-20T07:00:26.322988837Z 'trailer', 'Grpc-Message'
2020-04-20T07:00:26.322991246Z 'trailer', 'Grpc-Status-Details-Bin'
2020-04-20T07:00:26.32301797Z 'x-envoy-upstream-service-time', '0'
2020-04-20T07:00:26.323021968Z

16 Answers

✔️Accepted Answer

+1 for this ticket, I would also like to disable this feature as it is generating large numbers of logs making it harder to debug issues with Ambassador.

we are not using this feature but I couldn't find how to disable the requests in the documentation.

p.s. we already disabled in the Helm chart the mappings with createDevPortalMappings: false but we can't disable the developer portal entirely.

Other Answers:

this is quite a misfeature. Please make it optional and also make it possible to change the URL.

As of Ambassador 1.9.0, which is now available, the Dev Portal no longer automatically polls .ambassador-internal/openapi-docs. See https://www.getambassador.io/docs/pre-release/topics/using/dev-portal/#developer-portal for more details.

Related Issues:

13
emissary Dev Portal attempts to find docs for every mapping by default
+1 for this ticket I would also like to disable this feature as it is generating large numbers of lo...
373
Zappa bad magic number in 'application': b'\x03\xf3\r\n': ImportError
I was able to fix this too by doing find -name \*.pyc -delete Mine definitely wasn't caused by PY2 -...
94
Zappa AttributeError: 'module' has no attribute 'get_installed_distributions'
You probably have a newer pip they made changes to their public api top level functions recently Try...
59
aws sam cli Error when installing python version on MACOS
Try with --user flag? Description: I got an error when trying to install the new version on MacOS St...
27
aws sam cli AttributeError in arn_generator.py
I am a new user following the hello world tutorial linked from the README and hit this issue I don't...
26
aws sam cli fork/exec /var/task/main: permission denied
Not sure if this is the case here I successfully deployed a golang lambda api service using sam pack...
24
aws sam cli AWS::Serverless::SimpleTable not Created in DynamoDB Local
@PaulMaddox wrote: Sorry for the confusion SAM Local currently only does anything with AWS::Serverle...
23
aws sam cli sam build fails when required to install some 3rd party libraries in requirements.txt
For those that don't want to run the build inside a container: pip install wheel fixed it for me wit...
22
kong how to let kong start automatically
you can use systemd for Control kong service First create kong.service file : and then put this line...
19
serverless express [Question] Internal server error when request method OPTIONS
With the help from staff on AWS forums I've resolved it by adding: which now returns a correct 200 r...
16
aws sam cli sam init errors with "No module named functools_lru_cache"
I had the same problem today Resolved be uninstalling and installing backports.functools_lru_cache ...
15
aws sam cli Local API Gateway responds with Internal Server error
Your timeout looks 3s try to increase in your template. Hi ...
14
spring cloud gateway Spring-cloud-gateway application not starting up.
Gateway is not compatible with org.springframework.boot:spring-boot-starter-web ...
14
spring cloud gateway io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
You usually see this when you have a http vs https mismatch I encountered this problem when my ssl r...
14
serverless express Async handler doesn't work on node 8.10
PR for this here #173 Note that I plan on improving the interface in a future breaking change (aws-s...
13
Zappa Error loading psycopg2 module on Python 3.6 (when on Lambda)
I solved the issue by installing psycopg2-binary Context Shiny new Zappa 0.41 and Python 3.6 Django=...
13
spring cloud gateway Cors Pre Flight Request
To version pre to 2.1.3.RELEASE this works for me: Spring Cloud Version: Greenwich.RELEASE To make m...
12
serverless next.js Started getting the Error: Cannot find module '@sls-next/next-aws-cloudfront'
Hi folks please pin the version number in your serverless.yml like it states in the README For examp...
12
serverless express Can't get the API Gateway event object
In case any other lost souls who are encountering Missing x-apigateway-event or x-apigateway-context...
11
spring cloud gateway Doubled CORS headers after upgrade to Greenwich
This works for me No need to add any bean. Some of the legacy back ends behind our gateway have thei...
10
spring cloud gateway Preflight Request
@hnxuruochen Just did a workaroundd below to set RoutePredicateHandlerMapping CORS configuration ...
9
aws sam cli sam local: --env-vars parameter does not work with "start-api" subcommand
If it helps anyone I found out that by setting empty environment variables in the template.yml ...
7
serverless next.js Using webpack 5 causes 503
Same for 10.2.3 Using webpack 5 returns the following from Cloudfront: Using versions: ...
6
serverless next.js Error: Source and destination must not be the same. When using useServerlessTraceTarget
I was also facing the same issue (getting Error: Source and destination must not be the same) and sp...
4
kong request-transformer plugin fails if multipart/form-data contains file
@arvileino that's really valuable Summary When request-transformer plugin enabled multipart/form-dat...
4
spring cloud gateway How to modify request / response body?
I didn't get how to change the response body I have a json in the body and I want to add a property ...
4
Zappa Error when deploying AWS Lambda with Zappa package
I seems that https://pypi.org/project/troposphere/ causes the problem: release 3.0.0 does not instal...
3
aws sam cli Conditionals, Refs, etc. in template yaml don't appear to be evaluated
Should this caveat be mentioned in the documentation? It took a lot of searching to hunt down this c...
3
serverless next.js 503 when using image optimization
It should be fixed now in@sls-next/serverless-component@1.19.0-alpha.19 after also include 7256ab9 W...
35
istio Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?)
I have the same problem (This is used to report product bugs ...
27
istio how to configure envoyfilter to support ratelimit in istio 1.5.0?
Any plans to support this natively in istio? because of the mixer policy was deprecated in Istio 1.5...
25
istio Better support for sidecar containers in batch jobs
Sidecar containers do not play well with k8s jobs The job will keep running so long as the sidecar p...
21
istio Istio give x-forwarded-proto=http to upstream server when using SSL termination
OK came across this issue #12549 so it's out as part of v1.3.x i have tested it with istio v1.4.0 an...
20
istio Connection Failure to a MySQL Service
Hi We are also facing this issue and found that it is related to the default MeshPolicy ...
19
istio EKS - creating networking configs time out
the following helm line (based upon Zack's Nightly article) fixed this for me: I am attempting to se...
19
istio To support Single Sign-On scenario, Istio Origin Authentication should accept a JWT Token sent in a cookie
Hey guys We ended up doing something similar to what @yskopets is doing so I'll share the code here ...
15
istio Support multiple virtual services with same host
I know the spec currently has A host name can be defined by only one VirtualService however ...
15
istio Secure communication between Prometheus and Istio components
I just wanted to follow-up although that PR did enable using built-in Prometheus with mTLS (thanks f...
14
istio examples on gRPC-JSON transcode needed
@begoat There are two ways: Use k8s configmap Create configmap from descriptor set Add annotations t...
14
istio Helm delete does not clean the custom resource definitions
That's right Helm: Istio: 1.0.0 Kubectl: Following a deletion of Istio 1.0.0 using helm delete --pur...
13
istio gRPC-web
I found that these settings work @arbarlow commented on Mon Oct 30 2017 As discussed in istio/old_pi...
13
istio Connection problems with large deployments (1200+ pods)
We have experienced a similar issue on our staging environment recently with Istio 1.0.1 ...
12
istio Istio Ingress TLS key management use ACM
@naveeng68 you need to set the tls.httpsRedirect on your gateway to true as used here: According to ...
9
istio Allow egress requests to variable IP CIDR ranges
That is unfortunately not good enough Currently outbound requests are blocked by default ...
9
istio Deploy ingressgateway as daemonset
@anthonyvk You can use: istiooperator.yaml I'm in the process of setting up a bunch of microservices...
8
istio istioctl install fails on windows
Finally got my hands on a windows machine Bug description The istioctl manifest apply --set profile=...
5
istio istioctl get authentication problem with heptio aws authenticator
I don't have Heptio or any Kubernetes client config that requires the Exec Provider ...
5
istio Can not create gateway
https://istio.io/docs/setup/kubernetes/helm-install/#option-1-install-with-helm-via-helm-template ...
4
istio Envoy filter not allowing headers in Istio?
I fixed this issue which I opened using below yaml file with envoy.lua filter I am using Istio 1.0.3...