Solvedcharts [bitnami/redis] Sentinel cluster doesn't elect new master after master pod deletion
30 Answers
Share
Original✔️Accepted Answer
Mitigation: We've been able to mitigate this issue by reducing
sentinel.downAfterMilliseconds
andsentinel.failoverTimeout
to beat the pod restart delay.
Looks like this works, I was able to workaround the issue by setting these two config options to
- downAfterMilliseconds: 60000
- failoverTimeout: 18000
+ downAfterMilliseconds: 4000
+ failoverTimeout: 2000
Other Answers:
Hi @wilsoniya,
sorry, is still a work in progress, when we have more information we will update the issue
Related Issues:
13
charts [bitnami/redis] Sentinel cluster doesn't elect new master after master pod deletion
Mitigation: We've been able to mitigate this issue by reducing sentinel.downAfterMilliseconds and se...
3
charts Keycloak not working behind nginx ingress controller
@javsalgar @boxcee i have tried your example and as i mentioned above it doesn't work i have tried a...
746
helm Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "configmaps" in API group "" in the namespace "kube-system"
Not good to close an issue without explaining the way to fix :-) Let me do it instead then ...
673
helm User "system:serviceaccount:kube-system:default" cannot get namespaces in the namespace "default"
That's because you don't have the permission to deploy tiller add an account for it: Console output:...
513
helm Error: no available release name found
Per #2224 (comment) the following commands resolved the error for me: Hi folks i just don't have any...
497
helm unable to retrieve the complete list of server APIs
For anyone who hits this it's caused by api-services that no longer have backends running.. In my ca...
290
helm helm upgrade --install no longer works
I thought I was experiencing the same problem but it turned out I just had an old delete (but not pu...
202
helm system:default not sufficient for helm, but works for kubectl
Per jayunit100 's advice add role cluster-admin to serviceaccount kube-system:default works for me b...
180
helm Values not exist with loop over a range
@worldsayshi For what it's worth I just ran into this same issue and it turns out there's a relevant...
150
helm Support for using {{ values }} within values.yaml
YAML also provides a handy feature called anchors which let you easily duplicate content across your...
145
helm Accessing values of the subchart with dash in the name
Use the index template function to access a value with a '-' in the name: {{ index .Values gitlab-ru...
144
helm Helm init fails on Kubernetes 1.16.0
The following sed works-for-me: The issue with @mattymo solution (using kubectl patch --local) is th...
135
helm Scoping releases names to namespaces
Explicitly It would be really nice if I could do the same things as I can do with services and other...
118
helm Error converting YAML to JSON could not find expected ':'
For what it's worth I have just run into very curious bug with (.Files.Glob configOverrides/*).AsCon...
106
helm Error: UPGRADE FAILED: cannot re-use a name that is still in use
For those stuck when initial deployment fails After you fix your chart but still cannot deploy with ...
94
react chartjs 2 Mix chart, labels.slice is not a function error
After fiddling with this for a little while I found a way to fix the example It is very easy and I d...
93
charts Helm Install for prometheus fails v3.0.1
In Helm 3 the stable repository is not set You have to manually add it Then you can use stable chart...
90
helm Error: UPGRADE FAILED: no resource with the name "anything_goes" found
Completely removing release from Helm via helm delete release works but it is not a viable solution ...
88
helm Warning after upgrading to 3.3.3
To close out this ticket.. Your ~/.kube/config should only be readable by the current user not the g...
78
helm Connection refused error when issuing helm install command
@cookkkie The service account needs to have a clusterRole: Admin default does not have a cluster adm...
77
react chartjs 2 Defaults creating error
got the same issue solve it by change the chart.js: ^2.9.4 in your package.json then delete node mod...
70
helm Add 'helm install --app-version' command/versioning a chart against an app version
Any update on this It seems PR from @Eraac does what is requested As @TD-4242 mentioned we also run ...
58
primeng PrimeNG 4.0.0-rc.1 error: Found the synthetic property @dialogState. Please include either "BrowserAnimationsModule" or "NoopAnimationsModule" in your application
fixed by: If you rely on Animations you’ll also need to install the animations package @angular/anim...
55
apexcharts.js x-axis is displayed as local datetime instead of UTC
for incoming people facing this: as @drpuur showed in his codepen just add: this will fix it for bot...
51
helm Error when updating Statefulsets
I had a similar error when adding a 2nd container (k8s 1.7.8) The first time I'm updating a stateful...
48
helm Using .Chart.name in range function
Just for reference here in case you are landing to this issue the documented way is working ...
44
helm Unable to perform helm upgrade due to resource conflict
This doesn't really help because I still need to manually remove old resources Output of helm versio...
43
helm Naming convention of Release and Chart
We ran into the exact same issue Usually the chart name is your app name I'm having difficulties und...
41
primeng Primeng breaks new Angular 9 project
Hello @mehmet-erim I have tested on this version Angular version: 9.0.0-rc.6 Primeng version: 9.0.0-...
40
charts [stable/prometheus-operator] Default servicemonitors are not scraping correctly: coredns, kube-controller-manager, kube-etcd, kube-scheduler, kubelet
On my AWS EKS cluster I have opted to not have prometheus scrape/monitor etcd kube-scheduler or kube...
39
helm Helm3: No 'init', doesn't use existing ~/.helm
Found this in another issue: ./helm version version.BuildInfo{Version:v3.0.0-beta.1 GitCommit:f76b5f...
37
charts stable/kubernetes-dashboard: User "system:serviceaccount:default:default" cannot create secrets in the namespace "kube-system"
The issue exists quite a while Would be good for new helm users to address this in the Readme ...
37
helm app-name has no deployed releases
Cannot agree more Our production is experiencing the same error So deleting the chart is not an opti...
34
helm Error forwarding ports: error upgrading connection
Hi I have Helm install locally and Tiller on my cluster everything looks healthy but running helm in...
33
helm Release failure or error logs
helm status only shows that the release is failed not quite useful for debugging helm install will p...
33
primeng Chart is not defined error
Adding: To my index.html page resolved this issue for me. when we try to use doughnutChart component...
32
helm Proposal: Allow templating in values.yaml
@alexbrand You could use the tpl function to render values that contain variables: values.yaml templ...
29
helm "required" template function breaks linter
Having a default value for a required values in the charts values.yaml file defeats the whole purpos...
28
apexcharts.js Make Chart Margins and Padding Optional/Adjustable
@benhaynes The way to remove all paddings/margins would be hide the y-axis by yaxis.show = false Rem...
28
charts [stable/rabbitmq] endpoints does not exist
Hi @rnkhouse PVC is Persistent Volume Claim To delete them first obtain the name with: Copy the name...
27
helm UPGRADE FAILED: No resource with the name "" found
This is a process I use to recover from this problem (so far it has worked every time without any in...
27
helm Error: could not find a ready tiller pod
Run this command may solve your issue kubectl logs --namespace kube-system tiller-deploy-2654728925-...
27
nivo Support calendars less than a year long
Having granularity of months would be awesome and should solve most use cases smaller than a year th...
26
victory native victory-native is not compatible with react-native^0.56
This work is published as victory-native@30.0.0 Thank you to everyone in this thread for their help ...
26
helm helm v3.0 mapping values are not allowed in this context
I get the same error message with our SonarQube chart for OpenShift The same install works with Helm...
26
helm charts prometheus-kube-stack unknown fields probenamespaceselector and probeSelector
Can you try to delete the crds before retrying to install the chart? See https://github.com/promethe...
25
charts [stable/nginx-ingress] toYaml can't render configmap with booleans
same here :( I was able to make it work (fix) by using --set-string For example: ...
25
helm Secret Management in Helm
The method proposed above does not seem like it would work with helm rollback or simple helm upgrade...
25
helm Helm 2.2.3 not working properly with kubeadm 1.6.1 default RBAC rules
After running helm init When installing a cluster for the first time using kubeadm v1.6.1 the initia...
24
helm helm cant upgrade already created resources
For the benefit of the group I am using helm 2.9.1 and I want to create a namespace with an annotati...
Which chart:
Chart:
bitnami/redis
Version:
13.0.1
Describe the bug
When a master pod is manually deleted, occasionally the remaining replicas appear to continue re-electing the nonexistent master. When the replacement pod reappears, it's unable to connect to the existing master as reported by the remaining replicas, which corresponds to the IP of the now nonexistent previous master pod.
To Reproduce
I'm not able to deterministically reproduce the behavior described above. I'd say the errant behavior occurs ~20% of the time.
Steps to reproduce the behavior:
Expected behavior
When a pod is deleted the cluster members should elect a new master among themselves and the replacement pod should be able to connect to the elected master when the replacement comes online.
Version of Helm and Kubernetes:
helm version
:kubectl version
:Additional context
values
installation command
cluster log output
The output below occurs on an otherwise healthy sentinel cluster after I run
kubectl delete pod redis-node-2
(please note: the logging is collected viastern
which I believe explains theunexpected error: stream error: stream ID 19; INTERNAL_ERROR
occurrences).