SolvedRocket.Chat Migration Issues after Update Release Version

Description:

• Just updated my six weeks old Rocket Chat Docker Stack to the current "lastest" Version and got database migration issues.

  • image: registry.rocket.chat/rocketchat/rocket.chat:latest

• My docker-compose Template is the following:

  • https://github.com/RocketChat/Rocket.Chat/blob/develop/docker-compose.yml

  rocketchat:
    image: registry.rocket.chat/rocketchat/rocket.chat:latest
    command: >
      bash -c
        "for i in `seq 1 30`; do
          node main.js &&
          s=$$? && break || s=$$?;
          echo \"Tried $$i times. Waiting 5 secs...\";
          sleep 5;
        done; (exit $$s)"
    restart: unless-stopped
    volumes:
      - ./uploads:/app/uploads
    environment:
      - PORT=3000
      - ROOT_URL=http://localhost:3000
      - MONGO_URL=mongodb://mongo:27017/rocketchat
      - MONGO_OPLOG_URL=mongodb://mongo:27017/local
    depends_on:
      - mongo
    ports:
      - 3000:3000
    labels:
      - "traefik.backend=rocketchat"
      - "traefik.frontend.rule=Host: your.domain.tld"

Actual behavior:

Rocket Chat seems to be stuck in its Database Migration process. Maybe just a database Script is missing.

Got this logs:

{"level":40,"time":"2021-11-17T06:51:34.602Z","pid":57,"hostname":"6543064ea404","name":"Migrations","msg":"Not migrating, control is locked. Attempt 28/30. Trying again in 10 seconds."}
/app/bundle/programs/server/node_modules/fibers/future.js:280
                                                throw(ex);
                                                ^

Error: Can't find migration version 243
    at migrateDatabase (server/lib/migrations.ts:229:9)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
    at migrateDatabase (server/lib/migrations.ts:181:11)
Tried 4 times. Waiting 5 secs...
LocalStore: store created at
LocalStore: store created at
LocalStore: store created at
{"level":40,"time":"2021-11-17T06:52:00.966Z","pid":73,"hostname":"6543064ea404","name":"Migrations","msg":"Not migrating, control is locked. Attempt 0/30. Trying again in 10 seconds."}

Server Setup Information:

The first Rocket Chat installation is 6 weeks ago.

• Version of Rocket.Chat Server: latest
• Operating System: Debian
• Deployment Method: docker
• Number of Running Instances: 1
• DB Replicaset Oplog: ??
• NodeJS Version: container
• MongoDB Version: container

registry.rocket.chat/rocketchat/rocket.chat:latest

That is a really bad habit to use tag latest with docker deployment in general and with rocketchat particular.
There already was problems in near past with that tag, just don't use it if you don't want to broke something,
Always use specific version tag to update.

For me it was enough to just pin the version to 4.1.2 without restoring the db

So you reacted with dislike on my advice, but still followed same advice. Nice!

Not really. Your advice is that using latest is generally a bad idea while I never had a problem with it exept for rocket chat.
My advice now is to pin it to 4.1.2, because latest just isn't the actual latest version. In other words pinning the version is a workaround while using latest should be the way to go imo

If you have been around here long enough you will know that Anton is one of the longest serving and most loyal community members running a very large deployment.

He doesn't offer advice for fun, and doesn't require down votes just because you don't happen to like his (correct) advice.

You'd also know if you followed things closely here the advice has been the same for a long time - NEVER use 'latest' on a production machine. No sysadmin wants unpleasant surprises, and a prudent one will always check releases for breakages etc before upgrading. This is not the first time this has ever happened at Rocket, and I very much doubt it will be the last.

Check the next release on a test machine, make sure it works as expected, upgrade on your own prepared schedule to a specific version knowing that it will work. That is just plain sensible.

Just because you got lucky and have 'got away with it' before doesn't mean you always will, as you have discovered.

Have a read online about 'don't use docker latest tag'. I'd read all of this which just found for starters

https://vsupalov.com/docker-latest-tag/

Thanks.

Workaround here - pinned the RocketChat docker image to 4.1.2 and restored the Mongo db from last night's backup. Seems ok so far

For me it was enough to just pin the version to 4.1.2 without restoring the db

For me it was enough to just pin the version to 4.1.2 without restoring the db

So you reacted with dislike on my advice, but still followed same advice. Nice!

Not really. Your advice is that using latest is generally a bad idea while I never had a problem with it exept for rocket chat. My advice now is to pin it to 4.1.2, because latest just isn't the actual latest version. In other words pinning the version is a workaround while using latest should be the way to go imo