[Done] MongoDB Version 5 support

[smerchek]

Following up on this discussion.

In addition, we just received this email from MongoDB regarding upgrading cluster, so hoping something before February 2022 is possible.

Your M0/M2/M5 Atlas cluster(s) will automatically be upgraded to MongoDB 5.0 in early February 2022.

What’s in the update?
MongoDB 5.0 introduces new features and improvements such as:
A Versioned API, designed to preserve application behavior through upgrades
Native time series collections and clustered indexing
Window functions and new temporal operators

What do I need to do?
To ensure a seamless transition, we recommend updating your driver version to ensure continued compatibility with your applications. Visit our Docs to see which driver versions are compatible with 5.0 and download the latest version.

[filipenevola]

Hi, we've started studying the necessary changes to support MongoDB 5.0.

We are reviewing all the changes log and we are going to share our finds here soon.

The first step was to check the changes highlighted here. These changes are ok for us, the only change necessary at first was a new error name to be handled.

Now we need to check the full list here. It seems they removed a few deprecated options so we are going to double check all the use cases of these options inside Meteor.

[dkoo761]

Hi @filipenevola, can we get an update on the timeline for MongoDB 5.0 support in Meteor?

The big issue is that I presume many of us (myself included) have apps running on shared Atlas instances (M0/M2/M5) and in early February (only a few weeks away), all those instances will be automatically upgraded to Mongo 5.0 with NO ability to use setFeatureCompatibilityVersion to support older versions of Mongo since that flag can only be set on dedicated Mongo instances.

So....from my understanding, every Meteor app using Atlas M0/M2/M5 will stop working in a few weeks unless a solution is implemented. This seems like a ticking time bomb to me and could affect many Meteor apps including many of your Galaxy customers all on the same day.

If a new version of Meteor with MongoDB 5.0 support isn't released within the next couple weeks (to give us time to upgrade and test), I think there will be only 2 options for customers on shared Atlas instances:

1. Upgrade to a more expensive dedicated Atlas instance OR
2. Migrate from Atlas to a different MongoDB provider

It might also be worthwhile sending an email to the Meteor mailing list to ensure they're aware of the severity of this time-sensitive situation as I'm assuming many probably aren't.

[filipenevola]

Hi @dkoo761 we are working in a branch making Meteor mongo package compatible with MongoDB 5.

Now we are fixing tests that are breaking.

We are going to post updates frequently in this PR #11831

Our goal is to have beta versions of Meteor with these changes before the end of this month.

[dkoo761]

@filipenevola That's great news, that's very much for the quick update!

[renanccastro]

📣 Hello, everyone. 📣
We've just released a new Meteor beta version, 2.6-beta.2 which includes the support and update to MongoDB v5.
We really appreciate any time you can spend giving it a try and reporting any bugs/comments in the related PR here: #11831
To use it, run the following in an existing meteor project:
meteor update --release 2.6-beta.2
Thanks!

[filipenevola]

Hi, Meteor 2.6-beta.4 was released today.

• beta.3 fixed an issue with oplog.
• beta.4 merged 2.5.5 into Meteor 2.6 branch.

So please check your app behavior with Meteor 2.6-beta.4: meteor update --release 2.6-beta.4.

We've also started this draft for a Migration Guide to Meteor 2.6. Please provide your feedback in the release PR.

[rhymn]

Excellent writeup for the 2.6 migration 👍

[polygonwood]

Hi Filip,

upgraded my meteor app to 2.6-beta.4

A 'cookbook' aggregation

 initializing = false;
  // Run the aggregation initially to add some data to our aggregation collection
  runAggregation('added');
  // Wrap the aggregation call inside of a function
  // since it will be called more than once
  function runAggregation(action) {
    const results = Promise.await(Events.aggregate(pipeline).toArray());
    _.each(results, function (r) {

now produces this error :
I20220124-08:15:58.504(1)? Exception from sub eventstats.potential id 9e2bK8sB2LTKo2uSc MongoInvalidArgumentError: Argument "options" must not be function
I20220124-08:15:58.505(1)? at Collection.aggregate (/home/rdw/.meteor/packages/npm-mongo/.4.3.0-beta260.4.uv8jy2.o45h++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/mongodb/lib/collection.js:371:19)
I20220124-08:15:58.506(1)? at packages/meteor.js:311:21
I20220124-08:15:58.506(1)? at Collection.Mongo.Collection.aggregate (packages/meteorhacks_aggregate.js:39:46)
I20220124-08:15:58.506(1)? at runAggregation (server/eventstats.js:295:42)
I20220124-08:15:58.507(1)? at Subscription._handler (server/eventstats.js:291:3)
I20220124-08:15:58.507(1)? at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1885:12)
I20220124-08:15:58.507(1)? at packages/ddp-server/livedata_server.js:1107:9
I20220124-08:15:58.507(1)? at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1257:12)
I20220124-08:15:58.507(1)? at Subscription._runHandler (packages/ddp-server/livedata_server.js:1106:60)
I20220124-08:15:58.508(1)? at Session._startSubscription (packages/ddp-server/livedata_server.js:917:9)
I20220124-08:15:58.508(1)? at Session.sub (packages/ddp-server/livedata_server.js:673:12)
I20220124-08:15:58.508(1)? at packages/ddp-server/livedata_server.js:603:43

This example might help to make 2.6 a stable version soon.

regards
Ronny

[perbergland]

You are using meteorhacks_aggregate which needs to be updated for the new driver version (the collection.aggregate method is no longer asynchronous and no longer has a callback method).

It is that package that sends in a callback function as a second disallowed parameter to the collection.aggregate method on the rawCollection. Alternatively, you can remove the use of meteorhacks_aggregate and just invoke Promise.await(Events.rawCollection().aggregate(pipeline).toArray()) yourself or write a simple wrapper for this.

[polygonwood]

Hi Per Thanks for the swift response ! regards Ronny

…

On Mon, Jan 24, 2022 at 10:25 AM Per Bergland ***@***.***> wrote: You are using meteorhacks_aggregate which needs to be updated for the new driver version (the collection.aggregate method is no longer asynchronous and no longer has a callback method). It is that package that sends in a callback function as a second disallowed parameter to the collection.aggregate method on the rawCollection. Alternatively, you can remove the use of meteorhacks_aggregate and just invoke Promise.await(Events.rawCollection().aggregate(pipeline).toArray()) yourself or write a simple wrapper for this. — Reply to this email directly, view it on GitHub <#11749 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMTJC234EOTUXX5KHL77RC3UXULKJANCNFSM5HKJP7ZA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[polygonwood]

at startup of application, server side, these messages appear ... I20220124-20:50:02.970(1)? [FilesCollection.storagePath] Set to: assets/app/uploads/uploadedFiles D20220124-20:50:03.195(1) (collection.js:743) _ensureIndex has been deprecated, please use the new 'createIndex' instead, index: {"createdAt":1} ... I presume ostrio:files (the package) has to be adapted to get rid of the _ensureIndex (new warning issued by 2.6)

[afrokick]

@dr-dimitru FYI

[dkoo761]

Thanks everyone for working on this. I won't have time to upgrade and test both my apps before the mid-Feb cutoff so I'm going to upgrade to a Mongo Atlas M10 dedicated plan in order to continue to use Mongo 4.4 for now.

[chip]

@dkoo761 - Hello again. 😄

I've just upgraded to M10 and am getting errors from the console even though my user has many privileges for the admin database (e.g., atlasAdmin, backup, clusterMonitor, dbAdmin, dbAdminAnyDatabase, readAnyDatabase, readWrite, readWriteAnyDatabase.)

Atlas atlas-hu2kj6-shard-0 [primary] admin> db.adminCommand( { setFeatureCompatibilityVersion: "4.4" } )
MongoServerError: not authorized on admin to execute command { setFeatureCompatibilityVersion: "4.4", lsid: { id: UUID("4748e0c5-6963-42f7-b500-6e7400b90b5c") }, $clusterTime: { clusterTime: Timestamp(1644688103, 1), signature: { h
ash: BinData(0, 7F3DF7CA04D2088597B29752464D8DAD7DA24433), keyId: 7063867780415619076 } }, $db: "admin" }

I've heard that adding clusterManager would do the trick, but I cannot find it anywhere in the Atlas UI. Do you know what privileges are required? I'm totally stuck and am restricted from contacting Atlas support because we're not on a higher tier. Any suggestions are greatly appreciated. Thanks in advance.

[dkoo761]

@chip If your permissions are correct, then I'm guessing you would most likely get that error if you were still connecting to the older M0/M2/M5 cluster or the upgrade hasn't completed?

Are you sure the upgrade has completed and you're actually connecting to the M10 cluster? Is there a way in the shell to check to see if your cluster is running as M10?

If you run this from the shell: db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } ), what is the output? If it mentions 'targetVersion' then it's possible the upgrade isn't done yet as per the docs: https://docs.mongodb.com/manual/reference/command/setFeatureCompatibilityVersion/

Perhaps the connection string for M10 is different and you're using the old connection string? I would click the Connect button from your cluster page and go through the "Connect with MongoDB shell" to ensure you're using the correct connection string.

FYI - I'm not on M10 as I decided to self-host my databases, so you may have better luck asking on the Meteor forums if anyone else is having this issue with M10.

[chip]

@dkoo761 - Thanks for the prompt reply.

Before you posted, I discovered I was using the wrong connection string. The one provided by Atlas also didn't work until I removed the retryWrites portion.

Also was able to confirm the following in spite of not being able to run db.adminCommand( { setFeatureCompatibilityVersion: "4.4" } ). Here's the output of your suggestion:

Atlas atlas-hu2kj6-shard-0 [primary] admin> db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )
{
  featureCompatibilityVersion: { version: '4.4' },
  ok: 1,
  '$clusterTime': {
    clusterTime: Timestamp({ t: 1644696933, i: 1 }),
    signature: {
      hash: Binary(Buffer.from("c3e8efcc8e18dc2eb4079d259fe0e2953209ad1d", "hex"), 0),
      keyId: Long("7063867780415619076")
    }
  },
  operationTime: Timestamp({ t: 1644696933, i: 1 })
}

The app ultimately deployed and is running, but is EXTREMELY slow on reads. I may self-manage down the road as well.

Thanks again for your help. Very much appreciated!

[dkoo761]

Glad to hear you got it resolved quickly. Hmmm, that's strange about the slow reads. I would think, if anything, dedicated should be no slower than shared. Perhaps you can try the slow queries tool(s) in Atlas to see if that sheds any light on things? Otherwise I'm sure a support ticket will get to the bottom of it now that you're paying the big bucks :)

[chip]

Hehe - I know, right. Thanks for the query tool suggestion!

[renanccastro]

This has landed on 2.6. thanks everyone!

[filipenevola]

We've published [email protected].

If you run meteor update in your project using 2.6, you should be able to use this version of mongo package.

Fixes this issue.

[renanccastro]

We have released a fix for this issue on [email protected]

If you run meteor update in your project using 2.6, you should be able to use this version of the mongo package.

[ronylabib]

I updated to Meteor 2.7 and the problem went away but after the update, the mongo URL I was using "Sharded URL", is not working anymore.