Aggregation filter after $lookup

Asked
Active3 hr before
Viewed126 times

8 Answers

aggregationfilter
90%

The question here is actually about something different and does not need $lookup at all. But for anyone arriving here purely from the title of "filtering after $lookup" then these are the techniques for you:,So the only real issue here is "filtering" any null result from the array, created when the current document was the parent in processing items to $push.,For releases of MongoDB 3.6 and onwards, then the more expressive "sub-pipeline" is generally what you want to "filter" the results of the foreign collection before anything gets returned into the array at all.,How can I add a filter after an $lookup or is there any other method to do this?

MongoDB 3.6 - Sub-pipeline

db.test.aggregate([{
      "$match": {
         "id": 100
      }
   },
   {
      "$lookup": {
         "from": "test",
         "let": {
            "id": "$id"
         },
         "pipeline": [{
            "$match": {
               "value": "1",
               "$expr": {
                  "$in": ["$$id", "$contain"]
               }
            }
         }],
         "as": "childs"
      }
   }
])
load more v
88%

Specifies the collection in the same database to perform the join operation. The joined collection cannot be sharded (see Sharded Collection Restrictions).,Performs a left outer join to an unsharded collection in the same database to filter in documents from the "joined" collection for processing. To each input document, the $lookup stage adds a new array field whose elements are the matching documents from the "joined" collection. The $lookup stage passes these reshaped documents to the next stage.,Specifies the collection in the same database to perform the join with. The from collection cannot be sharded. For details, see Sharded Collection Restrictions.,The following operation first uses the $lookup stage to join the two collections by the item fields and then uses $mergeObjects in the $replaceRoot to merge the joined documents from items and orders:

.leafygreen-ui-iokrs{color:inherit;font-size:13px;font-family:'Source Code Pro',Menlo,monospace;line-height:24px;}.leafygreen-ui-1v41da1{border-spacing:0;width:100%;}.leafygreen-ui-7razhx{border-spacing:0;vertical-align:top;padding:0 16px;}{   $lookup:     {       from: <collection to join>,       localField: <field from the input documents>,       foreignField: <field from the documents of the "from" collection>,       as: <output array field>     }}
load more v
72%

How can I add a filter after an $lookup or is there any other method to do this?,The question here is actually about something different and does not need $lookup at all. But for anyone arriving here purely from the title of "filtering after $lookup" then these are the techniques for you:,Therefore you must "de-normalise" the content first prior to performing the $lookup operation in order for this to work. And that means using $unwind:,So looking at this with a fresh mind the answer is staring me in the face. The key thing that you have already stated is that you want to find the "intersection" of two queries in a single response.

My data collection test is:

{
   "_id": ObjectId("570557d4094a4514fc1291d6"),
   "id": 100,
   "value": "0",
   "contain": []
} {
   "_id": ObjectId("570557d4094a4514fc1291d7"),
   "id": 110,
   "value": "1",
   "contain": [100]
} {
   "_id": ObjectId("570557d4094a4514fc1291d8"),
   "id": 120,
   "value": "1",
   "contain": [100]
} {
   "_id": ObjectId("570557d4094a4514fc1291d9"),
   "id": 121,
   "value": "2",
   "contain": [100, 120]
}

I select id 100 and aggregate the childs:

db.test.aggregate([{
   $match: {
      id: 100
   }
}, {
   $lookup: {
      from: "test",
      localField: "id",
      foreignField: "contain",
      as: "childs"
   }
}]);

I get back:

{
   "_id": ObjectId("570557d4094a4514fc1291d6"),
   "id": 100,
   "value": "0",
   "contain": [],
   "childs": [{
         "_id": ObjectId("570557d4094a4514fc1291d7"),
         "id": 110,
         "value": "1",
         "contain": [100]
      },
      {
         "_id": ObjectId("570557d4094a4514fc1291d8"),
         "id": 120,
         "value": "1",
         "contain": [100]
      },
      {
         "_id": ObjectId("570557d4094a4514fc1291d9"),
         "id": 121,
         "value": "2",
         "contain": [100, 120]
      }
   ]
}

At the end I expect this result:

{
   "_id": ObjectId("570557d4094a4514fc1291d6"),
   "id": 100,
   "value": "0",
   "contain": [],
   "childs": [{
         "_id": ObjectId("570557d4094a4514fc1291d7"),
         "id": 110,
         "value": "1",
         "contain": [100]
      },
      {
         "_id": ObjectId("570557d4094a4514fc1291d8"),
         "id": 120,
         "value": "1",
         "contain": [100]
      }
   ]
}
load more v
65%

A collection orders contains the following documents:,Consider a collection orders with the following document:,To each input document, the $lookup stage adds a new array field whose elements are the matching documents from the “joined” collection. The $lookup stage passes these reshaped documents to the next stage.,The operation returns the following document:

{
   $lookup:
     {
       from: <collection to join>,
       localField: <field from the input documents>,
       foreignField: <field from the documents of the "from" collection>,
       as: <output array field>
     }
}
load more v
75%

The $match stage allows us to choose just those documents from a collection that we want to work with. It does this by filtering out those that do not follow our requirements.,In the following example, we only want to work with those documents which specify that Spain is the value of the field country, and Salamanca is the value of the field city.,In this example, we want to know the number of documents per university in our ‘universities’ collection:,In this example, we only need the fields country, city and name.

This is an example of how we build the aggregation query:

pipeline = [{
         $match: {
            …},
         {
            $group: {
               …},
            {
               $sort: {
                  …},
               ...
            ]
            db.collectionName.aggregate(pipeline, options)
load more v
40%

db.getCollection('widgets').aggregate([{
      $unwind: '$info'
   },
   {
      $lookup: {
         from: 'icons',
         localField: 'info.iconId',
         foreignField: '_id',
         as: 'info.icon'
      }
   },
   {
      $unwind: '$info.icon'
   },
   {
      $group: {
         _id: '$_id',
         root: {
            $mergeObjects: '$$ROOT'
         },
         info: {
            $push: '$info'
         }
      }
   },
   {
      $replaceRoot: {
         newRoot: {
            $mergeObjects: ['$root', '$$ROOT']
         }
      }
   },
   {
      $project: {
         root: 0
      }
   }
]);
22%

How can I perform this filter with a join operation in MongoDB?,Making statements based on opinion; back them up with references or personal experience., AWS will soon be sponsoring Database Administrators , Soil Modelling: Getting accurate Thermal Conductivity values

A solution was less obvious than I expected since your source lookup field is an array, but the following should return the expected outcome:

db.collection1.aggregate([{
      $match: {
         _id: "col1id1"
      }
   },

   // Filtered lookup based on agg pipeline
   {
      $lookup: {
         from: "collection2",
         // let references fields in the original collection
         let: {
            property: "$properties"
         },
         pipeline: [{
               $match:
               // use $expr to compare fields using let variables
               {
                  $expr: {
                     $and: [{
                           $in: ["$_id", "$$property"]
                        },
                        {
                           $eq: ["$boolean_field", false]
                        }
                     ]
                  }
               }
            },
            {
               $project: {
                  "_id": 0,
                  "name": 1
               }
            }
         ],
         as: "property_names"
      }
   },

   // Project desired output fields 
   {
      $project: {
         "name": "$name",
         "property_names": {
            $map: {
               input: "$property_names",
               in: "$$this.name"
            }
         }
      }
   }
]).pretty()
load more v
60%

So as you can see the aggregate query is much advanced than the find query and can perform data manipulation operations on the filtered data as well, unlike the find query where one is limited to only data retrieval.,We will be discussing all the various operators available in the aggregation query. Below is the diagram of the simple database on which we will perform all the queries,,In every aggregation query, the pipeline stated appears in the form of an array, in which all the operations are placed sequentially. These are called Aggregation Pipeline Stages, in this article, we will be referring to those as stages.,In this article, we will learn about some of the most important and used aggregation techniques in MongoDB. You can bookmark this article and it can be very handy whenever you need some assistance.

Let’s take an example of a find query,

Collection.find(query, projection); // query - query for filtering out the data// projection - Specifies the fields to return in the documents that match the query filter
load more v

Other "aggregation-filter" queries related to "Aggregation filter after $lookup"