TL;DR
What is MongoDB?
Open-source NoSQL database.
NoSQL: catch-all term for database that generally aren’t relational and don’t have a query language like SQL.
document-oriented great for unstructured data,especially when you have a lot of it.
MongoDB comparison to SQL
SQL — MongoDB
- database — database
- table — collection
- row — document
SQL is relational and mongodb is document-oriented
Relation vs Document-oriented
relational database management systems save data in rows within tables.
mongodb saves data as documents within collections.
感觉NoSQL大部分设计 也有可能是初级设计 几乎都接近一范式
比如SQL关联表 A和B
(A)potions name price vendor_id
(B)vendor_id name
在NoSQL里就会被设计成
Potions collections name "Love" Price: 3.99 Vendor: "Brewers"
Collections group Documents
Collections are simply groups of documents.since documents exist independently,they can have different fields.Potions can have different data! This is referred to as a “dynamic schema”.
下面是干货
How do we interact with mongodb?
All instances of mongodb come with a command line program we can use to interact with our database using JavaScript.
Regular Javascript variable assignment
var potion = { "name": "Invisibility", "vendor": "Kettlecooked" } //Access the variable to see the contents potion { "name": "Invisibility", "vendor": "Kettlecooked" }
Inserting a Document into a Collection
Insert Document 没什么花头 公式简单
>db.potions.insert( { "name": "Invisibility", "vendor": "Kettlecooked" } ) // or another ways potion_one { "name": "Invisibility", "vendor": "Kettlecooked" } db.potions.insert(potion_one)
So, The Potion document as a parameter of the insert method.
“_id” Unique id that gets automatically generated.
Finding all potions
db.options.find() db.options.find().pretty() // use that is better!
Finding a Specific Potion with a Query.
带条件的查询
db.options.find({"name": "Invisibility"})
BSON
Documents are persisted in a format called BSON.
BSON is like JSON, so you can store:
String "Invisibility" Numbers 3.14 Booleans true false Arrays ["newt toes","pickles"] Objects {"type": "potion"} Null null ObjectID ObjectId(...) Date ISODate(...)
Learn more at http://go.codeschool.com/bson-spec
Add Date type
{ "name": "Invisibility", "tryDate": new Date(2012,8,13) //Reads as September 13,2012,since Javascript months begin at 0 //dates get converted to an ISO format when saved to the database //JS 是从0开始计算的 }
Add List into a collection
//Array are a great option for storing lists of data { "ingredients": ["newt toes",42,"laughter"] //We can store any data type within an array }
Key words: any data
Embedded Documents
{ "ratings": {"strength":2, "flavor":5} } //We can easily query embedded documents db.potions.find({"ratings.flavor":5});
We embed documents simply by adding the document as a value for a given field .
Delete a single document
db.potions.remove( {"vendor": "Kettlecooked"} ) //Delete single or Multiple Documents
Update Documents
Field Update Operators
- $mul: Multiplies the value of the field by the specified amount.
- $inc: Increments the value of the field by the specified amount.
- $rename: Renames a field
- $setOnInsert:Sets the value of a field if an update results in an insert of a document. Has no effect on update operations that modify existing documents.
- $set: Sets the value of a field in a document.
- $unset: Removes the specified field from a document.
- $min: Only updates the field if the specified value is less than the existing field value.
- $max: Only updates the field if the specified value is greater than the existing field value.
- $currentDate: Sets the value of a field to current date, either as a Date or a Timestamp.
$set
//Updating a Document db.potions.update( {"name": "Love"}, // Query parameter {"$set": {"price": 3.99}}//Update parameter ) //"$set" or $set is ok // updating Multiple Documents db.potions.update( {"vendor": "KC"}, {"$set": {"vendor": "Kettlecooked"}}, {"multi": true} // Change all documents )
常见套路
查询符合某一条件的documents 然后update为某一个数值
db.potions.update( {"vendor_id": null}, {"$set": {"vendor_id": "Super_Vendor"}}, {"multi": true} )
$inc || Increments
db.wands.update( {"powers": "Fire"}, //find all wands that include "Fire" in the powers {$inc: {"level_required": 2}}, // update that to increment the level_required by 2 {"multi": true} // The update will apply to all documents that match the query. )
$setOnInsert
db.products.update( {"item": "PS4"}, { $set: {"maker":"Japan"}, $setOnInsert: {"defaultQty":100}, $inc: {"count":1} }, {"upsert":true} )
defaultQty只有在这条result被insert的时候才会赋值,仅Update是不会有变化的。
另外注意一下第二个parameter是可以写很复杂的 所有的更新item都可以写在这里。
$min
如果$min字段的指定值小于字段的当前值,则将$min字段的值更新到document中
比如 count:10 , $min: {count : 5} 指定值小于字段值 将document中的count更新为5
> db.products.update( {"maker": "USA"}, { $min: {"count":6} }, {"multi": true} )
db.wands.update( {}, {$mul: {"damage.melee":10}}, {"multi": true} )
Update or insert
Find or Create with Upsert
The upsert potion either updates an existing document or creats a new one,if the field doesn’t exist,it gets created with the value
db.logs.update( {"potion": "Love"}, {"$inc": {"count": 1}}, {"upsert": true} // update or insert )
Drop Column | Removing Fields From documents
We initially thought we’d need a potion’s color,but we never use it.
The $unset operator can be used to remove specified fields .
db.options.update( {"name": "Love"},//query for all potions {"$unset": {"color": ""}},//"" the value we pass doesn't impact the operation {"multi": true} // update all potions ) // color field removed from documents
Demo || Drop a Column in collection
> db.log.find() { "_id" : ObjectId("590156b2d507117ce9466c2b"), "potion" : "Shrinking", "count" : 3 } { "_id" : ObjectId("590156ecd507117ce9466c3e"), "potion" : "BookShop", "count" : 1 } > db.log.update( {}, {"$unset": {"potion":""}}, {"multi": true} // Removing fields from documents!!! ) WriteResult({ "nMatched" : 2, "nUpserted" : 0, "nModified" : 2 }) > db.log.find() { "_id" : ObjectId("590156b2d507117ce9466c2b"), "count" : 3 } { "_id" : ObjectId("590156ecd507117ce9466c3e"), "count" : 1 }
Rename Fields
db.potions.update( {}, {"$rename": {"old_Field": "New_Field"}}, {"multi": true} )
Updating Single Array Value
db.potions.update( {"name": "Shrinking"}, // A Query {"$set": {"Array_instancs.1": 99}} ) // count start with 0 !!! db.wands.update( {"name": "Dream Bender"}, {$set: {"powers.0": "Fire Deflection" }} )
Array的element的位置不是固定不变的
We need to change “secret” in multiple documents ,but the location isn’t always the same for every potion.
Updating values without knowing position
db.wands.update( {"powers": "Love"}, //First , we need searching the element {$set: {"powers.$": "Love Burst"}}, {"multi": true} )
And Updating an Embedded Value
db.potions.update( {"name": "Shrinking"}, {"$set": {"ratings.strength": 5}} )
Useful Update Operators
MongoDB provides a variety of ways to modify the values of fields
- $max //updates if new value is greater than current or inserts if empty
- $min //Updates if new value is less than current or inserts if empty
- $mul //Multiple current field value by specified value. if empty , it inserts 0.
Read more https://docs.mongodb.com/manual/reference/operator/update/
$pop
Removing the first or Last Value of an Array
db.potions.update( {"name": "Shrinking"}, {"$pop": {"categories": 1}} ) // -1 Removes the first element // 1 Removes the last element
$push
Adding Values to the end of an Array.
db.potions.update( {"name": "Shrinking"}, {"$push": {"categories": "budget"}} ) //Added to the end
$addToSet
Adding Unique Values to an Array. 避免添加重复的element
db.potions.update( {"name": "Shrinking"}, {"$addToSet": {"categories": "budget"}} ) // value already exists, so it doesn't get added again.
$pull
Removing Values From an Array.
If value isn’t unique, then all instances will be removed from the array.
db.potions.update( {"name":"Shrinking"}, {"$pull": {"categories": "tasty"}} )
多条件查询
Querying With Multiple Criteria
db.potions.find( { "vendor": "Kettlecooked", "ratings.strength": 5 } )
Based on conditions
- $gt //greater than
- $lt //less than
- $gte //greater than or equal to
- $lte // less than or equal to
- $ne // not equal to
Finding Potions that are less than $20
db.potions.find( {"price": {"$lt": 20}} ) //less than or equal 5 db.wands.find( {"level_required": {$lte : 5}} )
Finding Potions Between prices
db.potions.find({"price": {"$gt":10, "$lt": 20}}) db.wands.find( {"damage.melee": {$gte: 30, $lte: 40}} )
Queries of Non-equality lIKE sql
db.potions.find({"vendor": {"$ne": "Brewers"}})
Range Queries on an Array
/* Each potion has a size field that contains an array of available sizes. We can use $elemMatch to make sure at least 1 element matches all criteria */ db.potiond.find({"sizes": {"$elemMatch": {"$gt":8, "$lt": 16}}}) //至少要有一个符合 大于8 小于16的标准 lengths = [2,3,5] db.wands.find( {"lengths": {$elemMatch: {$gte :2, $lt:5}}} ).pretty() // 只要有一个符合条件就可以
Customizing Queries
We’re putting together a list of the best potions we’ve used.
Let’s find potions with a grade equal to or greater than 80
Introducing Projections
find() takes a second parameter called a “projection” that we can use to
specify the exact fidlds we want back by setting their value to true .
find里面的第一个parameter是需要过滤的field 第二个parameter是一个投影 projection,
简单理解就是只映射想看的fields
db.potions.find( {"grade": {"$gte": 80}}, {"vendor": true, "name": true} ) // Just onlu show 2 fields // Only retrieve what's needed
Excluding Fields
db/potions.find( {"grade": {"$gte": 80}}, {"vendor": false, "price": false} )
不能在同一行写 including and Excluding
db.potions.find( {"grade": {"$gte": 80}}, {"name": true , "vendor": false} -- Could be raised an Error ) db.wands.find({},{"name": true, "powers": true, "_id": false})
The results of our query should show the name and powers for each of our wands,and exclude the _id values.
关于显示的问题
By default, the first 20 documents are printed out
type it , display the next 20 documents in the cursor
find()模式下的count
return the count of matching documents
db.wands.find({"level_requires":2}).count()
Sorting Potions
db.potions.find().sort({"price": 1})
Demo for find() & sort()
db.wands.find( {}, // find all wands {"name":true} ).sort({"name":1}) // 1 to order ascending // -1 to order descending //another one //First, let's write a query to match all wands. //Next, add a cursor method to sort the wands in descending order by their price field //Lastly, add another cursor method to limit the results to only 3 documents. db.wands.find({}).sort({"price":-1}).limit(3)
分页设置 Paginating the Potions Page
这个分页说实话有点不实用,如果要用到的时候肯定需要DIY一些内容的
// Basic Pagination // ABC DEF GHI db.potions.find().limit (3) // ABC skip 0 limit 3 db.potions.find().skip(3).limit (3) // DEF skip 3 limit 3 db.potions.find().skip(6).limit (3) // GHI skip 6 limit 3
Common Aggregate
常见的聚合函数的应用
$group
相当于SQL里面的 Distinct写法上注意 fields需要加 $的
db.potions.aggregate([ {"$group": {"_id": "$vendor_id"}} ]) { "_id" : "KPL" } { "_id" : "Kettlecooked" } { "_id" : "Super_Vendor" }
The aggregate framework allows for advanced computations.
$group : stage poerator that’s used to group data by any field we specify
$vendor_id : Field names that begin with a “$” are called “field paths” and are links to a field a document.
“_id” : This is know as the “group key” and is required .
Distinct& Group By
db.potions.aggregate([ {"$group": {"_id": "$vendor_id", "total": {"$sum": 1}} } ]) { "_id" : "KPL", "total" : 1 } { "_id" : "Kettlecooked", "total" : 2 } { "_id" : "Super_Vendor", "total" : 5 }
$sum will add 1 for each matching document,Total number of documents per vendor
实战
Add an accumulator with the total_cost field that sums the price for each wand per maker.
db.wands.aggregate([ {"$group": {"_id": "$maker", "total_cost": {"$sum": "$price"}}} ]) // 以maker 分组 然后sum一下 price { "_id" : null, "total_cost" : 77.7 } { "_id" : "Olivemist", "total_cost" : 110.98 } { "_id" : "Foxmond", "total_cost" : 344.84999999999997 } { "_id" : "Sageseer", "total_cost" : 89.94 } { "_id" : "Pinebriar", "total_cost" : 49.96 } { "_id" : "Moonsap", "total_cost" : 215.93 }
进阶版 统计一下total的Vendor然后sum一下Grade
> db.potions.aggregate([ { "$group": { "_id": "$vendor_id", "total": {"$sum": 1}, "grade_total": {"$sum": "$grade"} } } ]) // sums the grade values for potions in their group { "_id" : "KPL", "total" : 1, "grade_total" : 99 } { "_id" : "Kettlecooked", "total" : 2, "grade_total" : 200 } { "_id" : "Super_Vendor", "total" : 5, "grade_total" : 450 }
Average & Max or Min
db.potions.aggregate([ {"$group": {"_id": "$vendor_id","avg_grade": {"$avg": "$grade"}} } ]) db.potions.aggregate([ {"$group": {"_id": "$vendor_id", "Max_grade": {"$max": "$grade"}, "Min_grade": {"$min": "$grade"}, "Avg_grade": {"$avg": "$grade"} } } ])
The Aggregation Pipeline
Pipeline
管道的概念,管道在Unix和Linux中 一般用于将当前命令的输出结果作为下一个命令的参数。
MongoDB的聚合管道将MongoDB文档在一个管道处理完毕后将结果传递给下一个管道处理。管道操作是可以重复的。
表达式:处理输入文档并输出。表达式是无状态的,只能用于计算当前聚合管道的文档,不能处理其它的文档。
这里我们介绍一下聚合框架中常用的几个操作:
- $project:修改输入文档的结构。可以用来重命名、增加或删除域,也可以用于创建计算结果以及嵌套文档。
- $match:用于过滤数据,只输出符合条件的文档。$match使用MongoDB的标准查询操作。
- $limit:用来限制MongoDB聚合管道返回的文档数。
- $skip:在聚合管道中跳过指定数量的文档,并返回余下的文档。
- $unwind:将文档中的某一个数组类型字段拆分成多条,每条包含数组中的一个值。
- $group:将集合中的文档分组,可用于统计结果。
- $sort:将输入文档排序后输出。
- $geoNear:输出接近某一地理位置的有序文档。
$match
match符合条件下 按照maker分组 统计最小的level required
db.wands.aggregate([ {$match: {"powers": "Air Bolt"}}, {$group: { "_id":"$maker", "lowest_level": {$min: "$level_required"} }} ])
一个实例
- 1) Write an aggregate to match wands that have a price that is less than 50.
- 2)Add the aggregate stage to group the wands by their maker.
- 3)Add an accumulator with a field named average_magic to find the average damage.magic per maker using the $avg accumulator.
- 4)Now that we’ve got our data all set, let’s filter it down further.Below the existing $match and $group aggregates,add another $match aggregate so that we’re only retrieving results with an average_magic that is greater than 40.
db.wands.aggregate([ {$match: {"price": {$lt: 50}}}, {$group: { "_id": "$maker", "average_magic": {$avg: "$damage.magic"} }}, {$match: {"average_magic" : {"$gt": 40}}} ])
Top 3 Vendors with Potions Under $15 #Demo
/* 1) Query for potions with a price less than 15 2) Group potions by vendor and average their grades 3) Sort the results by grade average 4) Limit result to only 3 vendors */ db.potions.aggregate([ {"$match": {"price": {"$lt": 15}}}, // 1) //It's good practice to limit the number of results early on {"$group": {"_id": "$vendor_id", "avg_grade": {"$avg": "$grade"}}}, // Vendor name as group key // Average of potion grades {"$sort": {"avg_grade": -1}}, // avg_grade: we can sort the field we created during the group stage // sort by the field in descending order {"$limit": 3} ]) { "_id" : "Super_Vendor", "avg_grade" : 317.25 } { "_id" : "Kettlecooked", "avg_grade" : 100 } { "_id" : "KPL", "avg_grade" : 99 }
We can limit the fields we send over by using $project, which functions the sameway as Projections when we’re querying with find()
We want to us $project as soon as possible.
It’s common to see $match and $project used together early on and throughout the pipeline
Projections While Aggragating
$Project 的作用就是重构结果集显示的内容
db.potions.aggregate([ {"$match": {"price": {"$lt": 15}}}, {"$project": {"_id": false, "vendor_id": true, "grade": true}}, //added // vendor_id: Vewndor and grade for each potion after the match stage. {"$group": {"_id": "$vendor_id", "avg_grade": {"$avg": "$grade"}}}, {"$sort": {"avg_grade": -1}}, {"$limit": 3} ]) // another one db.wands.aggregate([ {$match: {"level_required": {"$lte":5}}}, {$project: {"_id":false, "maker":true, "damage.magic":true}}, {$group: {"_id": "$maker", "max_damage": {"$max":"$damage.magic"}}}, {$sort: {"max_damage":-1}}, {$limit:4} ])
Write an aggregate that finds wands that have a level_required that’s less than or equal to 5.
Add the aggregate stage to group the wands by their maker.
Add an accumulator with the field max_damage that finds the max damage.magic per maker.
Now that we have the bulk of our data, let’s go ahead and sort the max_damage in descending order.
Add a limit stage so that we only find the first 4 results. After all, we don’t have all day to look through wands!
There’s one more stage we can add to our pipeline to make sure it’s fully optimized.
Since we only need the maker and damage.magic fields,
add a $project stage that only passes those fields along to the rest of operators.
Remember, where you place this is important!