MongoDB

TL;DR

What is MongoDB?

Open-source NoSQL database.

NoSQL: catch-all term for database that generally aren’t relational and don’t have a query language like SQL.
document-oriented great for unstructured data,especially when you have a lot of it.

MongoDB  comparison to SQL

SQL —  MongoDB

• database — database
• table — collection
• row — document

SQL is relational and mongodb is document-oriented

Relation vs Document-oriented

relational database management systems save data in rows within tables.
mongodb saves data as documents within collections.

感觉NoSQL大部分设计 也有可能是初级设计 几乎都接近一范式
比如SQL关联表 A和B
(A)potions name price vendor_id
(B)vendor_id name
在NoSQL里就会被设计成

 Potions collections
 name "Love"
 Price: 3.99
 Vendor: "Brewers"

Collections group Documents
Collections are simply groups of documents.since documents exist independently,they can have different fields.Potions can have different data! This is referred to as a “dynamic schema”.

下面是干货

How do we interact with mongodb?

All instances of mongodb come with a command line program we can use to interact with our database using JavaScript.

Regular Javascript variable assignment

var potion = {
  "name": "Invisibility",
  "vendor": "Kettlecooked"
}
//Access the variable to see the contents
potion
{
  "name": "Invisibility",
  "vendor": "Kettlecooked"
}

Inserting a Document into a Collection

Insert Document 没什么花头 公式简单

>db.potions.insert(
  {
    "name": "Invisibility",
    "vendor": "Kettlecooked"
  }
)
// or another ways
potion_one
{
  "name": "Invisibility",
  "vendor": "Kettlecooked"
}
db.potions.insert(potion_one)

So, The Potion document as a parameter of the insert method.

“_id” Unique id that gets automatically generated.

Finding all potions

db.options.find()
db.options.find().pretty() // use that is better!

Finding a Specific Potion with a Query.

带条件的查询

db.options.find({"name": "Invisibility"})

BSON

Documents are persisted in a format called BSON.
BSON is like JSON, so you can store:

	String "Invisibility"
	Numbers 3.14
	Booleans true false
	Arrays ["newt toes","pickles"]
	Objects {"type": "potion"}
	Null null

	ObjectID ObjectId(...)
	Date ISODate(...)

Learn more at http://go.codeschool.com/bson-spec

Add Date type

	{
	  "name": "Invisibility",
	  "tryDate": new Date(2012,8,13)
		//Reads as September 13,2012,since Javascript months begin at 0
	  //dates get converted to an ISO format when saved to the database
	  //JS 是从0开始计算的
	}

Add List into a collection

		//Array are a great option for storing lists of data
		{
		  "ingredients": ["newt toes",42,"laughter"]
		  //We can store any data type within an array
		}

Key words: any data

Embedded Documents

{
  "ratings": {"strength":2, "flavor":5}
}
//We can easily query embedded documents
db.potions.find({"ratings.flavor":5});

We embed documents simply by adding the document as a value for a given field .

Delete a single document

	db.potions.remove(
	  {"vendor": "Kettlecooked"}
	)
//Delete single or Multiple Documents

Update Documents

Field Update Operators

• $mul: Multiplies the value of the field by the specified amount.
• $inc: Increments the value of the field by the specified amount.
• $rename: Renames a field
• $setOnInsert:Sets the value of a field if an update results in an insert of a document. Has no effect on update operations that modify existing documents.
• $set: Sets the value of a field in a document.
• $unset: Removes the specified field from a document.
• $min: Only updates the field if the specified value is less than the existing field value.
• $max: Only updates the field if the specified value is greater than the existing field value.
• $currentDate: Sets the value of a field to current date, either as a Date or a Timestamp.

$set

//Updating a Document
db.potions.update(
	  {"name": "Love"}, // Query parameter
	  {"$set": {"price": 3.99}}//Update parameter
)
//"$set" or $set is ok

// updating Multiple Documents
db.potions.update(
  {"vendor": "KC"},
  {"$set": {"vendor": "Kettlecooked"}},
  {"multi": true} // Change all documents
)

常见套路

查询符合某一条件的documents 然后update为某一个数值

db.potions.update(
  {"vendor_id": null},
  {"$set": {"vendor_id": "Super_Vendor"}},
	{"multi": true}
)

$inc || Increments

	db.wands.update(
	  {"powers": "Fire"},
		//find all wands that include "Fire" in the powers
	  {$inc: {"level_required": 2}},
		// update that to increment the level_required by 2
	  {"multi": true}
		// The update will apply to all documents that match the query.
	)

$setOnInsert

	db.products.update(
	{"item": "PS4"},
	{
	  $set: {"maker":"Japan"},
	  $setOnInsert: {"defaultQty":100},
	  $inc: {"count":1}
	},
	{"upsert":true}
	)

defaultQty只有在这条result被insert的时候才会赋值，仅Update是不会有变化的。
另外注意一下第二个parameter是可以写很复杂的 所有的更新item都可以写在这里。

$min
如果$min字段的指定值小于字段的当前值，则将$min字段的值更新到document中
比如 count:10 , $min: {count : 5} 指定值小于字段值 将document中的count更新为5

> db.products.update( {"maker": "USA"}, {   $min:  {"count":6} }, {"multi": true} )

db.wands.update(
  {},
  {$mul: {"damage.melee":10}},
  {"multi": true}
)

Update or insert

Find or Create with Upsert
The upsert potion either updates an existing document or creats a new one,if the field doesn’t exist,it gets created with the value

	db.logs.update(
	  {"potion": "Love"},
	  {"$inc": {"count": 1}},
	  {"upsert": true} // update or insert
	)

Drop Column | Removing Fields From documents

We initially thought we’d need a potion’s color,but we never use it.
The $unset operator can be used to remove specified fields .

	db.options.update(
	  {"name": "Love"},//query for all potions
	  {"$unset": {"color": ""}},//"" the value we pass doesn't impact the operation
	  {"multi": true} // update all potions
	)
	// color field removed from documents

Demo || Drop a Column in collection

> db.log.find()
	{ "_id" : ObjectId("590156b2d507117ce9466c2b"), "potion" : "Shrinking", "count" : 3 }
	{ "_id" : ObjectId("590156ecd507117ce9466c3e"), "potion" : "BookShop", "count" : 1 }
> db.log.update(
  {},
	{"$unset": {"potion":""}},
	{"multi": true} // Removing fields from documents!!!
)
	WriteResult({ "nMatched" : 2, "nUpserted" : 0, "nModified" : 2 })
> db.log.find()
	{ "_id" : ObjectId("590156b2d507117ce9466c2b"), "count" : 3 }
	{ "_id" : ObjectId("590156ecd507117ce9466c3e"), "count" : 1 }

Rename Fields

db.potions.update(
	  {},
	  {"$rename": {"old_Field": "New_Field"}},
	  {"multi": true}
	)

Updating Single Array Value

db.potions.update(
	  {"name": "Shrinking"}, // A Query
	  {"$set": {"Array_instancs.1": 99}}
) // count start with 0 !!!

db.wands.update(
	  {"name": "Dream Bender"},
	  {$set: {"powers.0": "Fire Deflection" }}
)

Array的element的位置不是固定不变的
We need to change “secret” in multiple documents ,but the location isn’t always the same for every potion.

Updating values without knowing position

	db.wands.update(
	  {"powers": "Love"}, //First , we need searching the element
	  {$set: {"powers.$": "Love Burst"}},
	  {"multi": true}
	)

And Updating an Embedded Value

	db.potions.update(
	  {"name": "Shrinking"},
	  {"$set": {"ratings.strength": 5}}
	)

Useful Update Operators

MongoDB provides a variety of ways to modify the values of fields

• $max //updates if new value is greater than current or inserts if empty
• $min //Updates if new value is less than current or inserts if empty
• $mul //Multiple current field value by specified value. if empty , it inserts 0.

Read more https://docs.mongodb.com/manual/reference/operator/update/

$pop

Removing the first or Last Value of an Array

	db.potions.update(
	  {"name": "Shrinking"},
	  {"$pop": {"categories": 1}}
	)
	// -1 Removes the first element
	// 1 Removes the last element

$push

Adding Values to the end of an Array.

	db.potions.update(
	  {"name": "Shrinking"},
	  {"$push": {"categories": "budget"}}
	)
	//Added to the end

$addToSet

Adding Unique Values to an Array. 避免添加重复的element

	db.potions.update(
	  {"name": "Shrinking"},
	  {"$addToSet": {"categories": "budget"}}
	) // value already exists, so it doesn't get added again.

$pull

Removing Values From an Array.
If value isn’t unique, then all instances will be removed from the array.

	db.potions.update(
	  {"name":"Shrinking"},
	  {"$pull": {"categories": "tasty"}}
	)

多条件查询

Querying With Multiple Criteria

	db.potions.find(
	  {
	    "vendor": "Kettlecooked",
	    "ratings.strength": 5
	  }
	)

Based on conditions

• $gt //greater than
• $lt //less than
• $gte //greater than or equal to
• $lte // less than or equal to
• $ne // not equal to

Finding Potions that are less than $20

	db.potions.find(
	  {"price": {"$lt": 20}}
	)
	//less than or equal 5
	db.wands.find(
	 {"level_required": {$lte : 5}}
	)

Finding Potions Between prices

	db.potions.find({"price": {"$gt":10, "$lt": 20}})
	db.wands.find(
	  {"damage.melee": {$gte: 30, $lte: 40}}
	)

Queries of Non-equality lIKE sql

	db.potions.find({"vendor": {"$ne": "Brewers"}})

Range Queries on an Array

/*
Each potion has a size field that contains an array of available sizes.
We can use $elemMatch to make sure at least 1 element matches all criteria
*/
db.potiond.find({"sizes": {"$elemMatch": {"$gt":8, "$lt": 16}}})
//至少要有一个符合  大于8 小于16的标准
lengths = [2,3,5]
db.wands.find(
  {"lengths": {$elemMatch: {$gte :2, $lt:5}}}
).pretty()
// 只要有一个符合条件就可以

Customizing Queries

We’re putting together a list of the best potions we’ve used.
Let’s find potions with a grade equal to or greater than 80

Introducing Projections

find() takes a second parameter called a “projection” that we can use to
specify the exact fidlds we want back by setting their value to true .

find里面的第一个parameter是需要过滤的field 第二个parameter是一个投影 projection,
简单理解就是只映射想看的fields

	db.potions.find(
	  {"grade": {"$gte": 80}},
	  {"vendor": true, "name": true}
	)
// Just onlu show 2 fields
// Only retrieve what's needed

Excluding Fields

db/potions.find(
  {"grade": {"$gte": 80}},
  {"vendor": false, "price": false}
)

不能在同一行写 including and Excluding

	db.potions.find(
	  {"grade": {"$gte": 80}},
	  {"name": true , "vendor": false} -- Could be raised an Error
	)

	db.wands.find({},{"name": true, "powers": true, "_id": false})

The results of our query should show the name and powers for each of our wands,and exclude the _id values.

关于显示的问题
By default, the first 20 documents are printed out
type it , display the next 20 documents in the cursor

find()模式下的count

return the count of matching documents

db.wands.find({"level_requires":2}).count()

Sorting Potions

db.potions.find().sort({"price": 1})

Demo for find() & sort()

	db.wands.find(
	{}, // find all wands
	  {"name":true}
	).sort({"name":1})
// 1 to order ascending
// -1 to order descending

//another one
//First, let's write a query to match all wands.
//Next, add a cursor method to sort the wands in descending order by their price field
//Lastly, add another cursor method to limit the results to only 3 documents.
db.wands.find({}).sort({"price":-1}).limit(3)

分页设置 Paginating the Potions Page

这个分页说实话有点不实用，如果要用到的时候肯定需要DIY一些内容的

	// Basic Pagination
	// ABC DEF GHI
	db.potions.find().limit (3) // ABC skip 0 limit 3
	db.potions.find().skip(3).limit (3) // DEF skip 3 limit 3
	db.potions.find().skip(6).limit (3) // GHI skip 6 limit 3

Common Aggregate

常见的聚合函数的应用

$group
相当于SQL里面的 Distinct写法上注意 fields需要加 $的

db.potions.aggregate([
    {"$group": {"_id": "$vendor_id"}}
])

	{ "_id" : "KPL" }
	{ "_id" : "Kettlecooked" }
	{ "_id" : "Super_Vendor" }

The aggregate framework allows for advanced computations.
$group : stage poerator that’s used to group data by any field we specify
$vendor_id : Field names that begin with a “$” are called “field paths” and are links to a field a document.
“_id” : This is know as the “group key” and is required .

Distinct& Group By

db.potions.aggregate([
  {"$group":
    {"_id": "$vendor_id", "total": {"$sum": 1}}
	}
])

{ "_id" : "KPL", "total" : 1 }
{ "_id" : "Kettlecooked", "total" : 2 }
{ "_id" : "Super_Vendor", "total" : 5 }

$sum will add 1 for each matching document,Total number of documents per vendor

实战

Add an accumulator with the total_cost field that sums the price for each wand per maker.

db.wands.aggregate([
	  {"$group": {"_id": "$maker", "total_cost": {"$sum": "$price"}}}
])
// 以maker 分组 然后sum一下 price
	{ "_id" : null, "total_cost" : 77.7 }
	{ "_id" : "Olivemist", "total_cost" : 110.98 }
	{ "_id" : "Foxmond", "total_cost" : 344.84999999999997 }
	{ "_id" : "Sageseer", "total_cost" : 89.94 }
	{ "_id" : "Pinebriar", "total_cost" : 49.96 }
	{ "_id" : "Moonsap", "total_cost" : 215.93 }

进阶版 统计一下total的Vendor然后sum一下Grade

> db.potions.aggregate([
    { "$group":
		    {
            "_id": "$vendor_id",
            "total": {"$sum": 1},
            "grade_total": {"$sum": "$grade"}
		    }
	  }
])
 // sums the grade values for potions in their group
	{ "_id" : "KPL", "total" : 1, "grade_total" : 99 }
	{ "_id" : "Kettlecooked", "total" : 2, "grade_total" : 200 }
	{ "_id" : "Super_Vendor", "total" : 5, "grade_total" : 450 }

Average & Max or Min

db.potions.aggregate([
      {"$group":
		        {"_id": "$vendor_id","avg_grade": {"$avg": "$grade"}}
			}
])

db.potions.aggregate([
      {"$group":
		       {"_id": "$vendor_id",
						 "Max_grade": {"$max": "$grade"},
						 "Min_grade": {"$min": "$grade"},
						 "Avg_grade": {"$avg": "$grade"}
					 }
			}
])

The Aggregation Pipeline

Pipeline
管道的概念,管道在Unix和Linux中 一般用于将当前命令的输出结果作为下一个命令的参数。

MongoDB的聚合管道将MongoDB文档在一个管道处理完毕后将结果传递给下一个管道处理。管道操作是可以重复的。

表达式：处理输入文档并输出。表达式是无状态的，只能用于计算当前聚合管道的文档，不能处理其它的文档。

这里我们介绍一下聚合框架中常用的几个操作：

• $project：修改输入文档的结构。可以用来重命名、增加或删除域，也可以用于创建计算结果以及嵌套文档。
• $match：用于过滤数据，只输出符合条件的文档。$match使用MongoDB的标准查询操作。
• $limit：用来限制MongoDB聚合管道返回的文档数。
• $skip：在聚合管道中跳过指定数量的文档，并返回余下的文档。
• $unwind：将文档中的某一个数组类型字段拆分成多条，每条包含数组中的一个值。
• $group：将集合中的文档分组，可用于统计结果。
• $sort：将输入文档排序后输出。
• $geoNear：输出接近某一地理位置的有序文档。

$match

match符合条件下 按照maker分组 统计最小的level required

	db.wands.aggregate([
	  {$match: {"powers": "Air Bolt"}},
	  {$group: {
	    "_id":"$maker",
	    "lowest_level": {$min: "$level_required"}
	  }}
	])

一个实例

• 1) Write an aggregate to match wands that have a price that is less than 50.
• 2)Add the aggregate stage to group the wands by their maker.
• 3)Add an accumulator with a field named average_magic to find the average damage.magic per maker using the $avg accumulator.
• 4)Now that we’ve got our data all set, let’s filter it down further.Below the existing $match and $group aggregates,add another $match aggregate so that we’re only retrieving results with an average_magic that is greater than 40.

	db.wands.aggregate([
	  {$match: {"price": {$lt: 50}}},
	  {$group: {
	    "_id": "$maker",
	    "average_magic": {$avg: "$damage.magic"}
	  }},
	  {$match: {"average_magic" : {"$gt": 40}}}
	])

Top 3 Vendors with Potions Under $15 #Demo

	/*
	1) Query for potions with a price less than 15
	2) Group potions by vendor and average their grades
	3) Sort the results by grade average
	4) Limit result to only 3 vendors
	*/
	
	db.potions.aggregate([
	  {"$match": {"price": {"$lt": 15}}}, // 1)
	  //It's good practice to limit the number of results early on
	  {"$group": {"_id": "$vendor_id", "avg_grade": {"$avg": "$grade"}}},
	  // Vendor name as group key
	  // Average of potion grades
	  {"$sort": {"avg_grade": -1}}, // avg_grade: we can sort the field we created during the group stage
	  // sort by the field in descending order
	  {"$limit": 3}
	])
	
	{ "_id" : "Super_Vendor", "avg_grade" : 317.25 }
	{ "_id" : "Kettlecooked", "avg_grade" : 100 }
	{ "_id" : "KPL", "avg_grade" : 99 }

We can limit the fields we send over by using $project, which functions the sameway as Projections when we’re querying with find()
We want to us $project as soon as possible.
It’s common to see $match and $project used together early on and throughout the pipeline

Projections While Aggragating

$Project 的作用就是重构结果集显示的内容

	db.potions.aggregate([
	  {"$match": {"price": {"$lt": 15}}},
	  {"$project": {"_id": false, "vendor_id": true, "grade": true}}, //added
	  // vendor_id: Vewndor and grade for each potion after the match stage.
	  {"$group": {"_id": "$vendor_id", "avg_grade": {"$avg": "$grade"}}},
	  {"$sort": {"avg_grade": -1}},
	  {"$limit": 3}
	])

	// another one 
	db.wands.aggregate([
	  {$match: {"level_required": {"$lte":5}}},
	  {$project: {"_id":false, "maker":true, "damage.magic":true}},
	  {$group: {"_id": "$maker", "max_damage": {"$max":"$damage.magic"}}},
	  {$sort: {"max_damage":-1}},
	  {$limit:4}
	])

Write an aggregate that finds wands that have a level_required that’s less than or equal to 5.
Add the aggregate stage to group the wands by their maker.
Add an accumulator with the field max_damage that finds the max damage.magic per maker.
Now that we have the bulk of our data, let’s go ahead and sort the max_damage in descending order.
Add a limit stage so that we only find the first 4 results. After all, we don’t have all day to look through wands!
There’s one more stage we can add to our pipeline to make sure it’s fully optimized.
Since we only need the maker and damage.magic fields,
add a $project stage that only passes those fields along to the rest of operators.

Remember, where you place this is important!