Monday, 14 August 2017

BIG DATA: MONGODB


Source: https://www.bing.com/search?q=what%20is%20mongo%20db&pc=cosp&ptag=C1AE89FD93123&form=CONBDF&conlogo=CT3210127 


MongoDB Inc. (Company)
  • MongoDB Inc. is an American software company that develops and provides commercial support for the open source database MongoDB, a NoSQL database that stores data in JSON-like documents with flexible schemas.

MongoDB

MongoDB (from humongous) is a cross-platform document-oriented database. Classified as a NoSQL database, MongoDB eschews the traditional table-based relational database structure in favor of JSON-like documents with dynamic schemas (MongoDB calls the format BSON), making the integration of data in certain types of applications easier and faster.
****
Source: https://en.wikipedia.org/wiki/MongoDB

JUDICIOUS TERMS OF MANGODB:

MongoDB (from humongous) is a free and open-source cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with schemas. MongoDB is developed by MongoDB Inc. and is free and open-source, published under a combination of the GNU Affero General Public License and the Apache License.

History

The software company then began developing MongoDB in 2007 as a component of a planned platform as a service product. In 2009, the company shifted to an open source development model, with the company offering commercial support and other services. In 2013, 10gen changed its name to MongoDB Inc.[6]

Ad hoc queries
MongoDB supports field, range queries, regular expression searches.[7] Queries can return specific fields of documents and also include user-defined JavaScript functions. Queries can also be configured to return a random sample of results of a given size.
Indexing
Fields in a MongoDB document can be indexed with primary and secondary indices.
Replication
MongoDB provides high availability with replica sets.[8] A replica set consists of two or more copies of the data. Each replica set member may act in the role of primary or secondary replica at any time. All writes and reads are done on the primary replica by default. Secondary replicas maintain a copy of the data of the primary using built-in replication. When a primary replica fails, the replica set automatically conducts an election process to determine which secondary should become the primary. Secondaries can optionally serve read operations, but that data is only eventually consistent by default.
Load balancing[9]
MongoDB scales horizontally using sharding. The user chooses a shard key, which determines how the data in a collection will be distributed. The data is split into ranges (based on the shard key) and distributed across multiple shards. (A shard is a master with one or more slaves.). Alternatively, the shard key can be hashed to map to a shard – enabling an even data distribution.
MongoDB can run over multiple servers, balancing the load or duplicating data to keep the system up and running in case of hardware failure.
File storage
MongoDB can be used as a file system with load balancing and data replication features over multiple machines for storing files.
This function, called Grid File System,[10] is included with MongoDB drivers. MongoDB exposes functions for file manipulation and content to developers. GridFS is used in plugins for NGINX[11] and lighttpd.[12] GridFS divides a file into parts, or chunks, and stores each of those chunks as a separate document.[13]
Aggregation
MapReduce can be used for batch processing of data and aggregation operations.
The aggregation framework enables users to obtain the kind of results for which the SQL GROUP BY clause is used. Aggregation operators can be strung together to form a pipeline – analogous to Unix pipes. The aggregation framework includes the $lookup operator which can join documents from multiple documents, as well as statistical operators such as standard deviation.
Server-side JavaScript execution
JavaScript can be used in queries, aggregation functions (such as MapReduce), and sent directly to the database to be executed.
Capped collections
MongoDB supports fixed-size collections called capped collections. This type of collection maintains insertion order and, once the specified size has been reached, behaves like a circular queue.

Bug reports and criticisms[edit]

Due to the default security configuration of MongoDB, allowing anyone to have full access to the database, data from tens of thousands of MongoDB installations has been stolen. Furthermore, many MongoDB servers have been held for ransom.[14][15]
In some failure scenarios where an application can access two distinct MongoDB processes, but these processes cannot access each other, it is possible for MongoDB to return stale reads. In this scenario it is also possible for MongoDB to roll back writes that have been acknowledged.[16]
Before version 2.2, concurrency control was implemented on a per-mongod[definition needed] basis. With version 2.2, concurrency control was implemented at the database level.[17] Since version 3.0,[18] pluggable storage engines were introduced, and each storage engine may implement concurrency control differently.[19] With MongoDB 3.0 concurrency control is implemented at the collection level for the MMAPv1 storage engine,[20] and at the document level with the WiredTiger storage engine.[21] With versions prior to 3.0, one approach to increase concurrency is to use sharding.[22] In some situations, reads and writes will yield their locks. If MongoDB predicts a page is unlikely to be in memory, operations will yield their lock while the pages load. The use of lock yielding expanded greatly in 2.2.[23]
Another criticism, from 2009, is related to the limitations of MongoDB when used on 32-bit systems.[24] In some cases, this was due to inherent memory limitations.[25][self-published source] MongoDB recommends 64-bit systems and that users provide sufficient RAM for their working set.
Up until version 3.3.11, MongoDB could not do collation-based sorting and was limited to byte-wise comparison via memcmp,[26] which would not provide correct ordering for many non-English languages when used with a Unicode encoding. The issue was fixed on August 23, 2016.
MongoDB queries against an index are not atomic and can miss documents which are being updated while the query is running and match the query both before and after an update.[27]

Architecture[edit]

Programming language accessibility[edit]

MongoDB has official drivers for major programming languages and development environments.[28] There are also a large number of unofficial or community-supported drivers for other programming languages and frameworks.

Management and graphical front-ends[edit]


Record insertion in MongoDB with Robomongo 0.8.5.
The primary interface to the database has been the mongo shell. Since MongoDB 3.2, MongoDB Compass is introduced as the native GUI. There are products and third-party projects that offer user interfaces for administration and data viewing. [29]

Licensing[edit]

MongoDB is available at no cost under the GNU Affero General Public License, version 3.[30] The language drivers are available under an Apache License. In addition, MongoDB Inc. offers proprietary licenses for MongoDB.

MongoDB World[edit]

MongoDB World[31] is an annual developer conference hosted by MongoDB, Inc.

See also[edit]

*** 

MongoDB Basics:  https://www.codeproject.com/articles/321774/mongodb-basics 



MongoDB Basics


, 30 Jan 2012
Rate this:
vote 1vote 2vote 3vote 4vote 5
This article demonstrates how to setup MongoDB running on your machine

Introduction

Some time ago, I heard about MongoDb and I started searching on search engines to get to know its scope. I found lot of supporting material in scattered form on various websites and books. Then, I thought of compiling atleast some basic getting started sort of tutorial for MongoDB so that the developers who are new to MongoDB or unfamiliar with it, can get to know what it is and how to initiate development with this new technology.
I must mention here that this article is much inspired by Karl Seguin's book on MongoDB. His blog can be found here.
This article demonstrates the introduction of MongoDB, no-SQL, the document-oriented database. The developers who are unfamiliar with no-SQL database, will wonder how it works. As a document-oriented database, MongoDB is a more generalized NoSQL solution. It should be viewed as an alternative to relational databases. Like relational databases, it too can benefit from being paired with some of the more specialized NoSQL solutions. Take it as simple as just any database, which stores data in some particular structure. Let's first setup your environment and enjoy the MongoDB power and start thinking of using it in your projects where you find it appropriate.

Setting Up the MongoDB

It's easy to set up and running with MongoDB.
  1. Go to the official download page and get the binaries of your choice.
  2. Extract the archive (anywhere you want) and navigate to the bin subfolder. Note that mongod is the server process and mongo is the client shell - these are the two executables we'll be spending most of our time with.
  3. Create a new text file in the bin subfolder named mongodb.config.
  4. Add a single line to your mongodb.config: dbpath=PATH_TO_WHERE_YOU_WANT_TO_STORE_YOUR_DATABASE_FILES. For example, on Windows, you might do dbpath=c:\mongodb\data.
  5. Make sure the dbpath you specified exists.
  6. Launch mongod with the --config /path/to/your/mongodb.config parameter. As an example, if you extracted the downloaded file to c:\mongodb\ and you created c:\mongodb\data\, then within c:\mongodb\bin\mongodb.config, you would specify dbpath=c:\mongodb\data\. You could then launch mongod from a command prompt via c:\ mongodb\bin\mongod --config c:\mongodb\bin\mongodb.config.
Hopefully you now have MonogDB up and running. If you get an error, read the output carefully - the server is quite good at explaining what happens wrong. You can now launch mongo which will connect a shell to your running server. Try entering db.version() to make sure everything's working as it should. Hopefully, you'll see the version number you installed. Go ahead and enter db.help(), you'll get a list of commands that you can execute against the db object.

Some Basic Concepts

Let's start our journey by getting to know the basic mechanics of working with MongoDB. Obviously, this is core to understanding MongoDB, but it will also help to give the idea about some higher level questions about where MongoDB fits.
To get started, there are six basic concepts we need to understand.
  1. MongoDB has the same concept of a 'database' with which you are likely already familiar. Within a MongoDB instance, you can have zero or more databases, each acting as high-level containers for everything else. You could think of it as simple database object in Microsoft SQL Server just for understanding the idea more clearly.
  2. A database can have zero or more 'collections'. A collection shares the same concept as a traditional `table', that you can think of the two as the same thing.
  3. Collections are made up of zero or more 'documents'. A document can be thought of as a 'row'.
  4. A document is made up of one or more 'fields', which you can guess, are like 'columns'.
  5. 'Indexes' in MongoDB function much like their RDBMS counterparts.
  6. 'Cursors' are different than the other five concepts. When you ask MongoDB for data, it returns a cursor, which you can do your processing, such as counting or skipping ahead, without actually pulling down data.
In summary, MongoDB is made up of databases which contain collections. A collection is made up of documents. Each document is made up of fields. Collections can be indexed, which improves lookup and sorting performance. Finally, when we get data from MongoDB, we do so through a cursor which is delayed to execute until necessary, might be called as lazy loading.
While these concepts are similar to their relational database counterparts, they are not identical. The core difference comes from the fact that relational databases define columns at the table level whereas a document-oriented database defines its fields at the document level. Each document within a collection can have its own unique set of fields. As such, a collection is a container in comparison to a table, while a document has a lot more information than a row.

Let's Start Playing with MongoDB

First, we'll use the global use method to switch databases, go ahead and enter use mycompany. It doesn't matter that the database doesn't really exist yet. The first collection that we create will also create the actual mycompanydatabase.
Now that you are inside a database, you can start issuing database commands, like db.getCollectionNames(). If you do so, you should get an empty array ([ ]). Since collections are schema-less, we don't explicitly need to create them. We can simply insert a document into a new collection.
To do so, use the insert command, supplying it with the document to insert:
db.departments.insert({name: 'Human Resource', city: 'karachi', head: 'Muhammad Ibrahim'})
The above line is executing insert against the departments collection, passing it a single argument. Internally MongoDB uses a binary serialized JSON format. Externally, this means that we use JSON a lot, as is the case with our parameters. If we execute db.getCollectionNames() now, we'll actually see two collections: departments and system.indexes. system.indexes is created once per database and contains the information on our databases index. You can now use the find command against departments to return a list of documents:
db.departments.find()
Notice that, in addition to the data you specified, there's an _id field. Every document must have a unique _idfield. You can either generate one yourself or let MongoDB generate an ObjectId for you. Most of the time you'll probably want to let MongoDB generate it for you. By default, the _id field is indexed - which explains why the system.indexes collection was created. You can look at system.indexes:
db.system.indexes.find()
What you're seeing is the name of the index, the database and collection it was created against and the fields included in the index.
Now, back to our discussion about schema-less collections. Insert a totally different document into departments, such as:
db.departments.insert({name: 'Development', country: 'Pakistan', _
departmentManager: 'Saeed Anwar', annualBudget: 5000000})
And, again use find to list the documents. Hopefully now you are starting to understand why the more traditional terminology wasn't a good fit.
There's one practical aspect of MongoDB you need to have a good grasp of before moving to more advanced topics: query selectors. A MongoDB query selector is like the where clause of an SQL statement. As such, you use it when finding, counting, updating and removing documents from collections. A selector is a JSON object , the simplest of which is {} which matches all documents (null works too). If we want all departments in Karachi city, we could use {city:'Karachi'}.
Before delving too deeply into selectors, let's set up some data to play with. Let insert some data in Employeescollection, remember although it does not already exist but when you go to insert in that collection, MondoDB will create that collection in the current database:
db.employees.insert({name: 'Amir Sohail', dob: new Date(1973,2,13,7,47), 
hobbies: ['cricket','reading'], city: 'Karachi', gender: 'm'});
db.employees.insert({name: 'Inzama-ul-Haq', dob: new Date(1977,2,13,7,47), 
hobbies: ['cricket','browsing'], city: 'Lahore', gender: 'm'});
db.employees.insert({name: 'Muhammad Yousuf', dob: new Date(1978, 0, 24, 13, 0), 
hobbies: ['football','chatting'], city: 'Karachi', gender: 'm'});
db.employees.insert({name: 'Muhammad Younis', dob: new Date(1982, 0, 24, 13, 0), 
hobbies: ['watching movies'], city: 'Peshawar', 
gender: 'm', department:'Human Resource'});
db.employees.insert({name: 'Shahid Afridi', dob: new Date(1983, 0, 24, 13, 0), 
hobbies: ['basketball','chatting'], city: 'Karachi', 
gender: 'm', department:'Development'});
db.employees.insert({name: 'Moin Khan', dob: new Date(1978, 0, 24, 13, 0), 
hobbies: ['cricket','chatting', 'browsing'], city: 'Islamabad', gender: 'm'});
db.employees.insert({name: 'Afra Kareem', dob: new Date(1993, 0, 24, 13, 0), 
hobbies: ['reading','browsing'], city: 'Karachi', gender: 'f', department:'Development'});
db.employees.insert({name: 'Asma Khan', dob: new Date(1985, 0, 24, 13, 0), 
hobbies: ['reading','watching movies'], city: 'Lahore', 
gender: 'f', department:'Human Resource'});
db.employees.insert({name: 'Nazia Malik', dob: new Date(1984, 0, 24, 13, 0), 
hobbies: ['reading'], city: 'Karachi', gender: 'f', department:'Development'});
db.employees.insert({firstName: 'Waqar', lastName: 'Younis', 
dob: new Date(1978, 0, 24, 13, 0), 
hobbies: ['cricket','chatting', 'basketball', 'browsing'], city: 'Karachi', gender: 'm'});
db.employees.insert({name: 'Waseem Akram', dob: new Date(1975, 0, 24, 13, 0), 
hobbies: ['cricket','chatting'], city: 'Rawalpindi', gender: 'm'});
db.employees.insert({name: 'Shoaib Akhtar', dob: new Date(1980, 0, 24, 13, 0), 
hobbies: ['football'], city: 'Rawalpindi', gender: 'm'});
db.employees.insert({name: 'Muhammad Amir', dob: new Date(1978, 0, 24, 13, 0), 
hobbies: ['bowling'], city: 'Karachi', gender: 'm'});
db.employees.insert({name: 'Saeed Ajmal', dob: new Date(1983, 0, 24, 13, 0), 
hobbies: ['spin bowling'], city: 'Karachi', gender: 'm'});
db.employees.insert({name: 'Abdur Rehman', dob: new Date(1982, 0, 24, 13, 0), 
hobbies: ['bowling'], city: 'Lahore', gender: 'm'});
db.employees.insert({name: 'Muhammad Mushtaq', dob: new Date(1972, 0, 24, 13, 0), 
hobbies: ['cricket','chatting'], city: 'Lahore', gender: 'm'});
db.employees.insert({firstName: 'Saqlain', lastName: 'Mushtaq', 
dob: new Date(1978, 0, 24, 13, 0), 
hobbies: ['football','chatting'], city: 'Karachi', 
gender: 'm', department:'Development'});
Now that we have data, we can master selectors. {field: value} is used to find any documents where field is equal to value. {field1: value1, field2: value2} is how we do an and statement. The special $lt, $lte, $gt, $gte and $ne are used for less than, less than or equal, greater than, greater than or equal and not equal operations. For example, to get all male employees that have city Karachi, we could do:
db.employees.find({gender: 'm', city: 'Karachi'})
The $exists operator is used for matching the presence or absence of a field, for example:
db.employees.find({firstName: {$exists: false}})
should return a single document. If we want to OR rather than AND, we use the $or operator and assign it to an array of values we want or'd:
db.employees.find({gender: 'f', $or: [{hobbies: 'reading'}, {hobbies: 'browsing'}, {
city: 'Karachi'}]})
The above will return all female employees which either have hobbies reading or browsing or city is Karachi. There's something pretty neat going on in our last example. You might have already noticed, but the loves field is an array. MongoDB supports arrays as first class objects. This is an incredibly handy feature. Once you start using it, you wonder how you ever lived without it. What's more interesting is how easy selecting based on an array value is: {hobbies: ' cricket'} will return any document where cricket is a value of hobbies. There are more available operators than what we've seen so far. The most flexible being $where which lets us supply JavaScript to execute on the server. These are all described in the Advanced Queries section of the MongoDB website. What we've covered so far though is the basics you'll need to get started. It's also what you'll end up using most of the time. We've seen how these selectors can be used with the find command. They can also be used with the remove command which we've briefly looked at, the count command, which we haven't looked at but you can probably figure out. The ObjectId which MongoDB generated for our _id field can be selected like so:
db.employees.find({_id: ObjectId("TheObjectId")})
We have remove() command for deletion purpose. To delete all records, simply you could call it on the required collection.
db.employees.remove()
Or alternatively, you could place the desired query selectors to delete only the selective documents.

Points of Interest

We did get MongoDB up and running, looked briefly at the insert and remove commands. We also introduced find and saw what MongoDB selectors were all about. We've had a good start and laid a solid foundation for things to come. Believe it or not, you actually know most of what there is to know about MongoDB - it really is meant to be quick to learn and easy to use. Insert different documents, possibly in new collections, and get familiar with different selectors. Use find, count and remove. After a few tries on your own, things that might have seemed awkward at first will hopefully fall into place.
Hopefully, I am planning to write another article to use MongoDB with C#.NET environment. I appreciate your feedback/comments or any improvements you want to suggest in this regard, to help in making the article much better and helpful for others.

History

  • 30th January, 2012: Initial post

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

*** 

Source: https://www.mongodb.com/what-is-mongodb

Here we are connecting to a locally hosted MongoDB database called test with a collection named restaurants.

 # 1. Connect to MongoDB instance running on localhost client = pymongo.MongoClient() # Access the 'restaurants' collection in the 'test' database collection = client.test.restaurants
import pprint import pymongo def main(): # 1. Connect to MongoDB instance running on localhost client = pymongo.MongoClient() # Access the 'restaurants' collection in the 'test' database collection = client.test.restaurants # 2. Insert new_documents = [ {"name":"Sun Bakery Trattoria", "stars":4, "categories":["Pizza","Pasta","Italian","Coffee","Sandwiches"]}, {"name":"Blue Bagels Grill", "stars":3, "categories":["Bagels","Cookies","Sandwiches"]}, {"name":"Hot Bakery Cafe","stars":4,"categories":["Bakery","Cafe","Coffee","Dessert"]}, {"name":"XYZ Coffee Bar","stars":5,"categories":["Coffee","Cafe","Bakery","Chocolates"]}, {"name":"456 Cookies Shop","stars":4,"categories":["Bakery","Cookies","Cake","Coffee"]}] collection.insert_many(new_documents) # 3. Query for restaurant in collection.find(): pprint.pprint(restaurant) # 4. Create Index collection.create_index([('name', pymongo.ASCENDING)])

Need to run MongoDB?

We want to...


  •  High availability through built-in replication and failover
  •  Horizontal scalability with native sharding
  •  End-to-end security
  •  Native document validation and schema exploration with Compass
  •  Always available global support
  •  Management tooling for automation, monitoring, and backup
  •  Fully elastic database as a service with built-in best practices
  • **** 
  • Source: https://www.lynda.com/Moodle-tutorials/Learning-MongoDB/573253-2.html 


  



About this video

Get started with MongoDB. Learn about the differences between relational and document databases, as well as how to work with the Mongo shell, build an application in Node.js, and more.

·         CourseOverview

·         Transcript

·         View Offline

·         Exercise Files

·         Code Practice
- [Kirsten] Hi. I'm Kirsten Hunter. I'm an API evangelist, and I am committed to making sure that developers have a fantastic experienceworking with any type of technology. I love Mongo because it makes it super easy to bring up an application stack and get to the coding. In this course, I'm going to give you an overview of the differences between relational and document databases, and then we'll dive right in to using Mongo. Exercises in the shell, and then Node.js will give you a strong foundation to move forward and use Mongo for your projects.Mongo is a fantastic tool for your toolbelt.
Being able to prototype and iterate quickly is incredibly useful. I'm excited to give you the knowledge you need to work with Mongo.
***

MongoDB Training and Tutorials

https://www.lynda.com/MongoDB-training-tutorials/1475-0.html?utm_source=bing&utm_medium=cpc&utm_campaign=l1-US-Search-Dev-MongoDB&cid=l1-us:en:ps:lp:prosc:s50:1475:all:bing:mbm-mongodb_book&utm_content=%7Bcreative%7D&utm_term=%2BMongoDB%20%2BBook&src=pa-bi&veh=skwd-26622768171_pcrid_6785166425_pkw_%2BMongoDB%20%2BBook_pmt_bb_pdv_c_agid_1666935206_cmid_72542874_adp_%7Badposition%7D_net_o_qs_mongodb%20book%20pdf&lpk35=9137

**** 

 


Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)