M101J: MongoDB for Java Developers Homework 4.2

M101J: MongoDB for Java Developers Homework 4.2
Question:
Suppose you have a collection called tweets whose documents contain information about the created_at time of the tweet and the user's followers_count at the time they issued the tweet. What can you infer from the following explain output?

db.tweets.find({"user.followers_count":{$gt:1000}}).sort({"created_at" : 1 })

.limit(10).skip(5000).explain()
{
        "cursor" : "BtreeCursor created_at_-1 reverse",
        "isMultiKey" : false,
        "n" : 10,
        "nscannedObjects" : 46462,
        "nscanned" : 46462,
        "nscannedObjectsAllPlans" : 49763,
        "nscannedAllPlans" : 49763,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 205,
        "indexBounds" : {
                "created_at" : [
                        [
                                {
                                        "$minElement" : 1
                                },
                                {
                                        "$maxElement" : 1
                                }
                        ]
                ]
        },
        "server" : "localhost.localdomain:27017"
}

Option:

1) This query performs a collection scan.  

True, it perform a collection scan.

2)The query uses an index to determine the order in which to return result documents. 

True, it uses an index to determine result

3)The query uses an index to determine which documents match.

false, for matches there is no index, index are use only for ordering.

4) The query returns 46462 documents. 

false, You can see the query it has limit to 10 , so it return only 10 result.

5)The query visits 46462 documents. 

true,  yes it has scanned 46462 document.

6)The query is a "covered index query". 

false, query is colvered index query.

So answer for week 4 - Homework 4.2 are 

M101J: MongoDB for Java Developers Homework 4.1

M101J: MongoDB for Java Developers Homework 4.1

This HomeWork is related to find which query will be using index to get result.

you can use query to find list of index


db.system.indexes.find()

Document:

{
  "v" : 1,
  "key" : {
   "sku" : 1
  },
                "unique" : true,
  "ns" : "store.products",
  "name" : "sku_1"
 }

Key Indicates there exist an index sku on collection products with index name as

 sku_1.[at the time of index creation if we give {sku:-1} name of index will be

 sku_-1]

db.collectionname.getIndexes() can be used to get index in individual collection

Now lets complete the homework.

Which of the following queries can utilize an index. Check all that apply.

1) db.products.find({'brand':"GE"}) 

if you check in document , there is no index with brand, you will find brand

index as multi index with category. but in query we are only using brand so above

query cannot utilize and index.

2)db.products.find({'brand':"GE"}).sort({price:1}) 

if you check in document, you will see brand cannot use an index, but in document

if you see price is present as single index. so the above query use an index.

3)db.products.find({$and:[{price:{$gt:30}},{price:{$lt:50}}]}).sort({brand:1}) 

 price is present, therefore query is using index.

4) db.products.find({brand:'GE'}).sort({category:1, brand:-1}).explain() 

 category and brand are present in document as multi - index  but if you check

index is created with {category:1, brand:1} but while using in query it is using 

{category:1, brand:-1}, so for sorting mongo will perform complete scan and

 it will not use index . if index was created with {category:1, brand:-1} then

the above query will be using an index.

So your homework 4.1 answer is 

M101J: MongoDB for Java Developers - Week 4

M101J: MongoDB for Java Developers - Week 4

Mongodb week 4 is related with peformance, you will learn
1) what is index in mongodb and how will you  create it?
2) Identify slow query.
3) Overview of Sharding.

M101J: MongoDB for Java Developers homework 3.3

Homework 3:
Problem you should be able to insert comment in the blog post.

Solution:
You dont have to do anything. changes were made in homework 3.2 so click turn in button in mongoproc and submit.

so these complete your homework for week 3

M101J: MongoDB for Java Developers Homework 3.2

Step 1:
Download Handout

Step 2:
Problem:
we need to enhance the project and we should be able to insert the entries in post collection. changes that we need to made are marked as XXX in BlogPostDAO
.

Solution:

open BlogPostDAO.java

1) Go to findByPermalink method

add below line at XXX

post = postsCollection.findOne(new BasicDBObject("permalink", permalink));

 2) Go to findByDateDescending method

add below line at XXX

DBCursor cursor = postsCollection.find()
 .sort(new BasicDBObject().append("date", -1)).limit(limit); 

    //DBCursor cursor1 = postsCollection.find();
        posts = new LinkedList<DBObject>();
        for (DBObject value : cursor) {
            posts.add(value);
        }
        System.out.println("Value from the DB" + posts);

3) Go to addPost method

add below Line at XXX

Date now = new Date();
  List comments = new ArrayList<Object>();
  post.append("author", username).append("title", title)
    .append("body", body).append("permalink", permalink)
    .append("tags", tags).append("date", now)
    .append("comments", comments);
  postsCollection.insert(post);

4) Go to addPostComment method

add below Line at XXX
  BasicDBObject comment = new BasicDBObject().append("author", name)
    .append("body", body);
  if (email != null) {
   comment.append("email", email);
  }
  postsCollection.update(new BasicDBObject("permalink", permalink),
    new BasicDBObject("$push", new BasicDBObject("comments",
      comment)), true, false);

you require only above changes.

now re compile using mvn compile exec:java -Dexec.mainClass=course.BlogController

or run the run.sh file

login to http://localhost:8082/login

you should be able to insert post in the blog.

MongoProc 

1) now go to mongoproc and login with course ID 

2)go to your homework and Click on Test.

3)if you receive below feedback then everything is correct. you should click on turn in

and you score will be submitted. 

Feedback Welcome. enjoy you homework.

 
 

M101J: MongoDB for Java Developers Homework 3.1

Step 1:
Download students.json from download handout

Step 2:
Import the same using mongo import
mongoimport -d school -c students < students.json

Step 3:
Write a program in your language to delete that will remove the lowest homework score for each student.

Step 4:
Cross check your data

Check 1:
Go to mongo shell
type
> use school
> db.students.count();
200


Check 2:
> db.students.find({_id:100}).pretty();
{
        "_id" : 100,
        "name" : "Demarcus Audette",
        "scores" : [
                {
                        "type" : "exam",
                        "score" : 47.42608580155614
                },
                {
                        "type" : "quiz",
                        "score" : 44.83416623719906
                },
                {
                        "type" : "homework",
                        "score" : 19.85604968544429
                },
                {
                        "type" : "homework",
                        "score" : 39.01726616178844
                }
        ]
}


Check 3:
Aggregation Query for student with highest average

db.students.aggregate([{'$unwind':'$scores'},{'$group':{'_id':'$_id','average':{$avg:'$scores.score'}}},{'$sort':{'average':-1}},{'$limit':1}])


Next homework will do it tomorrow.

M101J: MongoDB for Java Developers Homework 2.3

Homework 2.3 (MongoProc)

1) Step 1:

 you need to setup mongoproc.

Please read Instrcution for mongoproc setup.

2) Step 2:

Download  hw2.3.zip from download handout link

3) Step 3:

you can start script

run.sh

and visit the link http://localhost:8082 

4) Problem:

Problem is when you create a user, you will be redirected to welcome page

and when you logout you will not be able to login again.

goal of this homework is to complete the code in UserDAO.java file.

wherever there is xxx in code we will add java statement.

5) Solution:

open UserDAO.java

in addUser method

add following code in first // XXX WORK HERE

BasicDBObject user = new BasicDBObject("_id", username).append(
"password", passwordHash);

then add in 2nd // XXX WORK HERE

user.append("email", email);

then add in 3rd //XXX work here

usersCollection.insert(user); 

then in validateLogin method

add in 1st //XXX work here

user = usersCollection.findOne(new BasicDBObject("_id", username));

that it. you have change the code, now re - run run.sh

now you should be able to relogin in page. if it is successful

open mongoproc

click on homework 2.3(mongoproc). click on test. you will get feedback 

if it is sucessfull. click on turn in. your homework is submitted.

Enjoy :)

Any feedback welcome.

 

 

 

 

M101J: MongoDB for Java Developers : Homework 2.2

Homework 2.2
Write a program in the language of your choice that will remove the grade of type "homework" with the lowest score for each student from the dataset that you imported in HW 2.1. Since each document is one grade, it should remove one document per student.

Solution:
the question say you need to write  your program to remove document of grade homework and lowest score for each student.

i will use java to solve above homework.

you have also got hint
Hint/spoiler: If you select homework grade-documents, sort by student and then by score, you can iterate through and find the lowest score for each student by noticing a change in student id. As you notice that change of student_id, remove the document.


First let us understand what we have to remove
1) hints say you need to find document with {"type":"homework"}  and then you need to sort by student_id in ascending order {"student_id":1} and then by score in ascending order {"score":1}

so your final query is
db.grades.find({"type":"homework"}).sort({"student_id":1},{"score":1})

so you will get below output  after executing above query



now the question say you need to remove each one document per student having lowest score and told us track on student_id. in the above result it is clear we need to remove first document for each student.

now let write a java program :

HomeWork.java

import java.net.UnknownHostException;

import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBCursor;
import com.mongodb.MongoClient;

public class HomeWork {
    public static void main(String[] args) throws UnknownHostException {
        MongoClient client = new MongoClient("127.0.0.1");
        DB db = client.getDB("students");
        DBCollection collection = db.getCollection("grades");
        BasicDBObject searchQuery = new BasicDBObject();
        BasicDBObject sortQuery = new BasicDBObject();
        searchQuery.put("type", "homework");
        sortQuery.put("student_id", 1);
        sortQuery.put("score", 1);
        DBCursor cursor = collection.find(searchQuery).sort(sortQuery);
        System.out.println(cursor.count());
        BasicDBObject tempDocument = null;
        int i = 0;
        while (cursor.hasNext()) {
            BasicDBObject document = (BasicDBObject) cursor.next();
            System.out.println(document);
            if (tempDocument != null) {
                if (!document.get("student_id").equals(
                        tempDocument.get("student_id"))) {
                    collection.remove(document);
                    i++;
                }
            } else {
                collection.remove(document);
                i++;

            }
            tempDocument = document;

        }
        System.out.println(i);
    }
}


this will remove first document of all student.






you can cross check by

1) db.grades.count()  - 600

2) db.grades.find().sort({'score':-1}).skip(100).limit(1)
{ "_id" : ObjectId("50906d7fa3c412bb040eb709"), "student_id" : 100, "type" : "homework", "score" : 88.50425479139126 } 

3)db.grades.find({},{'student_id':1, 'type':1, 'score':1, '_id':0}).sort({'student_id':1, 'score':1, }).limit(5)
{ "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }
{ "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
{ "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }
{ "student_id" : 1, "type" : "homework", "score" : 44.31667452616328 }
{ "student_id" : 1, "type" : "exam", "score" : 74.20010837299897 }

if all 3 give you correct result then 

execute below query

db.grades.aggregate({'$group':{'_id':'$student_id', 'average':{$avg:'$score'}}}, {'$sort':{'average':-1}}, {'$limit':1})

you will get below result

{
        "result" : [
                {
                        "_id" : 54,
                        "average" : 96.19488173037341
                }
        ],
        "ok" : 1
}

So answer for homework 2.2 is 54