Problem writing tests using mocha with vogels

[helloravi]

Given that there is no delete_all option, I am not able to empty the table when I write tests. How are you guys handling tests when you use dynamodb?

[set-killer]

Option 1:
Create the tables at the beginning of the tests and delete the whole table at the end the tests.
If the table is already created it wouldn't overwrite it so nothing will happen, not even exception (which is good).

Also use the create tables action in every start of the application. Since they are already created it wouldn't do anything.

Option 2:
Track the hash keys when you create them, and delete them afterwards.

Option 3:
Use delete expression and delete the newest records (if you have timestamps on your records).

[helloravi]

Thank you very much for the alternatives. Can you elaborate on Option 3? I have timestamps enabled. How do I delete the newest records?

[set-killer]

At the beginning of the tests get the current time as integer like

var startTime = Date.now();

Then run your tests where you put your createdAt timestamps (integer again).
After the tests are compleated run a delete expression where createdAt > startTime.

---

But if you are creating a REST API and you are testing the create and update methods why are you not testing the delete methods? The best options is after you create some items, keep track of their IDs and use the same IDs for the update and delete methods.

[helloravi]

I think I am missing something very fundamental. I tried creating deleting and creating table outside. It worked. The part under beforeEach(for deleting and recreating) is not working though it does not show any error.

Here is the code that worked:

const vogels = require('vogels');
if (process.env.NODE_ENV === "test" || process.env.NODE_ENV === "development") {
    vogels.AWS.config.update({ accessKeyId: "myKeyId", secretAccessKey: "secretKey", region: "us-west-2" });
    vogels.dynamoDriver(new vogels.AWS.DynamoDB({ endpoint: 'http://localhost:8000' }));
} else {
    vogels.AWS.config.update({accessKeyId: process.env.AWS_ACCESS_KEY, secretAccessKey: process.env.AWS_SECRET_KEY, region: process.env.AWS_REGION});
}

function wait(ms){
    var start = new Date().getTime();
    var end = start;
    while(end < start + ms) {
        end = new Date().getTime();
    }
}


const DeliveryOrder = require("../api/models/DeliveryOrder");

vogels.createTables({
}, function(err) {
    if (err) {
        console.log('Error creating tables: ', err);
    } else {
        console.log('First creation');
        DeliveryOrder.deleteTable(function(err) {
            if (err) {
                console.log('Error deleting table: ', err);
            } else {
                console.log('Table has been deleted');
                vogels.createTables({
                }, function(err) {
                    if (err) {
                        console.log('Error creating tables: ', err);
                    } else {
                        console.log('Tables have been created');
                    }
                });
            }
        });
    }
});

I tried doing the very same in mocha. it does not run or throw any error(I know this because none of the messages in console log are printed). I was under the impression that mocha should be able to run any js code as long as I require the necessary files. Here is my test file:

process.env.NODE_ENV = 'test'; //Just being safe. 
let vogels = require('vogels');
vogels.AWS.config.update({ accessKeyId: "myKeyId", secretAccessKey: "secretKey", region: "us-west-2" });
vogels.dynamoDriver(new vogels.AWS.DynamoDB({ endpoint: 'http://localhost:8001' }));

//Require the dev-dependencies
let chai = require('chai');
let chaiHttp = require('chai-http');
let server = require('../server');
let should = chai.should();
const DeliveryOrder = require("../api/models/DeliveryOrder");
const _      = require('lodash');
chai.use(chaiHttp);


describe('DeliveryOrders', () => {

    function wait(ms){
        var start = new Date().getTime();
        var end = start;
        while(end < start + ms) {
            end = new Date().getTime();
        }
    }
    beforeEach(() => { //Before each test we empty the database
        console.log("in before ............")
        vogels.createTables({
        }, function(err) {
            if (err) {
                console.log('Error creating tables: ', err);
            } else {
                console.log('First creation');
                DeliveryOrder.deleteTable(function(err) {
                    if (err) {
                        console.log('Error deleting table: ', err);
                    } else {
                        console.log('Table has been deleted');
                        vogels.createTables({
                        }, function(err) {
                            if (err) {
                                console.log('Error creating tables: ', err);
                            } else {
                                console.log('Tables have been created');
                            }
                        });
                    }
                });
            }
        });

        console.log("in after ............")
    });


    /*
     * Test the /GET route
     */

    describe('POST DO', () => {
        it('should throw an error on create do with empty data', (done) => {
            let doData = {

            }
            chai.request(server)
                .post('/api/dos')
                .send(doData)
                .end((err, res) => {
                    res.should.have.status(400);
                    res.body.should.have.property('error');
                    // res.body.error.should.have.property('message').eql('LR Number Missing');
                    done();  
                })
        })



        it('should create a delivery order', (done) => {
            let doData = {
                "deliveryOrderNumber" : "03.05.2017",
                "companyId" : "BMM",
                "locationId" : "TADIPATRI",
                "branch" : "KPCL",
                "createdDate" : "2017",
                "vesselName" : "MV ANANGEL VICTORY",
                "cargo" : "coal",
                "cargoType" : "steamCoal",
                "cargoGrade" : "SOUTH AFRICA",
                "partyName" : "PENNA CEMENTS LTD",
                "subPartyName" : "PENNA CEMENTS LTD",
                "totalIssuedQuantity" : "5000",
                "transinIssuedQuantity" : "1000",
                "companyInchargeName" : "RAMESH BABU",
                "companyInchargePhoneNumber" : "8096696663.0",
                "transinInchargeName" : "RAMESH REDDY",
                "transinInchargePhoneNumber" : "9666777636.0",
                "tolerance" : "1",
                "billOfEntry" : {
                    "billEntryNumber" : "9106716/31.03.2017",
                    "customHouseAgent" : "NL",
                    "portOfLoading" : "RICHARDS BAY",
                    "countryOfOrigin" : "SOUTH AFRICA",
                    "grossWeight" : "5000",
                    "seller" : "SWISS SINGAPORE OVERSEAS ENTERPRISES",
                    "netValuePrice" : "3365568"
                },
                "completedQuantity" : "999.21",
                "material" : "coal-steamCoal-SOUTH AFRICA",
                "status" : "inactive",
                "deliveryOrderType" : "has_a_tdo",
                "deliveryOrderTypeId": ["TDO001"]

            }

            chai.request(server)
                .post('/api/dos')
                .send(doData)
                .end((err, res) => {
                    // console.log("Post response", res.body);
                    res.should.have.status(200);
                    res.body.should.have.property('data');
                    done();  
                })
            // DeliveryOrder.destroy('03.05.2017', function (err) {
            //       if (err) {
            //         console.log('Error deleting item: ', err);
            //       } else {
            //         console.log('Item has been deleted');
            //       }
            // });

            // DeliveryOrder.deleteTable(function(err) {
            //   if (err) {
            //     console.log('Error deleting table: ', err);
            //   } else {
            //     console.log('Table has been deleted');
            //   }
            // });
            //  wait(2000);

        })

    });


    describe('/GET delivery orders by DO Number', () => {
        it('it should GET all the delivery orders', (done) => {
            chai.request(server)
                .get('/api/do/03.05.20178')
                .end((err, res) => {
                    console.log("GET by DO number response", res.body);
                    res.should.have.status(200);
                    res.body.should.be.a('object');
                    // res.body.data.length.should.be.eql(0);
                    done();
                });
        });
    });


    describe('/GET delivery orders by Location Id', () => {
        it('it should GET all the delivery orders', (done) => {
            chai.request(server)
                .get('/api/do/location/tadipatri')
                .end((err, res) => {
                    console.log("GET by Location Id response", res.body);
                    res.should.have.status(200);
                    res.body.should.be.a('object');
                    // res.body.data.length.should.be.eql(0);
                    done();
                });
        });
    });


    describe('/GET delivery orders by Branch Id', () => {
        it('it should GET all the delivery orders', (done) => {
            chai.request(server)
                .get('/api/do/branch/kpcl')
                .end((err, res) => {
                    res.should.have.status(200);
                    res.body.should.be.a('object');
                    // res.body.data.length.should.be.eql(0);
                    done();
                });
        });
    });

});

[helloravi]

I have no clue but the vogels commands are not working properly in the test files when run using mocha.

[set-killer]

Yes, you are creating an API. In that case you should not put vogels commands in your tests. You are writting test for your API methods. And when you are deleting and items you should rely on your delete API method.

Basically do not put vogels commands in your tests...

[helloravi]

Firstly, thank you helping me out so much!

Can you please elaborate on that? I was a rails developer and I never had problems writing tests in this way. I am not able to understand what restricts us from using Vogels commands in test files. There is something strange about vogels commands in test files. Sometimes they work and sometimes they dont work. For instance, I have seen that get command works well but destroy command does not work.

[set-killer]

It is just the way that javascript and mocha works.
Javascript runs all I/O operaions asynchronously, which means it does not waits for the API call to return the data [blocking the thread], but instead it executes a given callback when the data arrives and during the waiting time it executes some other tasks. That is why NodeJS performs so well with multiple requests.

Mocha executes a given API call, after that it validates the data, http status, content type, etc., and then terminates the execution of the test, without waiting of the vogels operations to finish.

[helloravi]

I have seen people writing the way I wrote using mongoose without any problems. For instance, the following worked without a problem. Here is the code https://pastebin.com/vjVXkE9C

here is the tutorial. https://scotch.io/tutorials/test-a-node-restful-api-with-mocha-and-chai

[set-killer]

Did you spot that done callback in the beforeEach? It is there for a reason...

Also did that tutorial mentions if those tests are running async? If so you are risking to delete that table in the middle of other test.

[helloravi]

@set-killer - Thank you very much. i am able to destroy-create the table before every test now :).

In the very first reply you mentioned "run a delete expression where createdAt > startTime." In all the delete expressions I see in the examples there is a necessity to mention the hashKey to delete(https://github.com/ryanfitz/vogels#deleting). How will I be able to delete all items where createdAt > startTime without mentioning the hashkey?

[set-killer]

Create query like this or a scan like this.
And delete them one by one with some library like Async JS.

OR

Keep tracking of the IDs of the items that you create and insert and after that delete them. Just put them in some kind of array and after that delete them one by one.

The batchDelete functionality is not yet implemented. Keep traking of issue #85.
Case closed.