An enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.
We’re always looking for people who value their work, so come and join us. We are hiring!
- yarn
- docker
- Babel 7
- Express
- Bunyan
Run the following script
./setup-shards/scripts/setup/base.sh
Take a look at this to create shards and replica sets.
Run the following command to begin seeding
node seeders/index.js
- cd `parcel-node-mongo-express`
- yarn
- ./setup-shards/scripts/setup/base.sh
- yarn start
- open browser to `localhost:9000` (port default to 9000)
When using NoSQLs you are optimising for read performance. We're doing this by denormalising data. There are multiple copies of the same data. For example
- Orders contains purchasedProducts which contains Products. Instead of referencing here we embed
- SupplierProducts contains embedded objects for both Suppliers and Products
- StoreProducts contains embedded objects for both Stores and Products
This makes our application write heavy. Every time there is a change to a product we need to make a change to
- SupplierProducts
- StoreProducts
- Products
Orders is not impacted since a change in the product after purchase will not affect the order.
However the application is able to perform extremely fast reads. 2 reasons for better performance is
- shards
- document embedding
NoSQLs are also good for handling large volumes of data. This is supported due to its ability to have shards. In this application we create 4 shards and the data is distributed amongst these shards.
These are the shard keys that we use
- Order
- _id
- name
- Products:
- Suppliers
- Stores
We got really good distribution across shards(24-26%) per shard after seeding 4 million records. It's possible to get a hot shard due to this but we're yet to see that.
- productId
- SupplierProducts
- StoreProducts
productId is chosen as the shard key since we anticipate that the queries for fetching all suppliers/stores that sell a particular product will be much more than fetching all products of a supplier/store.