-
Notifications
You must be signed in to change notification settings - Fork 0
Product Guide: Developer
The "developer guide" portion should describe the internal structure of your system in intermediate depth, as though for a professional engineer taking over your project. Help them to understand how the system is put together, in an orderly and compact description, without getting mired in lowest-level details. The document should provide a good working description of the system and its implementation, but not a line-by-line walkthrough of the code. This also is the place where you acknowledge the software systems that you used but did not create.
TigerTrade is implemented as a single page application with standard frontend and backend connection over a RESTful API.
We assume that you have installed and configured Git, Go 1.8 or later, and Yarn, and have access to our repository. (If you are running an Ubuntu-derived OS that uses Go 1.6 or older, you can install Go 1.8 by following the instructions on the Ubuntu wiki.) Additionally, make sure you have set the environment variable $GOPATH to a reasonable value. We recommend export GOPATH=~/go. We make no guarantees about operating systems besides macOS and Linux, as our team exclusively used those operating systems.
The following commands worked to set up on a fresh install of Ubuntu 14.04:
sudo apt install git npm curl
sudo apt-add-repository ppa:longsleep/golang-backports
curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt update
sudo apt install yarn golang-go nodejs
echo 'export GOPATH=~/go' > ~/.bashrc
source ~/.bashrcThen to install, run:
$ go get github.com/casey-chow/tigertrade # this downloads the code into the correct directory
$ cd $GOPATH/src/github.com/casey-chow/tigertrade
$ make dev # prepare a development environment
$ make install # install all dependencies
$ cp .env.example .env; nano .env # set up environment for running serverFrom here, make sure that you have a proper .env file, either by copying and configuring .env.example or by adding one given to you. Then, in one terminal, run:
$ make serve # this starts an auto-reloading development api serverand in another terminal, run:
$ make serve-client # this starts an auto-reloading frontend serverFrom there, your browser should automatically open the main page for development.
The following commands can be run from the project root, and should prove useful to a developer:
Server
make install Install all dependencies
make build Builds the server
make docs Starts a server for docs
make serve Runs a hot-reloading server for development
make test Runs the test suite once
make test-watch Runs a pretty testing server for the API code
Client
yarn start Runs an auto-reloading dev server for the frontend
yarn build Builds and bundles the client code
yarn test Runs the test suite for the client
Both
make dev Creates a development environment
make clean Removes all temporary files
make purge Uninstalls all dependencies, removes temp files
We found in practice that one of the most effective ways of reducing code written and app configuration is to smartly employ external hosted services.
Heroku: This is used to actually host the application (both the frontend and the backend). Our Heroku configuration is set up with a staging and production app for both our staging and production servers (the staging server, with dummy data, is available at https://stage.tigertra.de). Per industry practice, secret information used is stored as environment variables in the platform. Heroku also provides us with managed Postgres databases, of which we have one for staging, and one for production.
Cloudflare: We use Cloudflare for speed optimizations and DNS--this allows us to perform over-the-wire compression, asset caching, and content distribution. Cloudflare is set to reset all its caches whenever we deploy a new version of the site to production (see "Building and Deploying").
Amazon S3: Amazon's Simple Storage Service provides a simple way to store user-uploaded images. For security and cost-cutting, we set the service to delete images after a year, which is generally more than enough time for the listing the image is attached to to be sold. Storage is incredibly cheap nowadays, and because of resizing we expect very little cost from this platform.
Smartlook: To understand usage, we knew that Google Analytics woud largely be useless for such a small user base, so we instead used Smartlook, which allows us to get a screencast of every user's usage of our site.
Sentry: Sentry is an excellent error handling tool, which is very useful for catching problems in production. We use it to track errors that may occur while running the app.
SendGrid: We use SendGrid to send our emails. Each of the emails we send through the site are done using Transactional Templates sent through the backend.
Our database consists of the following tables:
TK PERRY
The project repository itself is architected with the following folder structure:
client/src/ client application code
server/ server code
hooks/ useful development hooks, installed using make dev
node_modules/ Javascript dependencies
vendor/ Go dependencies
We found that we are able to keep both client and server in the same repository, and thus sync any changes to them in revision control, by deploying to Heroku with different configurations for frontend and backend servers--essentially, the frontend server will ignore backend code and vice versa, allowing both of them to play together nicely.
We aim for Security by Simplicity--that is, taking simple approaches to development that make it as obvious as possible whether we have security issues. Here, we detail our defenses against common attacks on web applications.
Cross-Site Scripting: Our architecture does no server-side HTML rendering, and since React doesn't ever parse the data it receives as HTML, our site is inherently XSS-resistant as long as everything we do is rendered using React (which we believe it is).
Cross-Site Request Forgery: We prevent CSRF attacks using the Origin and referrer headers, which both easy to implement in a RESTful API and effective against these attacks. This check is implemented as a middleware in our server and so is taken care of automatically for any request the server receives. Put simply, the middleware checks whether the origin header is valid, then if the referrer header is valid, and rejects the request with a 403 Forbidden if both are invalid. (It does allow the request if there are no such headers, however. This runs against the OWASP guidelines but is useful for debugging, and at if a user's browser can be hijacked such that it wouldn't send the mandatory Origin and Referer headers, which cannot be modified by Javascript, period, we assume that the user's system is too compromised for any server-side logic to filter out invalid requests without CAPTCHAs or risk analysis.)
SQL Injection: We prevent SQL injection by using prepared statements in our SQL queries. This is handled automatically by our SQL generation library, Squirrel. At no point do we string concatenate or interpolate SQL off of user-provided values, and so there should be no vulnerability (at least in our code) to SQL injection.
Resource Overload: This is not an issue for client-side since we don't pay for the client resources and a single client being slow doesn't affect other users. On the server, since Go is an inherently fast language, and Cloudflare offers DoS protection, attempting to perform any abuse of our site to take it down would likely fail. Further, users can only add content once authenticated and their identities on the site are linked with real identities, disincentivizing such attacks.
Finally, rather than trying to secure the system against abusive use for images, we decided to set up our storage to log who uploads what image and delete images after a year. We have notifications set up if the amount stored exceeds a certain threshold, and can restrict photo uploads from there. We also reduce photo usage by resizing and compressing all photos. Additionally, we validate the filetypes of the uploaded images by magic number rather than by extension, and modify the images before storing them, so sending poisoned photos would be very difficult.
We expose a simple RESTful API for our frontend to consume. Importantly, our frontend and backend servers are separate--the frontend server simply sends bundled frontend code, while the backend server simply serves an API. This allows us to use CAS, which is whitelisted for *.herokuapp.com, without exposing an ugly Heroku subdomain as our site URL, and simplifies our backend.
This, however, means that we need to enable cross origin resource sharing, a technology used to work around the same origin policy. The URL for the server and client to communicate with one another are encoded in environment variables in their Heroku configurations, so that the client knows where to send its requests, and the server knows what client origins to expect.