Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
160 lines (124 loc) · 7.06 KB

CONTRIBUTING.md

File metadata and controls

160 lines (124 loc) · 7.06 KB

Contributing to UMD.io

Thank you for taking time to contribute! UMD.io is run and built by a community of volunteer developers, and its people like you who help make it possible.

Table of Contents

Getting Started

You should start by creating your own fork of UMD.io to work on locally. Then, check out some of these resources:

To submit your changes to this repository, submit a pull requests to the master branch. We'll give it a review, and once it looks good and no tests are failing, we will merge it into the main codebase.

Setup

Additionally, while not strictly required, it is a good idea to have Ruby installed locally. Although Ruby is not required to run the application because of the use of Docker containers, you will need it to run development tasks. We strongly recommend using RVM over installing Ruby directly.

If you choose to install Ruby, you can finish setting up your local environment by running

rvm use --install ruby-2.7.2 # Should be the same version used in Gemfile
gem install bundler   # Project-level gem management. https://bundler.io/
rvm docs generate     # Build docs for ruby stdlib and global gems
bundle exec yard gems # Build docs for local gems

Documentation

For the public-facing API, we use OpenAPI v3 to document everything. You can view our spec here. The docs are served with ReDoc and are automatically built on every tagged commit.

If you're actively working on the documentation, use the docker-compose-dev.yml file to view your changes live in ReDoc, on http://localhost:8080.

rake dev:up
# If rake throws a LoadError (e.g. "LoadError: cannot load such file -- rspec/core/rake_task"),
# try running the task below instead. You may need to run `bundle install` beforehand.
sudo -E rake dev:up

Development Tasks

We use Rake for running tasks. A comprehensive list of tasks can be found by running bundle exec rake -T.

Some of the more important, highly used tasks are

rake dev:up   # Launch the development environment with docker-compose
rake dev:down # Stops the development environment and removes its containers, networks, etc.
rake scrape   # Run all scrapers
rake validate # Type check and lint the codebase with solargraph and rubocop
rake rubocop:auto_correct # Run rubocop and apply all safe fixes

You can easily run rake tasks within the development environment with the umdio.sh script. All this script does is forward its arguments to rake running on the umdio container, making this script effectively a drop-in replacement for running rake.

Code Style

Within the codebase, comments and good practices are encouraged. We use Rubocop to enforce code style.

All rake tasks should have descriptions.

Tech Stack

umd.io runs on Ruby, with various libraries such as Rack, Sinatra, Puma, and Sequel. We use PostgreSQL as the database. Everything runs in Docker.

Adding New Data

If you're interested in adding a new endpoint, here's a rough guide on how to do it. Our data for majors is a great, simple example.

  1. Create a model in /app/models. We use Sequel on top of Postgres. It should include a to_v1 method that translates whatever is in your table into the object you want to return.
  2. Create a scraper in /app/scrapers. This is to populate the table for the model you just created.
    • If you're scraping a live webpage, courses_scraper.rb might be a good resource. We use nokogiri to parse HTML.
    • If you're parsing a JSON file, consider adding it to umdio-data, and creating an importer, such as map_scraper.rb. (NOTE: umdio-data is now included as a submodule; so this scraper should be updated)
  3. Create a controller in /app/controllers. Add endpoints as you see fit.
  4. Register the controller in server.rb.
  5. Write documentation in openapi.yaml.
  6. Add a test suite in the tests/ folder.

Logging

We use Ruby's built-in logger to output messages to standard output. Learn more about Ruby's logging module

Here's an example of output from the courses scraper:

[2018-10-18 01:35:01] INFO  (courses_scraper): Searching for courses in term 201801
[2018-10-18 01:35:02] INFO  (courses_scraper): 178 department/semesters so far
[2018-10-18 01:35:02] INFO  (courses_scraper): Searching for courses in term 201805
[2018-10-18 01:35:03] INFO  (courses_scraper): 301 department/semesters so far

The formatting for outputted messages is as follows:[DATE TIME] LOG_LEVEL (PROGRAM_NAME): {MESSAGE}

An example of a log call in ruby:

logger.info(prog_name) {"MESSAGE"}

You should use Ruby's built-in log-levels where appropriate, when displaying errors you should use logger.error, when displaying information you should use logger.info, and so on.

Our logger implementation is located at the scraper_common.rb file located at $app/scrapers/scraper_common.rb

Testing

We use RSpec to test. You can find the tests in the tests directory. Please make sure any new features you add have test cases to go along with them. Pull requests that do not include test cases may not be accepted.

To run the tests, run the following:

# Make sure the development environment is up
rake dev:up
# Populate the database with test data
./umdio.sh test_scrape
# Run all tests
./umdio.sh test
# Only run v1 test cases
./umdio.sh testv1