diff --git a/README.md b/README.md index 41605fb..8a22ac6 100644 --- a/README.md +++ b/README.md @@ -22,10 +22,13 @@ and changes. - **license_dbs** table has list of licenses and all the data related to the licenses. - **obligations** table has the list of obligations that are related to the licenses. +- **obligation_classifications** table has the list of classifications that are related to the obligations. +- **obligation_types** table has the list of types that are related to the obligations. - **obligation_maps** table that maps obligations to their respective licenses. - **users** table has the user that are associated with the licenses. - **audits** table has the data of audits that are done in obligations or licenses - **change_logs** table has all the change history of a particular audit. +- **oidc_clients** table has all the oidc clients used for communication with fossology. ![ER Diagram](./docs/assets/licensedb_erd.png) @@ -57,192 +60,57 @@ As the response of the request, a JWT will be returned. Use this JWT with the `Authorization` header (as `-H "Authorization: "`) to access endpoints requiring authentication. +## Prerequisites -## Prerequisite +Before setting up the project, ensure the following tools are installed on your system: -Before proceeding, make sure you have the following installed on your system: +### 1. Golang (Go) +You’ll need Go installed to build and run the project. +👉 [Official installation guide](https://go.dev/doc/install) +### 2. PostgreSQL (v14 or later) +The project uses PostgreSQL as its database. You can install it via: -- Install Golang - -Follow the official instructions to install Golang: -👉 [https://go.dev/doc/install](https://go.dev/doc/install) - ---- - -- Install golang-migrate CLI (For Linux & MacOs) +#### Option A: Package Manager (Linux example) ```bash -curl -L https://github.com/golang-migrate/migrate/releases/latest/download/migrate.linux-amd64.tar.gz | tar xvz -sudo mv migrate /usr/local/bin/ -``` - -## How to run this project? - -- Clone this Project and Navigate to the folder. - -``` bash -git clone https://github.com/fossology/LicenseDb.git -cd LicenseDb +sudo apt update +sudo apt install postgresql ``` -- Create the `external_ref_fields.yaml` file in the root directory of the project and change the - values of the extra license json keys as per your requirement. - -```bash -cp external_ref_fields.example.yaml external_ref_fields.yaml -vim external_ref_fields.yaml -``` +#### Option B: Official Installer -- Generate Go struct for the extra fields listed in the external_ref_fields.yaml. +Download and run the official installer for your operating system from the PostgreSQL website. +👉 [https://www.postgresql.org/download/](https://www.postgresql.org/download/) -```bash -go generate ./... -``` +### 3. Install golang-migrate CLI -- Build the project using following command. +#### For Linux & macOS ```bash -go build ./cmd/laas -``` - -- Create the `.env` file in the root directory of the project and change the - values of the environment variables as per your requirement. - -```bash -cp configs/.env.dev.example .env -vim .env -``` - -- Run the migration files. -```bash -migrate -path pkg/db/migrations -database "postgres://fossy:fossy@localhost:5432/licensedb?sslmode=disable" up -``` - -- Run the executable. - -```bash -./laas -``` - -- You can directly run it by the following command. - -```bash -go run ./cmd/laas -``` - -### Create first user -Connect to the database using `psql` with the following command. -```bash -psql -h localhost -p 5432 -U fossy -d licensedb -``` - -Run the following query to create the first user. -```sql -INSERT INTO users (user_name, user_password, user_level, display_name, user_email) VALUES ('', '', 'SUPER_ADMIN', '', ''); -``` -## LicenseDB with Docker - - -LicenseDB is available as a prebuilt Docker image on [GitHub Container Registry (GHCR)](https://ghcr.io/fossology/licensedb). -You can run it either using a simple `docker run` command or with the included `docker-compose.yml` file for a full setup with PostgreSQL. - ---- - -### Option 1: Run with Docker directly - -```bash -# Pull the latest stable release -docker pull ghcr.io/fossology/licensedb:latest - -# Run LicenseDB (requires a PostgreSQL instance) -docker run \ - --name licensedb \ - -p 8080:8080 \ - -e DB_HOST= \ - -e DB_PORT=5432 \ - -e DB_USER=fossy \ - -e DB_PASSWORD=fossy \ - -e DB_NAME=licensedb \ - ghcr.io/fossology/licensedb:latest +curl -L https://github.com/golang-migrate/migrate/releases/latest/download/migrate.linux-amd64.tar.gz | tar xvz +sudo mv migrate /usr/local/bin/ ``` -> ⚠️ This method requires you to have PostgreSQL running separately. -> If you’d like an **all-in-one setup** with PostgreSQL included, use the Docker Compose option below. +For other platforms and installation methods, check the official docs: +👉 [https://github.com/golang-migrate/migrate](https://github.com/golang-migrate/migrate) ---- +### 4. Install swagger document generator -### Option 2: Run with Docker Compose (recommended) - -A `docker-compose.yml` is included in this repository to make it easy to spin up LicenseDB along with PostgreSQL. +You'll need ```swag``` installed to build swagger docs. ```bash -# Build and start the containers -docker compose up -d +go install github.com/swaggo/swag/cmd/swag@latest ``` -This will start two services: -- **licensedb** → The LicenseDB API (exposed at `http://localhost:8080`) -- **postgres** → PostgreSQL 16 instance with preconfigured credentials - ---- - -## ⚙️ Environment Variables - -You can customize LicenseDB behavior via environment variables in the `docker-compose.yml`. - -| Variable | Default Value | Description | -|-----------------------------------|-------------------------|------------------------------------------------| -| `DB_HOST` | `postgres` | Hostname of the PostgreSQL database | -| `DB_PORT` | `5432` | PostgreSQL port | -| `DB_USER` | `fossy` | Database username | -| `DB_PASSWORD` | `fossy` | Database password | -| `DB_NAME` | `licensedb` | Database name | -| `GIN_MODE` | `debug` | Gin mode (`debug`, `release`) | -| `DEFAULT_ISSUER` | `http://localhost:8080` | Default issuer for authentication tokens | -| `SIMILARITY_THRESHOLD` | `0.9` | Similarity threshold for license matching | -| `PORT` | `8080` | Port where LicenseDB runs inside the container | -| `TOKEN_HOUR_LIFESPAN` | `24` | Token expiration time in hours | -| `READ_API_AUTHENTICATION_ENABLED` | `false` | Enable/disable authentication for read APIs | - ---- - -## 🔍 Verify the Setup - -Once containers are running, you can check: - -```bash -# List running containers -docker ps - -# View logs for LicenseDB -docker logs -f licensedb -``` +## How to run this project? -Backend should be available at: -👉 [http://localhost:8080](http://localhost:8080) - - -### Generating Swagger Documentation -1. Install [swag](https://github.com/swaggo/swag) using the following command. - ```bash - go install github.com/swaggo/swag/cmd/swag@latest - ``` -2. Run the following command to generate swagger documentation. - - ```bash - swag init --parseDependency --generalInfo api.go --dir ./pkg/api,./pkg/auth,./pkg/db,./pkg/models,./pkg/utils --output ./cmd/laas/docs - ``` -3. Swagger documentation will be generated in `./cmd/laas/docs` folder. -4. Run the project and navigate to `http://localhost:8080/swagger/index.html` to view the documentation. -5. Optionally, after changing any documentation comments, format them with following command. - ```bash - swag fmt --generalInfo ./pkg/api/api.go --dir ./pkg/api,./pkg/auth,./pkg/db,./pkg/models,./pkg/utils - ``` +👉 Please follow the [Setup Guide](./SETUP.md) for step-by-step instructions on how to run the project. ### Testing (local) + The PostgreSQL user `fossy` must have the `CREATEDB` privilege in order to: - Programmatically create and drop a test database. @@ -253,7 +121,9 @@ sudo -u postgres psql; // log into psql with postgres super user ALTER USER fossy CREATEDB; // alter the role for fossy \du ; // verify role ``` + Create the `.env.test` file file in the `configs` directory of the project. -``` + +```bash cp configs/.env.test.example .env.test ``` diff --git a/SETUP.md b/SETUP.md new file mode 100644 index 0000000..a1f9ab6 --- /dev/null +++ b/SETUP.md @@ -0,0 +1,158 @@ + + +# Project Setup + +There are 3 ways to run LicenseDb. + +## 1. Easy Install Script + +The easiest of them all is via the easy install script. + +- (Optional) Edit the ```configs/.env.dev.example``` to set the environment variables. If not done, default values will be taken. + +- (Optional) Edit the ```external_ref_fields.example.yaml``` to extend the schema of licenses and obligations with custom fields. If not done, default values will be taken. + +- Run the easy install script to generate the app binary. + +```bash +./easy_install.sh +``` + +- Run the executable + +```bash +./laas +``` + +Use the command below for more startup options. + +```bash +./laas --help +``` + +## 2. Docker Installation + +- Build the app image + +```bash +docker build -t licensedb/latest . +``` + +- Run the container + +```bash +docker compose up +``` + +## 3. Bare metal Installation + +### 1. Setting up the project + +- Create the `external_ref_fields.yaml` file in the root directory of the project to extend the schema of licenses and obligations with custom fields. + +```bash +cp external_ref_fields.example.yaml external_ref_fields.yaml +vim external_ref_fields.yaml +``` + +- Generate Go struct for the extra fields listed in the external_ref_fields.yaml. + +```bash +go generate ./... +``` + +- Create the `.env` file in the root directory of the project and change the + values of the environment variables as per your requirement. + +```bash +cp configs/.env.dev.example .env +vim .env +``` + +- Build the project using following command. + +```bash +go build ./cmd/laas +``` + +### 2. Setting up the database + +- Create database licensedb and provide user fossy all privileges to it. + +```sql +CREATE DATABASE licensedb; + +CREATE USER fossy WITH PASSWORD 'fossy'; + +GRANT ALL PRIVILEGES ON DATABASE licensedb TO fossy; +``` + +- Run the migration files. + +```bash +migrate -path pkg/db/migrations -database "postgres://fossy:fossy@localhost:5432/licensedb?sslmode=disable" up +``` + +- Create first user + +Connect to the database using `psql` with the following command. + +```bash +psql -h localhost -p 5432 -U fossy -d licensedb +``` + +Run the following query to create the first user. + +```sql +INSERT INTO users (user_name, user_password, user_level, display_name, user_email) VALUES ('', '', 'SUPER_ADMIN', '', ''); +``` + +### 3. Run the executable + +```bash +./laas +``` + +Use the command below for more startup options. + +```bash +./laas --help +``` + +- You can directly run it by the following command. + +```bash +go run ./cmd/laas +``` + +## Post Install + +- ### Generating Swagger Documentation + + - This step can be skipped if installation is done via the ```easy_install``` script. + - Install [swag](https://github.com/swaggo/swag) using the following command. + + ```bash + go install github.com/swaggo/swag/cmd/swag@latest + ``` + + - Run the following command to generate swagger documentation. + + ```bash + swag init --parseDependency --generalInfo api.go --dir ./pkg/api,./pkg/auth,./pkg/db,./pkg/models,./pkg/utils --output ./cmd/laas/docs + ``` + + - Swagger documentation will be generated in `./cmd/laas/docs` folder. + - Run the project and navigate to `http://localhost:8080/swagger/index.html` to view the documentation. + - After changing any documentation comments, format them with following command. + + ```bash + swag fmt --generalInfo ./pkg/api/api.go --dir ./pkg/api,./pkg/auth,./pkg/db,./pkg/models,./pkg/utils + ``` + +- Only the super admin user can create new app users, import licenses and obligations. + + To gain further capabilities, create a new admin user via the swagger docs or via the [LicenseDb UI](https://github.com/fossology/LicenseDb-UI). diff --git a/cmd/laas/main.go b/cmd/laas/main.go index 1de708a..b12bb86 100644 --- a/cmd/laas/main.go +++ b/cmd/laas/main.go @@ -31,8 +31,9 @@ import ( ) var ( - datafile = flag.String("datafile", "licenseRef.json", "datafile path") - populatedb = flag.Bool("populatedb", false, "boolean variable to update database") + datafile = flag.String("datafile", "licenseRef.json", "(optional) path of the file from which licenses are to be imported") + // auto-update the database + populatedb = flag.Bool("populatedb", false, "(optional) boolean variable to populate database with licenses, obligation types and classifications on start up") ) func main() { diff --git a/configs/.env.dev.example b/configs/.env.dev.example index 9449704..72f5a58 100644 --- a/configs/.env.dev.example +++ b/configs/.env.dev.example @@ -5,45 +5,46 @@ # How long the token can be valid(1 hour ) TOKEN_HOUR_LIFESPAN=1 # Secret key to sign tokens (openssl rand -hex 32) -API_SECRET=some-random-string +API_SECRET=0195067722c9213eee87c3db9b087ade # refresh token information(30 days) REFRESH_TOKEN_HOUR_LIFESPAN=720 -REFRESH_TOKEN_SECRET=some-other-random-string +REFRESH_TOKEN_SECRET=d78cded51eff6ab87e255b8f72f2215c READ_API_AUTHENTICATION_ENABLED=false PORT=8080 -# OIDC Provider (To be set if OIDC Authentication support required) +# Email Password JWT Token Configuration +DEFAULT_ISSUER=http://localhost:5000 + +# OIDC Provider (Optional, to be set if OIDC Authentication support required) # The URL for retrieving keys for Token Parsing -JWKS_URI=https://provider/keys +# JWKS_URI=https://provider/keys # The field in ID Token that is to be used as username -OIDC_USERNAME_KEY=employee_id +# OIDC_USERNAME_KEY=employee_id # The field in ID Token that is to be used as email -OIDC_EMAIL_KEY=mail +# OIDC_EMAIL_KEY=mail # The issuer url -OIDC_ISSUER=https://provider +# OIDC_ISSUER=https://provider # The field in ID Token that is used as display name -OIDC_DISPLAYNAME_KEY=display_name +# OIDC_DISPLAYNAME_KEY=display_name # Some OIDC providers do not provide the "alg" header in their key set(ex. AzureAD) # This env variable, if set, will be used for signing while verifying the JWT signature # (Make sure it's same as the signing algorithm used by the provider) # # For OIDC providers that provide the "alg" header in their key set, there is no need for this to be set -OIDC_SIGNING_ALG=RS256 +# OIDC_SIGNING_ALG=RS256 # LicenseDB M2M Configuration (To be set if M2M communication support required) # This field's should be equal to the oidc instance's client id -OIDC_CLIENT_TO_USER_MAPPER_CLAIM=azp - - +# OIDC_CLIENT_TO_USER_MAPPER_CLAIM=azp # Database info DB_NAME=licensedb @@ -56,12 +57,12 @@ DB_HOST=localhost # This value can be adjusted based on the requirements of the similarity search # A lower value will result in more matches, while a higher value will be more strict # Default is set to 0.7, but can be changed to a higher value like 0.8 or 0.9 for stricter matching -SIMILARITY_THRESHOLD = 0.8 +SIMILARITY_THRESHOLD=0.9 # SMTP Configuration ENABLE_SMTP=false -SMTP_HOST=smtp.gmail.com +SMTP_HOST=smtp.example.com SMTP_PORT=587 SMTP_USER=your_email@example.com SMTP_PASSWORD=your_password diff --git a/easy_install.sh b/easy_install.sh new file mode 100755 index 0000000..5446a0a --- /dev/null +++ b/easy_install.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# SPDX-FileCopyrightText: 2025 Dearsh Oberoi +# SPDX-License-Identifier: GPL-2.0-only + +set -euo pipefail + +# --- CONFIG --- +DB_NAME="licensedb" +DB_USER="fossy" +DB_PASS="fossy" +DB_PORT=5432 +DB_HOST="localhost" + +ADMIN_USER="fossy_super_admin" +ADMIN_PASS="fossy_super_admin" +ADMIN_DISPLAY="fossy_super_admin" +ADMIN_EMAIL="fossy_super_admin@example.org" + +# --- CONFIG FILES --- +if [ ! -f external_ref_fields.yaml ]; then + echo "Creating external_ref_fields.yaml..." + cp external_ref_fields.example.yaml external_ref_fields.yaml +fi + +echo "Generating Go structs..." +go generate ./... + +if [ ! -f .env ]; then + echo "Creating .env file..." + cp configs/.env.dev.example .env +fi + +# --- BUILD PROJECT --- +echo "Building project..." +go build ./cmd/laas + +# --- DATABASE SETUP --- +echo "Setting up PostgreSQL database..." +sudo -u postgres psql <