CellerCity/LinkVault
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
# Setup Instructions:- Clone the repository, using:- git clone [..../LinkVault] cd LinkVault * Check Node Version: (node -v) -> Developed on: v25.5.0 -> Minimum Requirement: v22.0.0+ (Required for Vite compatibility) -> npm version should be 10.9.4 or higher (automatically rectified when used with node version >= 22) * Switching to the compatible node version:- -> Install nvm (Node Version Manager) curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.0/install.sh | bash -> Restart your terminal to activate nvm nvm install 22 nvm use 22 ## Install dependencies, in each of the backend and frontend folders:- (use 2 separate terminals) cd backend npm install cd frontend npm install ## Set the .env variables Create 2 .env files in frontend and backend folders. ---> In ./frontend/.env Set the VITE_API_URL, take help from .env_example file. ---> In ./backend/.env Set the PORT_number, mongo_db connection string, Cloudinary details, see the .env_example for help. ## Launching the website:- start the server and frontend in two separate terminals. (ensure you are in the backend & frontend folders respectively) npm run dev npm run dev After that go to 'http://localhost:5173/' in the browser to open the website. ## Model definitions:- There are 2 databases, one for storing the users' login information and the other for storing the information related to files. The exact definitions can be found out in ./backend/models/ directory (User.js and File.js) 1. User schema (Login) username: [String] email: [String] password: [String] createdAt: [Date] 2. File Schema fileId: [String] type: [String] <'text' OR 'file'> content: [String] originalName: [String] createdAt: [Date] expiresAt: [Date] password: [String] // Optional security viewsLeft: [Number] // -1 = infinite, >0 = countdown // If null, it's an anonymous upload. If set, it belongs to a user. owner: [mongoose.Schema.Types.ObjectId, ref: 'User', default: null ] # HOW IT WORKS (A bird's eye view) User (Frontend) -> Validation -> API Request (Node.js) -> Middleware (Auth) -> [If File] -> Multer (Temp) -> Cloudinary (Storage) -> MongoDB (Metadata) [If Text] -> MongoDB (Content + Metadata) # API Overview Whenever an API link is typed or triggered somehow (internally), they perform some associated action(s). Each API has a type ranging from POST, GET, PUT, DELETE 1. Authentication Routes POST /api/auth/register Creates a new user account. Validates email uniqueness and hashes the password. POST /api/auth/login Authenticates a user and returns a JWT Token for session management. 2. Upload Routes POST /api/upload/text Uploads a text snippet. Supports optional password, custom alias(custom link-suffix if user wants to give some personalization), and expiry settings. POST /api/upload/file Uploads a file to Cloudinary and saves metadata to MongoDB. Supports password, alias, and expiry. 3. Publicly accessible Routes --> Even anonymous users could access these. GET /api/:id Retrieves file metadata. Returns 403 if expired/invalid. Triggers Kamikaze deletion if view count hits 0. POST /api/verify/:id Verifies the password for protected files. Returns the download link if the password matches. 4. User Dashboard Routes --> Only accessible for the duration in which a user stays logged in. (Even a logged in user is logged out after 1 hour, due to the beauty of the JWT token utilized) GET /api/user/files Fetches a list of all active files uploaded by the logged-in user, by a simple lookup from the MongoDB database. DELETE /api/user/files/:id Manually deletes a file from both MongoDB and Cloudinary (using the secureDelete helper). PUT /api/user/files/:id Updates file settings (Reset Expiry, Change/Remove Password, Update View Limit). # Design Choices ## Shift from server uploads/ to cloud storage Earlier I was saving all the uploads to the server using [multer] (local environment) itself. That is okay but is not scalable, as the deployed server couldn't handle a large quantity of huge files. To solve this issue I choose a free content delivery network (CDN) Cloudinary (as firebase is not free now). These cdn's are particularly optimized for handling such large files. A little implementation detail: Even though I am using Cloudinary, the implementation is such that multer needs to store a copy of the file on the server (uploads/), after which through another API call it is put to the cloud. The temp file is deleted right after. ## Multer Cleanup When a user uploads a file, it goes to the server's uploads/ folder before going to Cloudinary. If the Cloudinary upload fails (may happen due to poor network and stuff), the local file might sit there forever. This is extremely bad. I had the upload logic in a try...catch...finally block that performs fs.unlinkSync(localFilePath) --> simply deletes the local temp file, even if the upload fails. This guarantees the server stays clean no matter what happens. ## Background Service -> THE CRON JOB Cron Job: Runs every minute to automatically find an expired entry (file/text) from the MongoDB database and calls secureDelete to delete expired files from the database and Cloudinary cloud storage(for uploaded files). ## THE >>>KAMIKAZE<<< DELETE FEATURE (don't mind the unique name) Cloudinary as used in this app is a free version, with limits to uploads and accesses. So going to Cloudinary to check if a file has expired or not will probably exhaust the service. Instead I came up with an idea to store the all uploaded text/file information to the MongoDB database first. For files, the mongo_db has the file_id (randomly generated for each file using nanoid). The file is saved to Cloudinary with this exact name. Scenario-1: Now cron runs in the Background every minute. It looks for the files that have expired and triggers the secureDelete for only the particular file that expired. Scenario-2: In case the max_view count or download_count reaches 0, then too we trigger this secureDelete function. Scenario-3: One unpleasant race condition -- 1 time downloadable files [that led to Kamikaze deletes] This was a big issue. If a file was created to have only 1 download, it gets uploaded. Now when a user wants to view this, he is taken to the page that gives the cloud-link to download the file --> the view count is immediately decremented. The problem here is that a malicious user could share the cloudinary link to hundred's of other people exploiting the count mechanism. So I gave a timeout of 10s to the user to download the file from the cloudinary link, after which the cloud link also expires. ## The Non-existing file (Another Race Condition) When a file reaches 0 views, it now exists in the database for 10 more seconds. If a user refreshed their dashboard during this window, they would see a "Dead" file. Clicking delete would then return a 404[not found] error because the background timer killed it first. I filtered the dashboard query (viewsLeft: { $ne: 0 }) to hide these "walking dead" files. --> Another consequence: If the server says "404 Not Found" during a delete (the user was idle for 1 hour and then pressed delete), the UI treats it as a success (since the file is gone anyway). ## The user access control If a file is deleted due to expiry while the user is editing its access control (like changing its password or perhaps extending the expiry time) -> it displays the error message that the file was deleted, instead of crashing. ## Security Challenges and Fixes:- * Input Validation: user login email: whether it follows the proper format user login Password: its length must be > 6 Regex was used for aliases and type-checking for expiry/views on both frontend and backend. * Cloudinary Cache Busting: -> Normally cloudinary deletes the file by simply removing the entry from the (cloudinary user dashboard), but the link that is mapped to the file stays active. (a kind of caching you could say, as synchronous deletes can be expensive for the company) -> To solve this is used the parameter 'invalidate=true' in its API request, telling it immediately kill the file and unlink the 'link' leading to the uploaded file. * Defensive Checks: -> There all always some malicious users. The could set the expiry time to negative no. or a very huge no. causing integer overflow. The same thing could be done for views. So I set limits on Expiry (3 Days) and Views (100) to prevent storage abuse. I also allowed the user to set the view to INF (by putting a -1). # Assumptions and limitations * As the app is not deployed, all the users need to run the (server) backend as well as frontend on their machines to use this feature. (This is not a big issue as deploying a project is not too difficult) * The user limit. My current implementation possibly couldn't handle 1000's of users particularly due to the cloud limitations on size and latency.