README.md

Welcome to Log-Errors

This Repository contains a multitude of sub projects.

1. Log Parser - tokenizes log files of the format in here

2. StackOverflow Scraper - an api that generates a database of posts from stack overflow

3. Mongo Matcher - a matching algorithm for comparing errors to data scraped by the StackOverflow Scraper

4. Mongo DB API - an API for the mongo db containing information from StackOverflow Scraper

Installation Of Project

Java

git clone https://github.com/cloudhubs/log-errors.git

Install Lombok for your IDE

Python

This will use pycharm as the IDE of choice. (Assuming you have already cloned the repo)

1. Open log-errors/scraper in Pycharm

2. Indicate the python interpreter if prompted at the top.

3. Open a new terminal at the bottom of the window.

4. If the line does not start with the name of your virtual environment, follow this

   for example: note the (venv) indicating the virtual environment, the path and the brach (master)

   (venv) mark@MarksComputer ~/Documents/scraper/log-errors (master) $ 

5. Run the following command in scraper/ root

   pip install -r requirements.txt 

6. To run the project right click in api/main.py and select run

7. Alternatively using bash:

$ python3 -m venv venv
$ source venv/bin/activate
$ pip3 install -r requirements.txt
$ python3 main.py

Log Parser API

Start the parser here

• Parse File: POST /errors/

StackOverflow Scraper API

The mongo and scraper apis are run simultaneusly. Start them from this file

• Start Scraper POST /scrape/{language}
• Scrape Meta-data POST /scrape-meta/{language}
• Stop Scraper POST /scrape/stop

MongoDB API

The mongo and scraper apis are run simultaneusly. Start them from this file

• Home GET /mongo
• Find All GET /mongo/test
• Add POST /mongo/test
• Empty DELETE /mongo/test

Mongo Matcher

Start the parser here

-Find Matches GET /matcher