Implement background process manager

[dsychin]

Suggestion to use a background process library like Hangfire to handle processing background jobs.

Changes to the worker:

• Instead of start to end process, the worker will be constantly running (or waiting).
• Can process tasks in parallel.
  • E.g. Scraper loads html, adds each job and company to background queue.
  • Worker can process n jobs and companies at the same time.
  • E.g. 2. While updating jobs, new jobs can be detected and added to a queue for notifications to be processed and sent later
• Jobs can be scheduled
• We can view the jobs through a web interface if we enable it

[syahnur197]

when I read Hangfire, I notice there are many types of jobs, for this one, we should be using Recurring Jobs correct?