Skip to content

Refactor scraper to be modular and testable #47

@dsychin

Description

@dsychin

Separate logic with external dependencies.
External dependencies should have an interface.

Have a top level function which handles the state and call smaller functions to fetch more data at each stage.

  • Get job links and company links and add to map
  • Get job details
  • Get company details
  • Return results

WaitGroup details should be hidden in the implementation

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions