It describes the process of building a career database for students who want to find a business or data analyst job.
The data sources used were LinkedIn, which provides detailed profiles of professionals and job opportunities, and the web scraping routine involved using BeautifulSoup and Selenium to extract data on 200 data analyst job positions and their respective company websites.
The company information and position data are stored in a MongoDB database, and structured the collection to facilitate effortless storage and updates of the data. Suitable indexes are designed for ensuring the efficiency of retrieving information from the database.
The purpose of scraping this particular data was to analyze trends in the job market, including changes in demand for certain skills, industries, and job titles.