README.Rmd

---
output: github_document
---

<!-- README.md is generated from README.Rmd. Please edit that file -->

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)
```
# wikitablr <img src="man/figures/wikitablr_hex_logo.png" align="right" height=150/>

<!-- badges: start -->
[![Travis-CI Build Status](https://travis-ci.org/jkeast/wikitablr.svg?branch=master)](https://travis-ci.org/jkeast/wikitablr) 
<!-- badges: end -->


`wikitablr` is an R package that has the tools to simply webscrape tables from wikipedia, and clean for common formatting issues. The intention here is to empower beginners to explore data on practically any subject that interests them (as long as there's a wikipedia table on it), but anyone can utilize this package.

`wikitablr` takes data that looks like this:

```{r, warning = FALSE, message = FALSE, echo = FALSE}
# Edit this part
library(wikitablr)
head(read_wiki_raw("https://en.wikipedia.org/wiki/List_of_songs_recorded_by_the_Beatles", 2))

```

and makes it look like this:

```{r, warning = FALSE, message = FALSE, echo = FALSE}
# Edit this part
head(read_wiki_table("https://en.wikipedia.org/wiki/List_of_songs_recorded_by_the_Beatles", 2))

```

## Installation

You can install the development version from [GitHub](https://github.com/) with:

``` r
# install.packages("devtools")
devtools::install_github("baumer-lab/wikitablr")
```
## Example

The first step is to read in all tables on a given wikipedia page using `read_wikitables()`. The input is a url, and the output is a data frame including the tables on the page as well as some additional information about each table. 

```{r}
#read in tables
colleges <- read_wikitables("https://en.wikipedia.org/wiki/List_of_colleges_and_universities_in_Massachusetts")
head(colleges)
```

The next step is to clean the tables by either extracting a single table and applying cleaning functions individually, or mapping the cleaning functions over all of the tables.

```{r}
# extract table 1
table1 <- read_wikitables("https://en.wikipedia.org/wiki/List_of_colleges_and_universities_in_Massachusetts") %>%
   pull(table) %>%
   pluck(1) 
table1
```

```{r}
#remove footnotes from table
head(remove_footnotes(table1))
```

```{r}
# remove footnotes from all tables
colleges %>%
  mutate(clean_table = map(table, remove_footnotes))
```