---
title: "HW08-P1 Scraping"
format: 
  html: 
    embed-resources: true
---

## Instructions

**Requirements**

Ex 1

- Have a correct answer to who can scrape that data based on the information in robots.txt

Ex 2

- Write a description of what `httr_attr()` does
- Include a code example to demonstrate what it does

Ex 3

- Code should result in 2046 rows and 3 columns of data about cheese
- Code should result in unique information about name, url, and whether or not it has an image
- Code should include Sys.sleep(1) to not overtax website

Ex 4

- Code should result in 10 rows and 6 columns of data about 10 cheeses
- Code should result in unique information about cheese identifier (name or url), milk, country, family, type, flavor
- Code should include Sys.sleep(1) to not overtax website


## Goal

Scrape information from <https://www.cheese.com> to obtain a dataset of characteristics about different cheeses, and gain deeper insight into your coding process.


## Ex 1

Locate and examine the `robots.txt` file for <https://www.cheese.com>. Summarize (in words) what you learn from the cheese.com `robots.txt` file.


## Ex 2

Learn about the `html_attr()` function from `rvest`. Describe how this function works, and give a small code example to illustrate how it works.



```{r}


```


## Ex 3

Obtain the following information for **all** cheeses in the alphabetical database <https://www.cheese.com/alphabetical/>:

- Cheese name
- URL for the cheese's webpage (e.g., <https://www.cheese.com/gouda/>)
- Whether or not the cheese has a picture (e.g., [gouda](https://www.cheese.com/gouda/) has a picture, but [bianco](https://www.cheese.com/bianco/) does not)

To be kind to the website owners, please add a 1 second pause between page queries. (Note that you can view 100 cheeses at a time.)


```{r}
# Load required libraries
library(rvest)
library(dplyr)

# Define the URL


# Read the HTML content from the webpage
webpage <- read_html(url)

# Extract the cheese names and URLs


# Check if the cheese has a picture


# Create a data frame to store the results



```

Print the dimensions & first few rows of your final data frame:

```{r}
dim(cheese_df)
head(cheese_df)
```



## Ex 4

When you go to a particular cheese's page (like [gouda](https://www.cheese.com/gouda/)), you'll see more detailed information about the cheese. For **just 10** of the cheeses in the database, obtain the following detailed information: milk information, country of origin, family, type, and flavour. (Just 10 to avoid overtaxing the website. Continue adding a 1 second pause between page queries.) 

```{r}


```

Print the dimensions & first few rows of your final data frame:

```{r}


```


## Resources Reflection (required)

List the resources you used to help with this assignment then write 3-5 sentence reflection on which resources were most helpful in finishing this task.

**Resources:**

-
-
-

**Reflection:**