Pages

[codeimpossible]

Shing needs to parse all the pages of a wishlist....

Currently (in my latest pull) we have code in the WishList object to pull down the html for the first page and loop over the items in it:

var scraper = new WishListScraper(url);
var source = scraper.Scrape();
using(var parser = ParserFactory.GetParser(source))
{
    var creator = parser.GetCreator();
    foreach(var rawItem in parser.GetRawItems())
    {
        yield return creator.Create(rawItem);
    }
}

I propose that we give IScraper the concept of Pages(). So our new code would look like this:

var scraper = new WishListScraper(url);

foreach(var page in scraper.Pages()) 
{
    using(var parser = ParserFactory.GetParser(page))
    {
        var creator = parser.GetCreator();
        foreach(var rawItem in parser.GetRawItems())
        {
            yield return creator.Create(rawItem);
        }
    }
}

I'm thinking that Pages() could return an IEnumerable and we could fetch the source of each page when Next() is called.

thoughts?

[codeimpossible]

HAHAHAHHAHAHAHA

so not needed, we can just add ?items-per-page=300 to the url BAM! no need for pages!!!