manage proxies better

[vikstrous]

We have to address a few issues:

• For users who can access the pirate bay directly we need to prioritize the real pirate bay
• For users who can't we need to make sure they can configure their preferred proxy or proxy list to be first
• We need an extensible way of parsing proxy lists
• We should consider adding actual http proxies as an option in addition to reverse proxies
• We already have a way to blacklist proxies, but this is something that we might want to have more control over. I propose that we run a github hosted website with json lists that the client can periodically sync with. I imagine the client caching the data for about a day.
• We should prefer https over http whenever possible

We'll need some refactoring to make this happen and we really need proper testing once the codebase is testable.

I'd say this is blocked on #44 to avoid breaking anything in the process of adding these features.

[vindarel]

That's a good todo list :) I can't use pirate-get ATM because I can't reach proxybay.co/list.txt. I imagine doable to have a [Mirrors] section where one can add the mirror(s) that is working for him. (I tried, just have to sharpen my python3 tooling…).

[allgreed]

@vikstrous I'd also add to the list caching the last working proxy and trying that one first -> once I did a search and 10th proxy worked. I did another search seconds later and had to wait-out 9 requests fails -> maybe something like:

if original pirate bay fails and has cached working proxy:
    try that proxy
    if that proxy fails:
        go through the list (without the tried proxy)

That would reduce tries on subsequest request to maximum 2
Would you think of this as desired behavior?

[vikstrous]

Yeah that would be nice. We'd have to write out a file somewhere and check it first.

…

On December 7, 2017 2:56:17 PM EST, Olgied Kasprowicz ***@***.***> wrote: @vikstrous I'd also add to the list caching the last working proxy and trying that one first -> once I did a search and 10th proxy worked. I did another search seconds later and had to wait-out 9 requests fails -> maybe something like: ```pseudocode if original pirate bay fails and has cached working proxy: try that proxy if that proxy fails: go through the list (without the tried proxy) ``` That would reduce tries on subsequest request to maximum 2 Would you think of this as desired behavior? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #59 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.