Skip to content

One-post download mode #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
baldurk wants to merge 2 commits into
base: master
Choose a base branch
from
Open

One-post download mode #2

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7f855e3
Add hacky download-one-post mode
baldurk Mar 29, 2011
6d77a3a
Merge remote branch 'upstream/master'
baldurk Mar 30, 2011
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion fetch-thread.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
# These following settings should be set to 1 to enable them,
# or to 0 to disable them

$downloadOnePost = 0;

# Fetch images and videos
# (you might want to temporarily disable this for faster testing)
$downloadFiles = 1;
Expand Down Expand Up @@ -63,11 +65,16 @@


# renum'ing? - ie, does index.html exist?
if(-s "index.html") {
if(!$downloadOnePost and -s "index.html") {
&renumThread;
exit;
}

if($downloadOnePost and -s "Update XYZ/index.html") {
print "Already got a single post as XYZ! Move it out of the way before grabbing another.\n";
exit;
}

# Ok, we're fetching

# OS and cookie detection
Expand Down Expand Up @@ -147,6 +154,10 @@
$numVids = 0; $vidSize = 0;
$errors = ""; $skippedImgFile = ""; $skippedVidFile = "";

if($downloadOnePost) {
$autoChapters = 0;
}


if($autoChapters) {
# Start by running through to check how many valid posts have
Expand Down Expand Up @@ -255,6 +266,10 @@
# Delete temporary file if we downloaded the thread ourselves
if($threadURL) { unlink 'autoFetchedThread.txt'; }

if($downloadOnePost) {
exit;
}

# Need to delete the 'next' links from the final update

# Read in the existing file
Expand Down Expand Up @@ -640,6 +655,8 @@ sub getPost {
if($upd == -1) { $keepPost = 1; }
}

if($downloadOnePost) { $keepPost = 1; }

if(! $keepPost) {
# Keeping requires: $minImage images or a video href (if videoPosts is on)
# not contained inside a [quote] (a <blockquote>)
Expand All @@ -651,6 +668,8 @@ sub getPost {
$gotUpdate = 1;
$upd++;

if($downloadOnePost) { $upd = 50; }

# Fix definitely typo filter
$postBody =~ s/\[NOTE: I AM TOO STUPID TO SPELL THE WORD "DEFINITELY" CORRECTLY\]/definitely/g;

Expand All @@ -667,6 +686,9 @@ sub getPost {

# Create a directory to put it all in
$postDir = sprintf("Update %02d", $upd);

if($downloadOnePost) { $postDir = "Update XYZ"; }

mkdir $postDir || die $!;

# Generate the HTML file
Expand Down Expand Up @@ -958,6 +980,8 @@ sub getThread {
}
close(FIRST);

if($downloadOnePost) { $pages = 1; $authorID = 0; $authorPosts = 1; }

# If got the author ID and there's more than one page, need to reget first page to get correct number of pages
if($authorID && $pages > 1) {
print "* Fetching thread from $threadURL with author ID $authorID...\n";
Expand Down