Updated gui website images skills and events to current Global+JP#111
Open
l2ofl wants to merge 3 commits intoMagody:mainfrom
Open
Updated gui website images skills and events to current Global+JP#111l2ofl wants to merge 3 commits intoMagody:mainfrom
l2ofl wants to merge 3 commits intoMagody:mainfrom
Conversation
-Revised datasets/scrape_events.py to handle naming cards json name and id attributes more uniquely as well as the images for them. Additionally added an exception array to handle exception like 2x Silence Suzuku SPD SSR -Wrote a wrapper datasets/run_characters_supports_cpu.py that scrapes all availble data character and support data on gametora using lxml and robots.txt for the site. Additionally uses playwright to run gametora in a headless mode and capture a skills<hash>.json and then run it through scrape_skills.py -The latest data as of 2/20/2026 is reflected in \web\dist and \web\public -Updated requirements.txt to include beautifulsoup4, lxml and playwright -detailed features of run_characters_supports_cpu.py Confirm npm works (npm --version) Get sitemap URL from robots.txt Fetch sitemap(s), extract all URLs Extract support slugs + character slugs Merge+dedupe supports → list Merge+dedupe characters → list Combine both + final dedupe Write datasets/in_game/events.json Run build_catalog.py Run npm run build (retry with install if needed) Hardcoded scenario events to append to events.json Cleaned up old images in web/public/events folder Scrape skills from gametora using playwright to run headless chromium Probably some other stuff but I forgor and this is too long already
run_characters_supports_cpu wrapper creates a folder to hold each individual event since it multithreads calls to scrape_events.py does not need to be committed as script deletes it on each run anyways. Additionally update gitignore to well ignore it
Grab new icons if available for skills and grab more data from skills as well as update icons for skills if not updated
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
-Revised datasets/scrape_events.py to handle naming cards json name and id attributes more uniquely as well as the images for them. Additionally added an exception array to handle exception like 2x Silence Suzuku SPD SSR -Wrote a wrapper datasets/run_characters_supports_cpu.py that scrapes all availble data character and support data on gametora using lxml and robots.txt for the site. Additionally uses playwright to run gametora in a headless mode and capture a skills.json and then run it through scrape_skills.py -The latest data as of 2/20/2026 is reflected in \web\dist and \web\public -Updated requirements.txt to include beautifulsoup4, lxml and playwright -detailed features of run_characters_supports_cpu.py
Confirm npm works (npm --version)
Get sitemap URL from robots.txt
Fetch sitemap(s), extract all URLs
Extract support slugs + character slugs
Merge+dedupe supports → list
Merge+dedupe characters → list
Combine both + final dedupe
Write datasets/in_game/events.json
Run build_catalog.py
Run npm run build (retry with install if needed)
Hardcoded scenario events to append to events.json
Cleaned up old images in web/public/events folder
Scrape skills from gametora using playwright to run headless chromium
Probably some other stuff but I forgor and this is too long already