Update main #4
Merged
Update main #4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…oad for inference on Llama 3.1 8B and 70B * added the newest version on llm-inference-megatron-lm combined * Update workloads/llm-inference-megatron-lm/helm/mount/run_megatron.sh * precommit fixed * Resolve the last comments in the code * resolving and testing tokenizer path * adding the kaiwo comments * precommits made --------- Co-authored-by: aivanni <4340981+aivanni@users.noreply.github.com>
* Initial commit, skeleton, data prep Signed-off-by: Robert Talling <rtalling@amd.com> * Add Megatron checkpoint conversion details to the tutorial; Add override file for llama-3.1-8B in workloads/download-huggingface-model-to-bucket Signed-off-by: Robert Talling <rtalling@amd.com> * Rename tutorial and fix Signed-off-by: Robert Talling <rtalling@amd.com> * Fix line splits Signed-off-by: Robert Talling <rtalling@amd.com> * fixes and llama 70 overrides for model delivery Signed-off-by: Robert Talling <rtalling@amd.com> * Update readme Signed-off-by: Robert Talling <rtalling@amd.com> * Add multinode instructions + values Signed-off-by: Robert Talling <rtalling@amd.com> * Refine inference workload instructions for Llama-3.1-8B model and update k9s commands Signed-off-by: Robert Talling <rtalling@amd.com> * places were Values.yaml were adjusted for testing were reverted Signed-off-by: Robert Talling <rtalling@amd.com> * Update workloads/download-huggingface-model-to-bucket/helm/values.yaml Signed-off-by: Robert Talling <rtalling@amd.com> * Update docs/tutorials/tutorial-03-deliver-resources-and-run-megatron-cpt.md Signed-off-by: Robert Talling <rtalling@amd.com> * Add 16ddp template for multinode, remove explicit namespace mentions Signed-off-by: Robert Talling <rtalling@amd.com> * Update tutorial template for llama 8b inference Signed-off-by: Robert Talling <rtalling@amd.com> * Fix pre-commit errors Signed-off-by: Robert Talling <rtalling@amd.com> * Remove unused overrides and update readme Signed-off-by: Robert Talling <rtalling@amd.com> * Update readme Signed-off-by: Robert Talling <rtalling@amd.com> * Replace set up section with tutorial-0-prerequisites. * correction the helm template path for llm-inference-megatron-lm * resolved override path inside helm, and update README * Update docs/tutorials/tutorial-03-deliver-resources-and-run-megatron-cpt.md --------- Signed-off-by: Robert Talling <rtalling@amd.com> Co-authored-by: Saroosh Shabbir <saroosh.shabbir@amd.com> Co-authored-by: Robert Talling <rtalling@amd.com> Co-authored-by: eliecer diaz <eliecerecology@gmail.com>
* jupyterlab: improve documentation with finding correct url and reminding of namespace * jupyterlab docs: fix namespace
alexander-aurell-amd
previously approved these changes
Jul 15, 2025
* vllm0.9 best-known config update * update api benchmarking scripts to match vllm version * Add benchmark configuration options for input/output lengths and QPS * Update container environment variable handling to properly support secret references * Remove deprecated image references from model configuration files in the LLM inference Helm overrides for DeepSeek, Google Gemma, Meta Llama, Mistral, and Qwen models. * revert NUMA config for it is read-only * Remove extra sleep --------- Co-authored-by: Aku Rouhe <akurouhe@gmail.com>
aivanni
approved these changes
Jul 15, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.