-
Notifications
You must be signed in to change notification settings - Fork 2
Add dependency graph entry and test files. #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sahelib25
wants to merge
18
commits into
krai:master
Choose a base branch
from
sahelib25:master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
efb2553
Add entry for graph under varia_collection.
sahelib25 4d8cbcd
Upadate data_axs.json and code_axs.json
sahelib25 8093f91
Add dfs func for target node.
elimkwan f76c610
Update graph with parents of parents and multiple parents of output n…
sahelib25 65e207f
Fix the issue of accessing data_axs.json files based on given target …
sahelib25 b020883
Add visual features to the graph: colouring, grouping and clustering.
sahelib25 6101f26
Merge remote-tracking branch 'upstream/master'
sahelib25 bac5ed4
Add an extra functionality for extracting parents from output_entry_p…
sahelib25 549f744
Add more visual features to dependency graph.
sahelib25 50cce22
Enhance user readability of code_axs.py for graph entry.
sahelib25 032b5c7
Merge remote-tracking branch 'upstream/master'
sahelib25 d7bdfd9
Add README.md for graph
sahelib25 9a0b2ee
Enhance user readability for code_axs.py and README.md.
sahelib25 2ff306a
Merge branch 'krai:master' into master
sahelib25 569f570
Add features to print the hierarchical graph in a tree structure.
sahelib25 5cd9332
Remove redundant printing of entry_name with path
sahelib25 a690619
Merge branch 'krai:master' into master
sahelib25 406ffe8
Update Readme and fix typo.
sahelib25 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| ## Generating Hierarchy Graph | ||
| Set the target entry that we would like to draw a dependency graph for, e.g., image_classification_using_tf_py | ||
| ``` | ||
| export TARGET=image_classification_using_tf_py | ||
| ``` | ||
| ### `AXS` command to generate the graph: | ||
| ``` | ||
| axs byquery graph_output,target=${TARGET} | ||
| ``` | ||
|  | ||
| The figure shows a dependency graph generated for `image_classification_using_tf_py` entry. | ||
|
|
||
| - Red : target entry | ||
| - Lightcoral: Parents of the target entry | ||
| - Blue: output node | ||
| - Lightblue: Parents of the output node | ||
|
|
||
| If the run is successful, it should print `Graph is generated!` | ||
| ``` | ||
| saved to '/home/saheli/work_collection/generated_by_graph_on_draw_a579763d98044530962cc967ac659b28/data_axs.json' | ||
| byname_entries: ['base_imagenet_experiment'] | ||
| Graph is generated! | ||
| ['^', 'byname', 'generated_by_graph_on_draw_generated_by_graph_on_draw_a579763d98044530962cc967ac659b28'] | ||
| ``` | ||
| The image of the graph rendered in vector graphics, `image.svg` and `.dot` file are generated under `axs byquery graph_output,target=${target} , get_path`. Open `image.svg` with a GUI interface to view the generated graph. | ||
|
|
||
| The structure of the `.dot` file will look something like this: `image`. It can be interpreted as a normal text file. | ||
| ``` | ||
| digraph { | ||
| node [shape=ellipse] | ||
| dpi=400 | ||
| subgraph cluster_0 { | ||
| style=dotted | ||
| label="Entry and Its Parent(s)" | ||
| image_classification_using_tf_py [color=red style=filled] | ||
| python_script -> image_classification_using_tf_py | ||
| base_benchmark_program -> image_classification_using_tf_py | ||
| base_benchmark_program [color=lightcoral style=filled] | ||
| python_script [color=lightcoral style=filled] | ||
| python_in_shell -> python_script | ||
| python_in_shell [color=lightcoral style=filled] | ||
| shell -> python_in_shell | ||
| shell [color=lightcoral style=filled] | ||
| } | ||
| output [color=blue style=filled] | ||
| image_classification_using_tf_py -> output | ||
| base_imagenet_experiment -> output | ||
| subgraph cluster_1 { | ||
| style=dotted | ||
| label="Parent(s) of the Output Entry" | ||
| base_imagenet_experiment [color=lightblue style=filled] | ||
| base_experiment -> base_imagenet_experiment | ||
| base_experiment [color=lightblue style=filled] | ||
| } | ||
| } | ||
| ``` | ||
| ### `AXS` command to print the hierarchy tree: | ||
| ``` | ||
| axs byname graph , print_hierarchy llmcb_using_qaic_kilt | ||
| ``` | ||
|
|
||
| <Details><Pre> | ||
| saheli@chai:~/axs/core_collection/varia_collection/graph$ axs byname graph , print_hierarchy llmcb_using_qaic_kilt | ||
| output_entries_list: ['AS^IS', ['^', 'byname', 'base_llama2_loadgen_experiment'], ['^', 'byname', 'base_qaic_experiment'], ['^', 'byname', 'traced_kilt_entry']] | ||
| {'traced_kilt_entry', 'base_llama2_loadgen_experiment', 'base_qaic_experiment'} | ||
| output_entries_list: ['AS^IS', ['^', 'byname', 'base_mixtral_loadgen_experiment'], ['^', 'byname', 'base_qaic_experiment'], ['^', 'byname', 'traced_kilt_entry']] | ||
| {'traced_kilt_entry', 'base_llama2_loadgen_experiment', 'base_mixtral_loadgen_experiment', 'base_qaic_experiment'} | ||
| llmcb_using_qaic_kilt | ||
| | | ||
| +-/home/saheli/axs/core_collection/essentials_collection | ||
| shell | ||
| | | ||
| +-/home/saheli/work_collection/axs2kilt-dev | ||
| base_llmcb_kilt_program | ||
| | | ||
| +-/home/saheli/work_collection/axs2kilt-dev | ||
| base_kilt_program | ||
| | | ||
| +-/home/saheli/work_collection/axs2mlperf | ||
| base_loadgen_program | ||
| | | ||
| +-/home/saheli/work_collection/axs2mlperf | ||
| cpufreq_support | ||
| | | ||
| +-/home/saheli/work_collection/axs2mlperf | ||
| base_loadgen_program | ||
| | | ||
| +-/home/saheli/work_collection/axs2mlperf | ||
| cpufreq_support | ||
| | | ||
| +-/home/saheli/work_collection/axs2qaic-dev | ||
| base_qaic_program | ||
| | | ||
| +-/home/saheli/work_collection/axs2mlperf | ||
| base_loadgen_program | ||
| | | ||
| +-/home/saheli/work_collection/axs2mlperf | ||
| cpufreq_support | ||
| | | ||
| +-/home/saheli/work_collection/axs2qaic-dev | ||
| qaic_sdk_info | ||
| | | ||
| -->/home/saheli/work_collection/axs2kilt-dev :: Output Parents | ||
| traced_kilt_entry | ||
| | | ||
| -->/home/saheli/work_collection/axs2mlperf :: Output Parents | ||
| base_llama2_loadgen_experiment | ||
| | | ||
| +-/home/saheli/work_collection/axs2mlperf | ||
| base_loadgen_experiment | ||
| | | ||
| -->/home/saheli/work_collection/axs2mlperf :: Output Parents | ||
| base_mixtral_loadgen_experiment | ||
| | | ||
| +-/home/saheli/work_collection/axs2mlperf | ||
| base_loadgen_experiment | ||
| | | ||
| -->/home/saheli/work_collection/axs2qaic-dev :: Output Parents | ||
| base_qaic_experiment | ||
| Tree printed successfully! | ||
| </Details></Pre> | ||
|
|
||
| ## Running tests | ||
elimkwan marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| The next step is to run the tests. | ||
|
|
||
| ### Step 1: | ||
| The first step is to run the `create_json.py` to generate two json files containing dictionaries containing all the contained entries obtained as keys from `bert`, `image_classification`, `object_detection` folders and corresponding values as the `_parent_entries` and `output_parent_entries` for each entry. | ||
| ``` | ||
| python create_json.py | ||
| ``` | ||
| <Details><Pre> | ||
| output_parent_entries_dict.json parent_entries_dict.json </Details></Pre> | ||
|
|
||
| ### Step 2: | ||
| Next, to run the `test_parent_and_output_entries.py` file | ||
| ``` | ||
| pytest test_parent_and_output_entries.py | ||
| ``` | ||
| <Details><Pre> | ||
| ==================================================================================== 2 passed in 0.01s ===================================================================================== | ||
| collected 2 items | ||
|
|
||
| test_parent_and_output_entries.py::test_compare_dot_and_json_for_target PASSED | ||
| test_parent_and_output_entries.py::test_compare_dot_and_json_for_target_output PASSED | ||
| </Details></Pre> | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.