Skip to content
This repository was archived by the owner on Apr 8, 2024. It is now read-only.

[hold] Add batch latency measurement and percentiles to treelite_python#177

Open
jfomhover wants to merge 19 commits intomainfrom
jfomhover/treelitepercentiles
Open

[hold] Add batch latency measurement and percentiles to treelite_python#177
jfomhover wants to merge 19 commits intomainfrom
jfomhover/treelitepercentiles

Conversation

@jfomhover
Copy link
Contributor

@jfomhover jfomhover commented Nov 24, 2021 •
edited
Loading

This PR introduces num_threads and batch_size as parameters for inferencing benchmark.

Additions:

  • code to process batches of data in the treelite_python module,
  • unified code to record latency metrics and percentiles.
  • num_threads and batch_size in inferencing variant parameters
  • num_threads and batch_size in the inferencing pipeline (depending on support from each component)

@jfomhover jfomhover added the enhancement New feature or request label Nov 24, 2021
@jfomhover jfomhover added this to the Standardization milestone milestone Nov 24, 2021
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 07:08 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 07:11 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 07:14 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 17:07 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:22 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:23 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:31 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:44 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:46 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:49 Inactive
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 18:59 Inactive
@jfomhover jfomhover mentioned this pull request Nov 24, 2021
Closed
@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 19:03 Inactive
@github-actions
Copy link

github-actions bot commented Nov 24, 2021 •
edited
Loading

Unit Test Results for Build

  1 files  ±0    1 suites  ±0   7s ⏱️ ±0s
76 tests ±0  76 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 51c375b. ± Comparison against base commit 7b8a339.

♻️ This comment has been updated with latest results.

@jfomhover jfomhover temporarily deployed to mlops November 24, 2021 20:31 Inactive
@jfomhover jfomhover mentioned this pull request Nov 24, 2021
Merged
@jfomhover jfomhover temporarily deployed to mlops November 25, 2021 05:12 Inactive
@jfomhover jfomhover changed the title Add batch latency measurement and percentiles to treelite_python [hold] Add batch latency measurement and percentiles to treelite_python Nov 29, 2021
@jfomhover jfomhover added the hold This PR/Issue should be put on hold. label Nov 29, 2021
@jfomhover
Copy link
Contributor Author

jfomhover commented Nov 29, 2021

Split changes into a PR to unify metrics first, waiting for #180 merged

@jfomhover jfomhover temporarily deployed to mlops December 3, 2021 19:00 Inactive
@jfomhover jfomhover temporarily deployed to mlops December 3, 2021 19:02 Inactive
@jfomhover jfomhover removed this from the Standardization milestone milestone Feb 10, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

enhancement New feature or request hold This PR/Issue should be put on hold.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jfomhover