Clipper Manager Python Tests by Corey-Zumar · Pull Request #187 · ucbrise/clipper

Corey-Zumar · 2017-05-30T09:56:25Z

Test via:

# Execute short tests (total execution time < 1 min)
$ python /management/test/clipper_manager_test.py short
# Or execute long tests (total execution time ~ 10 mins)
$ python /management/test/clipper_manager_test.py long
# Or execute all tests
$ python /management/test/clipper_manager_test.py

This reverts commit aa9da7e.

This reverts commit 51c82a8.

This reverts commit 01ec137.

This reverts commit c85f2c0.

This reverts commit 57b0e97.

This reverts commit 4455ccb.

This reverts commit 5906661.

AmplabJenkins · 2017-05-30T10:15:36Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/340/
Test FAILed.

AmplabJenkins · 2017-05-30T10:33:24Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/341/
Test FAILed.

dcrankshaw

This is great! It will be really good to have test coverage of clipper_manager as we start refactoring it.

dcrankshaw · 2017-05-30T21:01:19Z

bin/run_unittests.sh

  echo -e "\nRunning management tests\n\n"
  ./src/management/managementtests --redis_port $REDIS_PORT
+  cd $DIR
+  python ../management/test/clipper_manager_test.py


clipper_manager.py got moved from the management/ to the clipper_admin/ directory as part of making the pip package. Move the tests to the same place (clipper_admin/test).

dcrankshaw · 2017-05-30T21:08:45Z

management/test/clipper_manager_test.py

+
+if __name__ == '__main__':
+    run_short = True
+    run_long = True


There should be a run_all option as well

dcrankshaw · 2017-05-30T21:18:46Z

management/test/clipper_manager_test.py

+        self.assertGreaterEqual(len(running_containers_output), 1)
+
+
+class ClipperManagerTestCaseLong(unittest.TestCase):


Can you import sklearn here so that the short test case can still be run without sklearn installed?

Both cases requiresklearn. It is required in test_model_deploys_successfully (short case) and test_deployed_model_queried_successfully (long case).

dcrankshaw · 2017-05-30T21:32:44Z

management/test/clipper_manager_test.py

+cur_dir = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.abspath('%s/../' % cur_dir))
+import clipper_manager
+


Can you add a comment at the top of the file briefly describing how these tests are run? Basically just say that at the beginning of each test suite a local Clipper instance is created and the tests issue various commands against the running Clipper instance and verify their results.

This is just so that it's documented that these tests start Docker containers and interact with them, as opposed to being pure Python tests.

Corey-Zumar · 2017-06-03T04:38:54Z

@dcrankshaw Addressed your comments.

AmplabJenkins · 2017-06-03T04:57:51Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/366/
Test FAILed.

AmplabJenkins · 2017-06-03T16:25:28Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/368/
Test FAILed.

dcrankshaw

I pushed a couple really minor cleanup changes.

I commented out the deploy_predict_function tests until #194 gets merged because dealing with conda environments in jenkins is a little messy and I don't want to be blocked on that.

AmplabJenkins · 2017-06-03T16:53:15Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/369/
Test FAILed.

AmplabJenkins · 2017-06-03T17:26:01Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/370/
Test FAILed.

AmplabJenkins · 2017-06-03T17:46:06Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/373/
Test FAILed.

AmplabJenkins · 2017-06-03T17:55:23Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/374/
Test FAILed.

AmplabJenkins · 2017-06-03T18:16:23Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/375/
Test PASSed.

Corey-Zumar added 19 commits May 30, 2017 03:51

Infra for default explanations

16b11d7

Propagate default explanation to response

9e1f49a

Format code

73208e3

Add latency default expl, format

8d0ba82

Remove msg_ from exception

0a2bdf8

Fix tests, update json schema

b257318

Fix test case

6daa18e

Basic manager tests

f58fda6

More complete test suite for clipper manager

87b1a2f

2 more tests, run all clipper mgr unit tests

049dcdc

Revert "Fix test case"

471a2c4

This reverts commit aa9da7e.

Revert "Fix tests, update json schema"

cb8f01f

This reverts commit 51c82a8.

Revert "Remove msg_ from exception"

ffb25a7

This reverts commit 01ec137.

Revert "Add latency default expl, format"

e75823d

This reverts commit c85f2c0.

Revert "Format code"

44459bc

This reverts commit 57b0e97.

Revert "Propagate default explanation to response"

c116a9f

This reverts commit 4455ccb.

Revert "Infra for default explanations"

e287933

This reverts commit 5906661.

EOF space

4ec5781

format code

0dd9784

Corey-Zumar added the status: needs review label May 30, 2017

Corey-Zumar requested a review from dcrankshaw May 30, 2017 09:56

Fix unittests script

a4482c3

dcrankshaw requested changes May 30, 2017

View reviewed changes

dcrankshaw added status: needs revision type: maintenance and removed status: needs review labels May 30, 2017

dcrankshaw assigned Corey-Zumar May 30, 2017

Rename test file

8fcd509

dcrankshaw removed the status: needs review label Jun 1, 2017

Corey-Zumar mentioned this pull request Jun 2, 2017

Assorted usability fixes #192

Merged

Corey-Zumar added 5 commits June 2, 2017 08:35

Pluralize test folder

93d72a5

Address comments

ac74e02

Format code

f042f59

Replace optparse with argparse

66fb3bb

format code

1abf588

Corey-Zumar force-pushed the python_tests branch from c6bb828 to 1abf588 Compare June 3, 2017 04:37

Corey-Zumar added status: needs review and removed status: needs revision labels Jun 3, 2017

dcrankshaw added 2 commits June 3, 2017 09:22

clean up and commented out deploy_predict_function tests

2731be5

Merge remote-tracking branch 'ucbrise/develop' into python_tests

fd111ca

dcrankshaw approved these changes Jun 3, 2017

View reviewed changes

dcrankshaw added status: accepted and removed status: needs review labels Jun 3, 2017

format

7f29196

fixed import

dfeebb5

dcrankshaw mentioned this pull request Jun 3, 2017

Make deploy_predict_function work without Anaconda environments #194

Merged

fixed unittests script

aff454b

made unit tests use random redis port

4f39f20

format code

5f19237

dcrankshaw merged commit bfa82b9 into ucbrise:develop Jun 3, 2017

		self.assertGreaterEqual(len(running_containers_output), 1)


		class ClipperManagerTestCaseLong(unittest.TestCase):

Conversation

Corey-Zumar commented May 30, 2017

Uh oh!

AmplabJenkins commented May 30, 2017

Uh oh!

AmplabJenkins commented May 30, 2017

Uh oh!

dcrankshaw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Corey-Zumar commented Jun 3, 2017

Uh oh!

AmplabJenkins commented Jun 3, 2017

Uh oh!

AmplabJenkins commented Jun 3, 2017

Uh oh!

dcrankshaw left a comment

Choose a reason for hiding this comment

Uh oh!

AmplabJenkins commented Jun 3, 2017

Uh oh!

AmplabJenkins commented Jun 3, 2017

Uh oh!

AmplabJenkins commented Jun 3, 2017

Uh oh!

AmplabJenkins commented Jun 3, 2017

Uh oh!

AmplabJenkins commented Jun 3, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants