|
1 | 1 | # sclblpy |
2 | 2 |
|
3 | | -[](https://pypi.org/project/sclblpy/) |
4 | | - |
5 | | - |
6 | | -Changelog: the package is currently being refactored. As such, it is currently a stub, as Scailable has changed its focus from Scikit Learn to ONNX, Scikit Learn support has been removed. |
7 | | -1. Removal of support for additional packages - most importantly in the 'upload()' function which now will throw an error if `model_type` isn't onnx |
8 | | -2. the `run()` function is going to be replaced by a function that allows the user to test their local setup; and as such is a stub |
9 | | - |
10 | | - |
11 | | -`sclblpy` is the core python package provided by Scailable for interacting with our API. The scope of this package is (roughly): |
12 | | -1. upload an `.onnx` model to the admin console |
13 | | -2. Assign and deploy the uploaded model to a device that has previously been installed with the Scailable runtime and registered |
14 | | -3. (upcoming) test device deployment |
15 | | - |
16 | | - |
17 | | - |
18 | | - |
19 | | -Also, there will probably be breaking changes in the API in one of the upcoming releases, for more information see [our API documentation](https://docs.sclbl.net/). |
20 | | - |
| 3 | +`sclblpy` is the core python package provided by Scailable for interacting with our API. |
21 | 4 |
|
22 | 5 | `sclblpy` is only functional in combination with a valid Scailable user account. |
23 | 6 |
|
24 | 7 | - **Website:** [https://www.scailable.net](https://www.scailable.net) |
25 | 8 | - **Docs:** |
26 | 9 | - On GitHub (you are here): [https://github.com/scailable/sclblpy](https://github.com/scailable/sclblpy/blob/master/README.md) |
27 | | - - On pypi: [https://docs.sclbl.net/sclblpy](https://docs.sclbl.net/sclblpy) |
28 | 10 | - API docs Scailable: [https://docs.sclbl.net](https://docs.sclbl.net) |
29 | | -- **Get an account:** [https://admin.sclbl.net/signup.html](https://admin.sclbl.net/signup.html) |
30 | | -- **Install the AI manager:** |
31 | | - - On any Linux device: [https://github.com/scailable/sclbl-tutorials/tree/master/solutions-manuals/sclbl-local-ai-manager](https://github.com/scailable/sclbl-tutorials/tree/master/solutions-manuals/sclbl-local-ai-manager) |
32 | | - - On an Advantech device: [https://github.com/scailable/sclbl-tutorials/tree/master/solutions-manuals](https://github.com/scailable/sclbl-tutorials/tree/master/solutions-manuals) |
33 | 11 |
|
34 | | -## Background |
35 | | -The sclblpy package allows users with a valid scailable account (get one at [https://admin.sclbl.net/signup.html](https://admin.sclbl.net/signup.html)) |
36 | | -to upload fitted ML / AI models to the Scailable toolchain server. This will result in: |
| 12 | +The scope of this package is: |
| 13 | + |
| 14 | +## Manage scailable account (auth.py) |
| 15 | +1. Create an account in the scailable plateform |
| 16 | +2. Sign in to an existing account |
| 17 | +3. Reset password or set a new password |
| 18 | +4. Get account information |
| 19 | + |
| 20 | +## Manage models (compute.py) |
| 21 | +1. Upload an `.onnx` model to the Scailable |
| 22 | +2. Update an existing model |
| 23 | +3. Delete a model |
| 24 | +4. Get model's details |
| 25 | +5. Get all the models accessible to your organisation. |
| 26 | +6. Add a catalogue (set of models) to your organisation |
| 27 | +7. Update an existing catalogue |
| 28 | +8. Delete a catalogue |
| 29 | +9. Get catalogue's details |
| 30 | +10. Get all the catalogues accessible for your organisation. |
| 31 | +11. Get the configured parameters for service |
| 32 | +12. Get statistics of all models for your organisation |
| 33 | + |
| 34 | +## Manage devices (device.py) |
| 35 | +1. Add a device to your organisation |
| 36 | +2. Update a device |
| 37 | +3. Get device's details |
| 38 | +4. Assign a model to a device |
| 39 | +5. Delete a device |
| 40 | +6. Get devices statistics |
| 41 | +7. Get all devices accessible for your organisation |
| 42 | +8. Get all groups accessible for your organisation |
| 43 | +9. Delete a group |
| 44 | +10. Add a device to a group |
| 45 | +11. Delete a device from a group |
37 | 46 |
|
38 | | -1. The model being tested for errors on the client side. |
39 | | -2. The model being uploaded to Scailable, tested again, and if all test pass it will be converted to [WebAssembly](https://webassembly.org). |
40 | | -3. The model being made available to deploy to a preregistered (link on how to do that) device |
41 | | -4. (upcoming) The model being testable through test_run(example_input, device_id) |
42 | 47 |
|
43 | 48 | ## Getting started |
44 | 49 |
|
45 | | -The following functions are likely most used: |
| 50 | +### Get a Scailable account |
46 | 51 |
|
47 | | -### `sp.upload_onnx()` can be used to create model: |
| 52 | +To create a Scailable account you can use `register_()` function or sign up at [https://admin.sclbl.net/register](https://admin.sclbl.net/register) |
48 | 53 | ```python |
49 | | -def upload_onnx(path, example="", docs={}, email=True) -> bool: |
50 | | - """upload_onnx uploads a fitted onnx model to Scailable. |
51 | | -
|
52 | | - - The function first checks if the supplied path indeed references a .onnx file |
53 | | - - Next, the docs are checked; if none are provided a warning is issued and a simple |
54 | | - name is provided based on the model type. |
55 | | - - Finally the onnx file and the supporting docs are uploaded to the toolchain. |
56 | | -
|
57 | | - Note: This method prints user-feedback by default. This feedback can be suppressed by calling the |
58 | | - stop_print() method. |
59 | 54 |
|
60 | | - Args: |
61 | | - path: The path referencing the onnx model location (i.e., the .onnx file location). |
62 | | - example: String example input for the onnx file. |
63 | | - docs: A dict{} containing the fields 'name' and 'documentation'. |
64 | | - email: Bool indicating whether a confirmation email of a successful conversion should be send. Default True. |
65 | | -
|
66 | | - Returns: |
67 | | - False if upload failed, true otherwise |
| 55 | +def register_(name: str, company: str, email: str, password: str, job_title: str, |
| 56 | + phone_number: str, newsletter_optIn: bool = True, accept_eula: bool = True) -> bool: |
| 57 | + """Performs the sign-up of a user. |
| 58 | + The function register performs a sign-up of a new user. |
| 59 | + It returns a boolean value indicating whether the sign-up was successful. |
| 60 | + Args: |
| 61 | + name: A string (name of the user) |
| 62 | + company: A string (name of the user's company) |
| 63 | + email: A string (email) |
| 64 | + password: A string (password) |
| 65 | + job_title: A string (job of the user) |
| 66 | + phone_number: A string (user's phone number) |
| 67 | + newsletter_optIn: Bool (whether to add this email address to the newsletter) |
| 68 | + accept_eula: Bool (Agreement to the EULA) |
| 69 | +
|
| 70 | + Returns: |
| 71 | + True if sign-up is successful. |
| 72 | + """ |
| 73 | +``` |
68 | 74 |
|
69 | | - Raises (in debug mode): |
70 | | - UploadModelError if unable to successfully bundle and upload the model. |
| 75 | +You already have a Scailable account? then you can sign in to your account using `log_in` function. |
| 76 | +```python |
| 77 | +def log_in(email: str, password: str) -> bool: |
| 78 | + """Performs the sign in of a user. |
| 79 | + The function _log_in performs a log in of a user based |
| 80 | + on the email (str) and password (str). It returns |
| 81 | + a boolean value indicating whether the log in was successful. |
| 82 | + Args: |
| 83 | + email: A string (email) to log in the user |
| 84 | + password: A string (password) for login |
| 85 | + Returns: |
| 86 | + True if sign in is successful. |
71 | 87 | """ |
72 | 88 | ``` |
73 | | -`sp.models()` lists all created models, and `sp.delete_model()` can be used to delete a model. Finally, `sp.update()` can be used to |
74 | | -overwrite / update an existing model. |
75 | 89 |
|
76 | | -### `sp.assign()` can be used to assign a model to a device: |
77 | | -```python |
78 | | -def assign(cfid, did, rid, _verbose=True): |
79 | | - """ Assign a model to a device. |
| 90 | +The following functionalities are likely most used: |
80 | 91 |
|
81 | | - Using the global JWT string this function assigns a model (using its cfid) to a device (using its did). |
| 92 | +### Upload an `onnx` model |
82 | 93 |
|
83 | | - Args: |
84 | | - cfid: String identifying the model / compute-function |
85 | | - did: String identifying the device |
86 | | - rid: String identifying the registration ID of the device (not, run "devices" |
| 94 | +To upload your models to Scailable, you can use `_upload_model`. (only models with onnx format are accepted) |
87 | 95 |
|
88 | | - Returns: |
89 | | - Boolean indicating whether the assignment was successful. |
| 96 | +```python |
| 97 | +def upload_model(name: str, documentation: str, input_driver: str = "", input_driver_details: dict = {}, |
| 98 | + output_driver: str = "", output_driver_details: dict = {}, |
| 99 | + alias: str = "", path: str = "", source_name=None, source_url=None) -> bool: |
| 100 | + """_upload_model uploads a fitted onnx model to Scailable. |
| 101 | +
|
| 102 | + - The function first checks if the supplied path indeed references a .onnx file |
| 103 | + - Next the onnx file and the supporting docs are uploaded. |
| 104 | +
|
| 105 | + Args: |
| 106 | + name: Name of the model to upload |
| 107 | + input_driver: |
| 108 | + input_driver_details: |
| 109 | + output_driver: |
| 110 | + output_driver_details: |
| 111 | + alias: Alias of the model |
| 112 | + documentation: Documentation of the model |
| 113 | + path: The path referencing the onnx model location (i.e., the .onnx file location). |
| 114 | + source_name: |
| 115 | + source_url: |
| 116 | + Returns: |
| 117 | + False if upload failed, True otherwise |
90 | 118 | """ |
91 | | -``` |
92 | | -`sp.assignments()` can be used to list current assignments, whereas `sp.delete_assignment()` can be used to delete an assignment. |
93 | | - |
| 119 | +``` |
94 | 120 |
|
95 | | -## A simple `.onnx` example |
| 121 | +### Add a device |
96 | 122 |
|
97 | | -The following code can be used to upload a stored `.onnx` model: |
| 123 | +To add a device to Scailable, you can use `add_device`. |
98 | 124 | ```python |
99 | | - |
100 | | -# Add docs |
101 | | -docs = {} |
102 | | -docs['name'] = "Name of ONNX model" |
103 | | -docs['documentation'] = "A long .md thing...." |
104 | | -check = upload_onnx("PATH-TO-MODEL/FILE-NAME.onnx", "", docs, email=True) |
| 125 | +def add_device(name: str, registration_token: str = "", runtime: str = "", serial: str = "", type: str = "") -> bool: |
| 126 | + """Add device to user's organisation. |
| 127 | + Args: |
| 128 | + name: Name of the model to upload |
| 129 | + registration_token: |
| 130 | + runtime: |
| 131 | + serial: |
| 132 | + type: |
| 133 | + Returns: |
| 134 | + True if device added, False otherwise |
| 135 | + """ |
105 | 136 | ``` |
106 | 137 |
|
107 | | -Note that the file will be send to the Scailable platform; after it has been transpiled to WebAssembly you will receive an email (by default), but you can choose not to, by setting ``email=False`` |
108 | | - |
109 | | -## Additional functionality |
110 | | -Next to the main ``upload_onnx()`` function, the package also exposes the following functions to administer endpoints: |
| 138 | +### Assign a model to a device |
111 | 139 |
|
112 | | -```` |
113 | | -# List all models owned by the current user: |
114 | | -sp.models() |
115 | | -
|
116 | | -# Remove an endpoint: |
117 | | -sp.delete_models(cfid) # Where cfid is the compute function id |
118 | | -
|
119 | | -# Update an existing endpoint: |
120 | | -sp.update(mod, fv, cfid, docs) # Where cfid is the compute function id |
121 | | -
|
122 | | -# Update an existing ednpoint without updating the docs: |
123 | | -sp.update(mod, fv, cfid) |
124 | | -
|
125 | | -# Update only the docs of an existing endpoint: |
126 | | -sp.update_docs(cfid, docs) |
127 | | -
|
128 | | -# See all devices: |
129 | | -sp.devices(offset=0, limit=20, _verbose=True, _return=False) |
130 | | -
|
131 | | -# Delete device: |
132 | | -sp.delete_device(did) |
133 | | -
|
134 | | -# See all assignments: |
135 | | -sp.assignments(offset=0, limit=20, _verbose=True, _return=False) |
136 | | -
|
137 | | -# Create an assignment: |
138 | | -sp.assign(cfid, did, rid, _verbose=True) |
139 | | -
|
140 | | -# Remove an assignment: |
141 | | -sp.delete_assignment(aid) |
142 | | -```` |
143 | | - |
144 | | -Additionally, the following methods are available: |
145 | | - |
146 | | -```` |
147 | | -# List all models currently supported by our toolchains: |
148 | | -sp.list_models() |
149 | | -
|
150 | | -# Prevent any user feedback from being printed: |
151 | | -sp.stop_print() |
152 | | -
|
153 | | -# Turn user feedback back on: |
154 | | -sp.start_print() |
155 | | -
|
156 | | -# Remove locally stored user credentials: |
157 | | -sp.remove_credentials() |
158 | | -
|
159 | | -```` |
160 | | - |
161 | | -## Supported `.onnx` files |
162 | | - |
163 | | -We currently support `ONNX` version 1.8 (and below) in full. However, if you encounter any problems |
164 | | -converting `.onnx` files please let us know. |
165 | | - |
166 | | -## Dependencies |
| 140 | +To assign a model to your device, you can use `_assign_model_to_device`. |
| 141 | +```python |
| 142 | +def assign_model_to_device(uuid: str, function_uuid: str) -> bool: |
| 143 | + """Assign a model to a device. |
| 144 | + Args: |
| 145 | + uuid: UUID of the device |
| 146 | +
|
| 147 | + Returns: |
| 148 | + True if model assigned, False otherwise |
| 149 | + """ |
| 150 | +``` |
167 | 151 |
|
168 | | -sclblpy needs python 3, and has been tested on python `> 3.7`. Furthermore, dependent on usage, sclblpy will import |
169 | | -the following packages: |
170 | 152 |
|
171 | | -* `numpy` |
172 | | -* `requests` |
173 | | -* `uuid` |
174 | 153 |
|
175 | | -No `onnx` packages are |
176 | | -necessary for the `sclblpy` package to run. |
177 | 154 |
|
178 | | -## Notes: |
179 | 155 |
|
180 | | -* We try to stick to the naming conventions in [http://google.github.io/styleguide/pyguide.html](http://google.github.io/styleguide/pyguide.html). |
181 | | -* The methods `_set_toolchain_URL(string)` and `_set_usermanager_URL(string)` can be used to change the default location of |
182 | | -the toolchain and user-management function. These are useful when running the Scailable stack locally. Also the method `_toggle_debug_mode()` can |
183 | | -be used for troubleshooting (this will raise exceptions and provide a trace upon errors). |
184 | | -* Docs generated using `pdoc3 --force --html --output-dir docs sclblpy/main.py` |
185 | | -* We are actively developing our stack; we try to list changes from one version to the next as clearly as possible. If you find any errors or issues please add an issue to this repo." |
186 | | -If you are having trouble using the `sclblpy` package, please [submit an issue to our github](https://github.com/scailable/sclblpy/issues/new), |
187 | | -we will try to fix it as quickly as possible! |
0 commit comments