Fails with latest models

[fenollp]

Out of the box everything works, albeit slowly (2-3s per inference).

However using the models at https://github.com/hpi-xnor/BMXNet/tree/d5d8c488ba06c3eaafd04039640f960e2f02e0de/smd_hpi/binary_models
for instance with binarized_imagenet-resnet18-64bit I get:

ios-image-classification.git/predict-ios/MXNet/mxnet_predict-all.cc:29091: Check failed: from.shape() == to->shape() operands shape mismatchfrom.shape = (576,) to.shape=(1152,)

Stack trace returned 26 entries:
[bt] (0) 0   predict-ios                         0x00000001001a17c8 _ZN4dmlc15LogMessageFatalD2Ev + 60
[bt] (1) 1   predict-ios                         0x0000000100053c58 _ZN4dmlc15LogMessageFatalD1Ev + 28
[bt] (2) 2   predict-ios                         0x00000001000ab7ac _ZN5mxnet10CopyFromToERKNS_7NDArrayEPS0_i + 388
[bt] (3) 3   predict-ios                         0x000000010014d810 MXPredCreatePartialOut + 21028
[bt] (4) 4   predict-ios                         0x00000001001485e0 MXPredCreate + 128
[bt] (5) 5   predict-ios                         0x0000000100c10d28 -[ViewController viewDidLoad] + 1648
[bt] (6) 6   UIKit                               0x0000000191babec0 <redacted> + 1036
[bt] (7) 7   UIKit                               0x0000000191baba9c <redacted> + 28
[bt] (8) 8   UIKit                               0x0000000191bb231c <redacted> + 76
[bt] (9) 9   UIKit                               0x0000000191baf7b8 <redacted> + 272
[bt] (10) 10  UIKit                               0x0000000191c21224 <redacted> + 48
[bt] (11) 11  UIKit                               0x0000000191e297ec <redacted> + 3632
[bt] (12) 12  UIKit                               0x0000000191e2f1e0 <redacted> + 1684
[bt] (13) 13  UIKit                               0x0000000191e43d18 <redacted> + 48
[bt] (14) 14  UIKit                               0x0000000191e2c474 <redacted> + 168
[bt] (15) 15  FrontBoardServices                  0x000000018d623884 <redacted> + 36
[bt] (16) 16  FrontBoardServices                  0x000000018d6236f0 <redacted> + 176
[bt] (17) 17  FrontBoardServices                  0x000000018d623aa0 <redacted> + 56
[bt] (18) 18  CoreFoundation                      0x000000018ba2942c <redacted> + 24
[bt] (19) 19  CoreFoundation                      0x000000018ba28d9c <redacted> + 540
[bt] (20) 20  CoreFoundation                      0x000000018ba269a8 <redacted> + 744
[bt] (21) 21  CoreFoundation                      0x000000018b956da4 CFRunLoopRunSpecific + 424
[bt] (22) 22  UIKit                               0x0000000191c16384 <redacted> + 652
[bt] (23) 23  UIKit                               0x0000000191c11058 UIApplicationMain + 208
[bt] (24) 24  predict-ios                         0x0000000100c15fdc main + 136
[bt] (25) 25  libdyld.dylib                       0x000000018a96559c <redacted> + 4

2017-06-02 19:53:29.934139-0700 predict-ios[8531:1238132] [MC] System group container for systemgroup.com.apple.configurationprofiles path is /private/var/containers/Shared/SystemGroup/systemgroup.com.apple.configurationprofiles
2017-06-02 19:53:29.935249-0700 predict-ios[8531:1238132] [MC] Reading from public effective user settings.
(lldb) 

Is this due to the mxnet_predict-all.cc file being too old? I am not sure how to generate a new one.
Is this because MXNet and BMXNet use different params formats?
Any ideas on how I could solve this?

[mtin]

Hey, which iPhone model are you using? (out of curiosity regarding your measured speed)

The included mxnet should be sufficient - we have two different formats of packing the params though: one works on 32 bit wide variables and the other on 64 bit. You are using a model trained with 64 bit (see the name) with an app compiled to use 32 bit.

Check your xcode build settings -> preprocessor macros -> change BINARY_WORD_32 to 0 and BINARY_WORD_64 to 1

[fenollp]

Okay this fixed it! Had to clean the whole project cause Xcode doesn't seem to understand what macros are...

I ran the code on an iPhone 6S. After enabling release mode I could get about 2 FPS.
Note though that just using https://github.com/hollance/Forge I get about 5 FPS.

Is 2 FPS the kind of speed I should be expecting or is there something wrong here?

Thanks!

[fenollp]

@mtin what kind of speed do you get & on which phones? Do you have numbers for the OnePlus3?