![]() But it’s probably safe to say that if a model has a better score on ImageNet, it will likely give better results on other tasks as well. We’ll use the ImageNet test set accuracy as an indicator of how good the model architecture is, but keep in mind this is only a proxy for the actual task. The feature extractor is the backbone that you build the rest of the model on top of, and the purpose of this blog post is to identify what the best backbone is to use these days on iOS - where “best” is a combination of size, speed, and prediction quality. Then you add some new layers on top and fine-tune these new layers on your own data. You take an existing model that was pre-trained on a popular generic dataset such as ImageNet or COCO, and use that as the feature extractor. Most of the time when building a new neural network, you’ll use transfer learning. This could be classification or another task such as object detection, pose estimation, segmentation, and so on. We’ll be evaluating how useful these models are as a feature extractor that is part of a larger neural network that performs some kind of computer vision task. However, that’s not the main reason we’re interested in these models - after all, not many apps will have a need for an ImageNet classifier… Usually these models are trained on ImageNet and we can tell how good they are by their classification accuracy on the ImageNet test set. Most of the models we’ll be covering are image classifiers. As some of these new architectures can rival the quality of larger models, switching to a smaller model will at least save some space (for example, ResNet-50 adds ~100 MB to the app). However, keep in mind that a large model also makes the app bundle a lot bigger. The model might take a second or so to compute the results, but that’s fine in such situations. Note: If your app only occasionally uses the neural network, you can get away with using a larger model such as ResNet-50. With this blog post I hope to make it easier to choose an appropriate architecture. The Neural Engine currently does not appear to support all of Core ML’s layer types. Not all models can (fully) run on the ANE.Conversely, if you’re primarily targeting iPhone X and earlier, it makes sense to pick a model that works well on the GPU, as these older devices do not have a Neural Engine (the X has an ANE but Core ML can’t use it).If you’re targeting only newer devices (iPhone XS and up), it might be smart to go for a neural network design that can take advantage of the ANE.To complicate matters, some model architectures work better on the GPU, while others are better on the Neural Engine (ANE): But there are many more choices now, so let’s see if we can do better. Previously, I would usually recommend MobileNet v1 or v2. Your job as mobile ML practitioner is to find a model architecture that is fast enough but also gives good enough results. That’s not a good user experience.Ī smaller model will run faster and use less battery power, but typically gives less accurate results. Large models quickly make the phone hot and drain the battery. But also, the slower it runs and the more energy it eats up. ![]() In general, the larger model is, the better results it gives. Of course, as with most things engineering, there is a trade-off: When putting a neural network into your app, it’s a good idea to use a model architecture that is fast and efficient.Įspecially if you want to make predictions often, for example on real-time video.Ī neural network such as ResNet-50 - which is a typical backbone model used in research papers - will use too much power and is unsuitable for real-time use. The story on Android may well be very different! Why should you care? Note: Since my focus is on iOS, we’ll be using Core ML and iPhone for the speed tests. Found an interesting architecture that isn’t covered here? Let me know and I’ll add it to the blog post. It’s possible I missed some architectures. In this blog post we’ll look at a number of these new neural network designs and do some speed measurements to see how fast they are.įirst we’ll revisit a few older architectures: But it’s 2020 and there are new kids on the block! I have previously written about MobileNet v1 and v2, and have used these models in many client projects. It seems that pretty much everyone has figured out now that large models such as VGG16 or ResNet-50 aren’t a good idea on small devices. ![]() Over the past 18 months or so, a number of new neural network achitectures were proposed specifically for use on mobile and edge devices. New mobile neural network architectures. ![]()
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |