I am searching for a neural network sample code in OpenCL, that I might optimize using GPU kernels. Please help me as I am a beginner in OpenCL.
I've started writing a deep convolutional neural network library for OpenCL at https://github.com/hughperkins/ClConvolve/tree/master . So far, it supports:
convolutional layers
max-pooling
softmax
random translations layer
random patches layer
normalization layer
multinet (aka 'multi-column', per http://arxiv.org/pdf/1202.2745.pdf )
fully-connected
... running on the GPU, using OpenCL. You can specify the network architecture on the commandline, like 100C5-MP2-100C5-MP2-100C4-MP2-300N-100N-6N.
Edit: can get 99.55% test accuracy on MNIST now :-)
You could look into this OpenCL enhanced version of FANN library for fast neural networks.
I am sure it will need some optimization :-)
I also plan to optimize THIS lib.
http://leenissen.dk/fann/wp/2011/02/opencl-enhanced-version-of-fann/
Related
I would appreciate clarity around hardware acceleration supported for ML Kit. Some applications make an explicit mention that models are run on CPU, implying that there can be other modes of acceleration. Using GPU via something like OpenCL seems like a natural way of doing so.
I wish to know if Google is capable and willing of mastering OpenCL for Machine Learning applications.
Currently, ML Kit features are all running in CPU to be compatible with all devices. We are adding GPU / NNAPI support for features, and will update them in future releases.
I use R, but lately are trying to implement Model Predictive Control as applied in Control Engineering on my models, but it seems that this area in R is non-existent as compared to Matlab where it is quite easy to do System Identification and create transfer functions which can be deployed inside the Model Predictive Control module. Anybody knows where to look, or which packages to use for MPC and transfer functions in R to build MPC models?
There are a growing number of Python packages for control engineering. One option is to use the reticulate R package to interface to Python functions. Here are some Control Engineering packages in Python:
SciPy.signal for signal processing and system modeling
SymPy for Laplace transforms and differential equation analytic solutions
Control Systems Library
Chemical Process Control from Jeff Kantor, Notre Dame
Process Dynamics and Control in Python at BYU (my course)
Advanced Control and Machine Learning in Python at BYU (also my course)
Here is an example of running MPC in Python on the Temperature Control Lab.
Is it possible to run OpenCL on a system designed by a user on a SoC prototyping board? To be more specific, I have a ZedBoard (Xilinx Zynq) that has Dual ARM cores and a Programmable Logic (PL) Area. If I design a simple system of my own that has a video processing accelerator implemented in the logic area, an ARM core and an AXI interconnect, what do I have to do to provide OpenCL support for this simple system? (In this simple system, the ARM core could be the "Host" and the video processing accelerator could be the "device").
I am a student and I have only some basic knowledge about OpenCL. I have researched about my question and have only ended up confusing myself. What are the things that have to be done to provide OpenCL support for a SoC? I understand that this may be a big project, but I need a guideline where to start and how to proceed.
what do I have to do to provide OpenCL support for this simple system?
Implement a OpenCL platform that makes either use of your ARM CPU or the FPGA (or both). I'd say that is pretty much impossible for you; ARM would surely offer one for the CPU if it was easy (and they definitely have the financial means to employ capable engineers/computer scientists), and implementing accelerators on an FPGA requires in-depth knowledge of FPGA development, as well as compiler theory and experience in systems design. I don't want to sound mean, but you seem to have none of these three.
You asked where to get started; I recommend just writing a first accelerator that e.g. adds up a vector of numbers; as soon as you have that, you will have a clearer idea of your task.
If you want to have a look at a reference: The Ettus USRP E310 is a zynq-based SDR device. Ettus has a technology called RFNoC, which allows users to write their own blocks to push data through. Notice that this took quite a few engineers and quite some time to get started. Notice further that it's much easier than implementing something that converts OpenCL to FPGA implementations.
If you have access to the Xilinx tools: Vivado HLS 15.1 System Edition should compile OpenCL kernels. This will also be included in the SDAccel tool suite.
Source: UG973: Vivado Design Suite User Guide Release Notes, Installation,and Licensing
An alternative might be switching to Altera. They provide some good examples for the Altera Cyclone V SoC which is comparable to Xilinx Zynq devices (also includes ARM Cortex-A9) :
AlteraSDK for OpenCL
I am also a student and my current project is also going on a similar direction, i have successfully installed a version of opencl called POCL on the zedboard, it successfully detects the arm cpu of the zedboard. To install pocl, you need llvm and a horde of other things as well. but basic steps to get pocl up on the zedboard are given below:-
Installing pocl:
http://www.hosseinabady.com/install-pocl-opencl
running example:
http://www.hosseinabady.com/embedded-system-by-examples/opencl_embedded_system/opencl-vector-addition
Lots of dependency: can resolved easily
but LLVM make sure you install 3.4 version for pocl 0.9
Steps to install llvm
https://github.com/pacs-course/pacs/wiki/Instructions-to-install-clang-3.1-on-ubuntu-12.04.1-and-12.10
POCL 0.9 is successfully working for me, as you do the installation you will face many other missing dependencies like hwloc, mesa libraries, open gl/cl headers icd loaders i hope you can resolve them as its a very big list to put up in stack overflow.
In order to detect your fpga as an open cl device, thats not going to be a trivial thing to do, you can refer to this link question i posted on github
https://github.com/pocl/pocl/issues/285
and also a research paper published by hosseinbady found on the publications link on the pocl website
http://pocl.sourceforge.net/publications.html
hope this helps you
Try the ARM OpenCL SDK. The Zedboard has an ARM A9 CPU, this should have a NEON SIMD vector unit http://www.arm.com/products/processors/technologies/neon.php which can run OpenCL. See http://www.arm.com/products/multimedia/mali-technologies/opencl-for-neon.php.
The Zedboard isn't listed as an OpenCL conformant platform https://www.khronos.org/conformance/adopters/conformant-products#opencl.
So there is a chance the ARM driver will not work.
Good luck!
If still relevant, try this paper OpenCL on ZYNQ [PDF]
Also note that Zynq-7000 is listed on https://www.khronos.org/conformance/adopters/conformant-products#opencl ( OpenCL_1_0 ), hence the compatibility.
I have a script in R that uses loess to find a curve of best fit for some data, integrates that curve and then does some simple arithmitic from there to come up with a number. It is for a testing unit which is handheld, and therefore I need to run it on a microcomputer. There are ones out there with Arm processors which will run linux (520MHz Low-power ARM processor, 128 MB of RAM).
My question is will this have enough power to do the calculations, and will R even run on a system like this?
Any suggestions or insight would be greatly appreciated!
Thanks.
You need to have compatible C and FORTRAN compilers as well as the GNU OS and Linux kernel to build R from sources. The lack of compatible compilers is likely to doom this strategy. I suspect you will have better success using for code from loess and simpleLoess as a template for a compiled version in C.
As to your second question, R can be made to run on ARM processors. See, for example, http://maemo.org/packages/view/r-base-core/ which is a port to Maemo which is an operating system on ARM processors for handheld devices. It also can be compiled for Android: http://rwiki.sciviews.org/doku.php?id=getting-started:installation:android.
As #Carl said, the first question can't be answered without knowing more details.
I stumbled on this post depicting an Open Source HW/SW R-Based Graphing Calculator based on this motherboard!
HIH!
I do a lot of computational intelligence research. I have used Matlab almost exclusively as my programming medium for a decade or so. I am now trying to move to OSS. I have settled on R as my new environment.
After a long search for neural net software, the only Matlab-comparable OSS packages are Stuttgart NN and FANN (this can be debated another time =). The former doesn't appear to be maintained so I'd like to go with the latter. So my question is:
Does anyone have experience using R and FANN?
FANN has C++ bindings and R seems to have a couple of packages for a C++ interface, but since I'm a R newbie I need an idea of where exactly to start. Any guidance or recommendations would be appreciated.
Cheers.
I do not know anything abuot FANN but I can assure you that R has an actively maintained interface to the Stuttgart Neural Net Simulator (SNNS) library via the
RSNNS package --- as RSNNS happens to employ the
Rcpp package for interfacing R and C++ which I am involved in.