IRIS Classifier Demo

The example application trains a small neural network on the device using Anderson’s Iris data set for iris flower type classification. For a more indepth look at optimizing android for DL4J, please see the Prerequisites and Configuration documentation here. This application has a simple UI to take measurements of petal length, petal width, sepal length, and sepal width from the user and returns the probability that the measurements belong to one of three types of Iris (Iris serosa, Iris versicolor, and Iris virginica). A data set includes 150 measurement values (50 for each iris type) and training the model takes anywhere from 5-20 seconds, depending on the device.

Contents

DL4JIrisClassifierDemo

Setting the Dependencies

Deeplearning4J applications require several dependencies in the build.gradle file. The Deeplearning library in turn depends on the libraries of ND4J and OpenBLAS, thus these must also be added to the dependencies declaration. Starting with Android Studio 3.0, annotationProcessors need to be defined as well, requiring dependencies for -x86 or -arm processors.

  1. compile (group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta4') {
  2.     exclude group: 'org.bytedeco.javacpp-presets', module: 'opencv-platform'
  3.     exclude group: 'org.bytedeco.javacpp-presets', module: 'leptonica-platform'
  4.     exclude group: 'org.bytedeco.javacpp-presets', module: 'hdf5-platform'
  5.     exclude group: 'org.nd4j', module: 'nd4j-base64'
  6. }
  7. compile group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta4'
  8. compile group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta4', classifier: "android-arm"
  9. compile group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta4', classifier: "android-arm64"
  10. compile group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta4', classifier: "android-x86"
  11. compile group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta4', classifier: "android-x86_64"
  12. compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3'
  13. compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-arm"
  14. compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-arm64"
  15. compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-x86"
  16. compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.3.3-1.4.3', classifier: "android-x86_64"
  17. compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3'
  18. compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3', classifier: "android-arm"
  19. compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3', classifier: "android-arm64"
  20. compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3', classifier: "android-x86"
  21. compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.3-1.4.3', classifier: "android-x86_64"
  22. compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3'
  23. compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3', classifier: "android-arm"
  24. compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3', classifier: "android-arm64"
  25. compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3', classifier: "android-x86"
  26. compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.76.0-1.4.3', classifier: "android-x86_64"

Compiling these dependencies involves a large number of files, thus it is necessary to set multiDexEnabled to true in defaultConfig.

  1. multiDexEnabled true

Finally, a conflict in the junit module versions will likely throw the following error: > Conflict with dependency ‘junit:junit’ in project ‘:app’. Resolved versions for app (4.8.2) and test app (4.12) differ.This can be suppressed by forcing all of the junit modules to use the same version.

  1. configurations.all {
  2.     resolutionStrategy.force 'junit:junit:4.12'
  3. }

Setting up the neural network on a background thread

Training even a simple neural network like in this example requires a significant amount of processor power, which is in limited supply on mobile devices. Thus, it is imperative that a background thread be used for the building and training of the neural network which then returns the output to the main thread for updating the UI. In this example we will be using an AsyncTask which accepts the input measurements from the UI and passes them as type double to the doInBackground() method. First, lets get references to the editTexts in the UI layout that accept the iris measurements inside of our onCreate method. Then an onClickListener will execute our asyncTask, pass it the measurements entered by the user, and show a progress bar until we hide it again in onPostExecute().

  1. public class MainActivity extends AppCompatActivity {
  4. @Override
  5.     public void onCreate(Bundle savedInstanceState) {
  6.         super.onCreate(savedInstanceState);
  7.         setContentView(R.layout.activity_main);
  9.         //get references to the editTexts that take the measurements
  10.         final EditText PL = (EditText) findViewById(R.id.editText);
  11.         final EditText PW = (EditText) findViewById(R.id.editText2);
  12.         final EditText SL = (EditText) findViewById(R.id.editText3);
  13.         final EditText SW = (EditText) findViewById(R.id.editText4);
  15.       //onclick to capture the input and launch the asyncTask
  16.         Button button = (Button) findViewById(R.id.button);
  18.         button.setOnClickListener(new View.OnClickListener() {
  19.             @Override
  20.             public void onClick(View v) {
  22.                 final double pl = Double.parseDouble(PL.getText().toString());
  23.                 final double pw = Double.parseDouble(PW.getText().toString());
  24.                 final double sl = Double.parseDouble(SL.getText().toString());
  25.                 final double sw = Double.parseDouble(SW.getText().toString());
  27.                 AsyncTaskRunner runner = new AsyncTaskRunner();
  29.            //pass the measurement as params to the AsyncTask
  30.                 runner.execute(pl,pw,sl,sw);
  32.                 ProgressBar bar = (ProgressBar) findViewById(R.id.progressBar);
  33.                 bar.setVisibility(View.VISIBLE);
  34.             }
  35.         });
  36.         }

Now let’s write our AsyncTask<Params, Progress, Results>. The AsyncTask needs to have a Params of type Double to receive the decimal value measurements from the UI. The Result type is set to INDArray, which is returned from the doInBackground() Method and passed to the onPostExecute() method for updating the UI. NDArrays are provided by the ND4J library and are essentially n-dimensional arrays with a given number of dimensions. For more on NDArrays, see https://nd4j.org/userguide.

  1. private class AsyncTaskRunner extends AsyncTask<Double, Integer, INDArray> {
  3.     // Runs in UI before background thread is called
  4.     @Override
  5.     protected void onPreExecute() {
  6.         super.onPreExecute();
  8.         ProgressBar bar = (ProgressBar) findViewById(R.id.progressBar);
  9.         bar.setVisibility(View.INVISIBLE);
  10.     }

Preparing the training data set and user input

The doInBackground() method will handle the formatting of the training data, the construction of the neural net, the training of the net, and the analysis of the input data by the trained model. The user input has only 4 values, thus we can add those directly to a 1x4 INDArray using the putScalar() method. The training data is much larger and must be converted from CSV lists to matrices through an iterative for loop. The training data is stored in the app as two arrays, one for the Iris measurements named irisData which contains a list of 150 iris measurements and another for the labels of iris type named labelData. These will be transformed to 150x4 and 150x3 matrices, respectively, so that they can be converted into INDArray objects that the neural network will use for training.

  1.     // This is our main background thread for the neural net
  2.     @Override
  3.     protected String doInBackground(Double... params) {
  4.     //Get the doubles from params, which is an array so they will be 0,1,2,3
  5.         double pld = params[0];
  6.         double pwd = params[1];
  7.         double sld = params[2];
  8.         double swd = params[3];
  10.         //Create input INDArray for the user measurements
  11.         INDArray actualInput = Nd4j.zeros(1,4);
  12.         actualInput.putScalar(new int[]{0,0}, pld);
  13.         actualInput.putScalar(new int[]{0,1}, pwd);
  14.         actualInput.putScalar(new int[]{0,2}, sld);
  15.         actualInput.putScalar(new int[]{0,3}, swd);
  17.         //Convert the iris data into 150x4 matrix
  18.         int row=150;
  19.         int col=4;
  20.         double[][] irisMatrix=new double[row][col];
  21.         int i = 0;
  22.         for(int r=0; r<row; r++){
  23.             for( int c=0; c<col; c++){
  24.         irisMatrix[r][c]=com.example.jmerwin.irisclassifier.DataSet.irisData[i++];
  25.             }
  26.         }
  28.         //Now do the same for the label data
  29.         int rowLabel=150;
  30.         int colLabel=3;
  31.         double[][] twodimLabel=new double[rowLabel][colLabel];
  32.         int ii = 0;
  33.         for(int r=0; r<rowLabel; r++){
  34.             for( int c=0; c<colLabel; c++){
  35.                 twodimLabel[r][c]=com.example.jmerwin.irisclassifier.DataSet.labelData[ii++];
  36.             }
  37.         }
  39.         //Converting the data matrices into training INDArrays is straight forward
  40.         INDArray trainingIn = Nd4j.create(irisMatrix);
  41.         INDArray trainingOut = Nd4j.create(twodimLabel);

Building and Training the Neural Network

Now that our data is ready, we can build a simple multi-layer perceptron with a single hidden layer. The DenseLayer class is used to create the input layer and the hidden layer of the network while the OutputLayer class is used for the Output layer. The number of columns in the input INDArray must equal to the number of neurons in the input layer (nIn). The number of neurons in the hidden layer input must equal the number inputLayer’s output array (nOut). Finally, the outputLayer input should match the hiddenLayer output. The output must equal the number of possible classifications, which is 3.

  1.     //define the layers of the network
  2.     DenseLayer inputLayer = new DenseLayer.Builder()
  3.             .nIn(4)
  4.             .nOut(3)
  5.             .name("Input")
  6.             .build();
  8.     DenseLayer hiddenLayer = new DenseLayer.Builder()
  9.             .nIn(3)
  10.             .nOut(3)
  11.             .name("Hidden")
  12.             .build();
  14.     OutputLayer outputLayer = new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
  15.             .nIn(3)
  16.             .nOut(3)
  17.             .name("Output")
  18.             .activation(Activation.SOFTMAX)
  19.             .build();

The next step is to build the neural network using nccBuilder. The parameters selected below for training are standard. To learn more about optimizing network training, see deeplearning4j.org.

  1.     NeuralNetConfiguration.Builder nncBuilder = new NeuralNetConfiguration.Builder();
  2.     long seed = 6;
  3.     nncBuilder.seed(seed);
  4.     nncBuilder.activation(Activation.TANH);
  5.     nncBuilder.weightInit(WeightInit.XAVIER);
  7.     NeuralNetConfiguration.ListBuilder listBuilder = nncBuilder.list();
  8.     listBuilder.layer(0, inputLayer);
  9.     listBuilder.layer(1, hiddenLayer);
  10.     listBuilder.layer(2, outputLayer);
  12.     listBuilder.backprop(true);
  14.     MultiLayerNetwork myNetwork = new MultiLayerNetwork(listBuilder.build());
  15.     myNetwork.init();
  17.     //Create a data set from the INDArrays and train the network
  18.     DataSet myData = new DataSet(trainingIn, trainingOut);
  19.     for(int l=0; l<=1000; l++) {
  20.     myNetwork.fit(myData);
  21.     }
  23.     //Evaluate the input data against the model
  24.     INDArray actualOutput = myNetwork.output(actualInput);
  25.     Log.d("myNetwork Output ", actualOutput.toString());
  27.     //Here we return the INDArray to onPostExecute where it can be 
  28.     //used to update the UI
  29.     return actualOutput;
  30. }

Updating the UI

Once the training of the neural network and the classification of the user measurements are complete, the doInBackground() method will finish and onPostExecute() will have access to the main thread and UI, allowing us to update the UI with the classification results. Note that the decimal places reported on the probabilities can be controlled by setting a DecimalFormat pattern.

  1. //This is where we update the UI with our classification results
  2.     @Override
  3.     protected void onPostExecute(INDArray result) {
  4.         super.onPostExecute(result);
  6.     //Hide the progress bar now that we are finished
  7.     ProgressBar bar = (ProgressBar) findViewById(R.id.progressBar);
  8.     bar.setVisibility(View.INVISIBLE);
  10.     //Retrieve the three probabilities
  11.     Double first = result.getDouble(0,0);
  12.     Double second = result.getDouble(0,1);
  13.     Double third = result.getDouble(0,2);
  15.     //Update the UI with output
  16.     TextView setosa = (TextView) findViewById(R.id.textView11);
  17.     TextView versicolor = (TextView) findViewById(R.id.textView12);
  18.     TextView virginica = (TextView) findViewById(R.id.textView13);
  20.     //Limit the double to values to two decimals using DecimalFormat
  21.     DecimalFormat df2 = new DecimalFormat(".##");
  23.     //Set the text of the textViews in UI to show the probabilites
  24.     setosa.setText(String.valueOf(df2.format(first)));
  25.     versicolor.setText(String.valueOf(df2.format(second)));
  26.     virginica.setText(String.valueOf(df2.format(third)));
  28.     }

Conclusion

Hopefully this tutorial has illustrated how the compatibility of DL4J with Android makes it easy to build, train, and evaluate neural networks on mobile devices. We used a simple UI to take input values from the measurement and then passed them as the Params in an AsyncTask. The processor intensive steps of data preparation, network layer building, model training, and evaluation of the user data were all performed in the doInBackground() method of the background thread, maintaining a stable and responsive device. Once completed, we passed the output INDArray as the AsyncTask Results to onPostExecute() where the UI was updated to demonstrate the classification results.The limitations of processing power and battery life of mobile devices make training robust, multi-layer networks somewhat unfeasible. To address this limitation, we will next look at an example Android application that saves the trained model on the device for faster performance after an initial model training.

The complete code for this example is available here.