3-3 High-level API: Demonstration

The examples below use high-level APIs in TensorFlow to implement a linear regression model and a DNN binary classification model.

Typically, the high-level APIs are providing the class interfaces for tf.keras.models.

There are three ways of modeling using APIs of Keras: sequential modeling using Sequential function, arbitrary modeling using API functions, and customized modeling by inheriting base class Model.

Here we are demonstrating using Sequential function and customized modeling by inheriting base class Model, respectively.

  1. import tensorflow as tf
  3. # Time stamp
  4. @tf.function
  5. def printbar():
  6.     today_ts = tf.timestamp()%(24*60*60)
  8.     hour = tf.cast(today_ts//3600+8,tf.int32)%tf.constant(24)
  9.     minite = tf.cast((today_ts%3600)//60,tf.int32)
  10.     second = tf.cast(tf.floor(today_ts%60),tf.int32)
  12.     def timeformat(m):
  13.         if tf.strings.length(tf.strings.format("{}",m))==1:
  14.             return(tf.strings.format("0{}",m))
  15.         else:
  16.             return(tf.strings.format("{}",m))
  18.     timestring = tf.strings.join([timeformat(hour),timeformat(minite),
  19.                 timeformat(second)],separator = ":")
  20.     tf.print("=========="*8+timestring)

1. Linear Regression Model

In this example, we used Sequential function to construct the model sequentially and use the pre-defined method model.fit for training (for the beginners).

(a) Data Preparation

  1. import numpy as np 
  2. import pandas as pd
  3. from matplotlib import pyplot as plt 
  4. import tensorflow as tf
  5. from tensorflow.keras import models,layers,losses,metrics,optimizers
  7. # Number of sample
  8. n = 400
  10. # Generating the datasets
  11. X = tf.random.uniform([n,2],minval=-10,maxval=10) 
  12. w0 = tf.constant([[2.0],[-3.0]])
  13. b0 = tf.constant([[3.0]])
  14. Y = X@w0 + b0 + tf.random.normal([n,1],mean = 0.0,stddev= 2.0)  # @ is matrix multiplication; adding Gaussian noise
  1. # Data Visualization
  3. %matplotlib inline
  4. %config InlineBackend.figure_format = 'svg'
  5. plt.figure(figsize = (12,5))
  6. ax1 = plt.subplot(121)
  7. ax1.scatter(X[:,0],Y[:,0], c = "b")
  8. plt.xlabel("x1")
  9. plt.ylabel("y",rotation = 0)
  11. ax2 = plt.subplot(122)
  12. ax2.scatter(X[:,1],Y[:,0], c = "g")
  13. plt.xlabel("x2")
  14. plt.ylabel("y",rotation = 0)
  15. plt.show()

3-3 High-level API: Demonstration - 图1

(b) Model Definition

  1. tf.keras.backend.clear_session()
  3. model = models.Sequential()
  4. model.add(layers.Dense(1,input_shape =(2,)))
  5. model.summary()
  1. Model: "sequential"
  2. _________________________________________________________________
  3. Layer (type)                 Output Shape              Param #   
  4. =================================================================
  5. dense (Dense)                (None, 1)                 3         
  6. =================================================================
  7. Total params: 3
  8. Trainable params: 3
  9. Non-trainable params: 0

(c) Model Training

  1. ### Training using method fit
  3. model.compile(optimizer="adam",loss="mse",metrics=["mae"])
  4. model.fit(X,Y,batch_size = 10,epochs = 200)  
  6. tf.print("w = ",model.layers[0].kernel)
  7. tf.print("b = ",model.layers[0].bias)
  1. Epoch 197/200
  2. 400/400 [==============================] - 0s 190us/sample - loss: 4.3977 - mae: 1.7129
  3. Epoch 198/200
  4. 400/400 [==============================] - 0s 172us/sample - loss: 4.3918 - mae: 1.7117
  5. Epoch 199/200
  6. 400/400 [==============================] - 0s 134us/sample - loss: 4.3861 - mae: 1.7106
  7. Epoch 200/200
  8. 400/400 [==============================] - 0s 166us/sample - loss: 4.3786 - mae: 1.7092
  9. w =  [[1.99339032]
  10.  [-3.00866461]]
  11. b =  [2.67018795]
  1. # Visualizing the results
  3. %matplotlib inline
  4. %config InlineBackend.figure_format = 'svg'
  6. w,b = model.variables
  8. plt.figure(figsize = (12,5))
  9. ax1 = plt.subplot(121)
  10. ax1.scatter(X[:,0],Y[:,0], c = "b",label = "samples")
  11. ax1.plot(X[:,0],w[0]*X[:,0]+b[0],"-r",linewidth = 5.0,label = "model")
  12. ax1.legend()
  13. plt.xlabel("x1")
  14. plt.ylabel("y",rotation = 0)
  16. ax2 = plt.subplot(122)
  17. ax2.scatter(X[:,1],Y[:,0], c = "g",label = "samples")
  18. ax2.plot(X[:,1],w[1]*X[:,1]+b[0],"-r",linewidth = 5.0,label = "model")
  19. ax2.legend()
  20. plt.xlabel("x2")
  21. plt.ylabel("y",rotation = 0)
  23. plt.show()

3-3 High-level API: Demonstration - 图2

2. DNN Binary Classification Model

This example demonstrates the customized model using the child class inherited from the base class Model, and use a customized loop for training (for the experts).

(a) Data Preparation

  1. import numpy as np 
  2. import pandas as pd 
  3. from matplotlib import pyplot as plt
  4. import tensorflow as tf
  5. from tensorflow.keras import layers,losses,metrics,optimizers
  6. %matplotlib inline
  7. %config InlineBackend.figure_format = 'svg'
  9. # Number of the positive/negative samples
  10. n_positive,n_negative = 2000,2000
  12. # Generating the positive samples with a distribution on a smaller ring
  13. r_p = 5.0 + tf.random.truncated_normal([n_positive,1],0.0,1.0)
  14. theta_p = tf.random.uniform([n_positive,1],0.0,2*np.pi) 
  15. Xp = tf.concat([r_p*tf.cos(theta_p),r_p*tf.sin(theta_p)],axis = 1)
  16. Yp = tf.ones_like(r_p)
  18. # Generating the negative samples with a distribution on a larger ring
  19. r_n = 8.0 + tf.random.truncated_normal([n_negative,1],0.0,1.0)
  20. theta_n = tf.random.uniform([n_negative,1],0.0,2*np.pi) 
  21. Xn = tf.concat([r_n*tf.cos(theta_n),r_n*tf.sin(theta_n)],axis = 1)
  22. Yn = tf.zeros_like(r_n)
  24. # Assembling all samples
  25. X = tf.concat([Xp,Xn],axis = 0)
  26. Y = tf.concat([Yp,Yn],axis = 0)
  28. # Shuffling the samples
  29. data = tf.concat([X,Y],axis = 1)
  30. data = tf.random.shuffle(data)
  31. X = data[:,:2]
  32. Y = data[:,2:]
  35. # Visualizing the data
  36. plt.figure(figsize = (6,6))
  37. plt.scatter(Xp[:,0].numpy(),Xp[:,1].numpy(),c = "r")
  38. plt.scatter(Xn[:,0].numpy(),Xn[:,1].numpy(),c = "g")
  39. plt.legend(["positive","negative"]);

3-3 High-level API: Demonstration - 图3

  1. ds_train = tf.data.Dataset.from_tensor_slices((X[0:n*3//4,:],Y[0:n*3//4,:])) \
  2.      .shuffle(buffer_size = 1000).batch(20) \
  3.      .prefetch(tf.data.experimental.AUTOTUNE) \
  4.      .cache()
  6. ds_valid = tf.data.Dataset.from_tensor_slices((X[n*3//4:,:],Y[n*3//4:,:])) \
  7.      .batch(20) \
  8.      .prefetch(tf.data.experimental.AUTOTUNE) \
  9.      .cache()

(b) Model Definition

  1. tf.keras.backend.clear_session()
  2. class DNNModel(models.Model):
  3.     def __init__(self):
  4.         super(DNNModel, self).__init__()
  6.     def build(self,input_shape):
  7.         self.dense1 = layers.Dense(4,activation = "relu",name = "dense1") 
  8.         self.dense2 = layers.Dense(8,activation = "relu",name = "dense2")
  9.         self.dense3 = layers.Dense(1,activation = "sigmoid",name = "dense3")
  10.         super(DNNModel,self).build(input_shape)
  12.     # Forward propagation
  13.     @tf.function(input_signature=[tf.TensorSpec(shape = [None,2], dtype = tf.float32)])  
  14.     def call(self,x):
  15.         x = self.dense1(x)
  16.         x = self.dense2(x)
  17.         y = self.dense3(x)
  18.         return y
  20. model = DNNModel()
  21. model.build(input_shape =(None,2))
  23. model.summary()
  1. Model: "dnn_model"
  2. _________________________________________________________________
  3. Layer (type)                 Output Shape              Param #   
  4. =================================================================
  5. dense1 (Dense)               multiple                  12        
  6. _________________________________________________________________
  7. dense2 (Dense)               multiple                  40        
  8. _________________________________________________________________
  9. dense3 (Dense)               multiple                  9         
  10. =================================================================
  11. Total params: 61
  12. Trainable params: 61
  13. Non-trainable params: 0
  14. _________________________________________________________________

(c) Model Training

  1. ### Customizing the training loop
  3. optimizer = optimizers.Adam(learning_rate=0.01)
  4. loss_func = tf.keras.losses.BinaryCrossentropy()
  6. train_loss = tf.keras.metrics.Mean(name='train_loss')
  7. train_metric = tf.keras.metrics.BinaryAccuracy(name='train_accuracy')
  9. valid_loss = tf.keras.metrics.Mean(name='valid_loss')
  10. valid_metric = tf.keras.metrics.BinaryAccuracy(name='valid_accuracy')
  13. @tf.function
  14. def train_step(model, features, labels):
  15.     with tf.GradientTape() as tape:
  16.         predictions = model(features)
  17.         loss = loss_func(labels, predictions)
  18.     grads = tape.gradient(loss, model.trainable_variables)
  19.     optimizer.apply_gradients(zip(grads, model.trainable_variables))
  21.     train_loss.update_state(loss)
  22.     train_metric.update_state(labels, predictions)
  24. @tf.function
  25. def valid_step(model, features, labels):
  26.     predictions = model(features)
  27.     batch_loss = loss_func(labels, predictions)
  28.     valid_loss.update_state(batch_loss)
  29.     valid_metric.update_state(labels, predictions)
  32. def train_model(model,ds_train,ds_valid,epochs):
  33.     for epoch in tf.range(1,epochs+1):
  34.         for features, labels in ds_train:
  35.             train_step(model,features,labels)
  37.         for features, labels in ds_valid:
  38.             valid_step(model,features,labels)
  40.         logs = 'Epoch={},Loss:{},Accuracy:{},Valid Loss:{},Valid Accuracy:{}'
  42.         if  epoch%100 ==0:
  43.             printbar()
  44.             tf.print(tf.strings.format(logs,
  45.             (epoch,train_loss.result(),train_metric.result(),valid_loss.result(),valid_metric.result())))
  47.         train_loss.reset_states()
  48.         valid_loss.reset_states()
  49.         train_metric.reset_states()
  50.         valid_metric.reset_states()
  52. train_model(model,ds_train,ds_valid,1000)
  1. ================================================================================17:35:02
  2. Epoch=100,Loss:0.194088802,Accuracy:0.923064,Valid Loss:0.215538561,Valid Accuracy:0.904368
  3. ================================================================================17:35:22
  4. Epoch=200,Loss:0.151239693,Accuracy:0.93768847,Valid Loss:0.181166962,Valid Accuracy:0.920664132
  5. ================================================================================17:35:43
  6. Epoch=300,Loss:0.134556711,Accuracy:0.944247484,Valid Loss:0.171530813,Valid Accuracy:0.926396072
  7. ================================================================================17:36:04
  8. Epoch=400,Loss:0.125722557,Accuracy:0.949172914,Valid Loss:0.16731061,Valid Accuracy:0.929318547
  9. ================================================================================17:36:24
  10. Epoch=500,Loss:0.120216407,Accuracy:0.952525079,Valid Loss:0.164817035,Valid Accuracy:0.931044817
  11. ================================================================================17:36:44
  12. Epoch=600,Loss:0.116434008,Accuracy:0.954830289,Valid Loss:0.163089141,Valid Accuracy:0.932202339
  13. ================================================================================17:37:05
  14. Epoch=700,Loss:0.113658346,Accuracy:0.956433,Valid Loss:0.161804497,Valid Accuracy:0.933092058
  15. ================================================================================17:37:25
  16. Epoch=800,Loss:0.111522928,Accuracy:0.957467675,Valid Loss:0.160796657,Valid Accuracy:0.93379426
  17. ================================================================================17:37:46
  18. Epoch=900,Loss:0.109816991,Accuracy:0.958205402,Valid Loss:0.159987748,Valid Accuracy:0.934343576
  19. ================================================================================17:38:06
  20. Epoch=1000,Loss:0.10841465,Accuracy:0.958805501,Valid Loss:0.159325734,Valid Accuracy:0.934785843
  1. # Visualizing the results
  2. fig, (ax1,ax2) = plt.subplots(nrows=1,ncols=2,figsize = (12,5))
  3. ax1.scatter(Xp[:,0].numpy(),Xp[:,1].numpy(),c = "r")
  4. ax1.scatter(Xn[:,0].numpy(),Xn[:,1].numpy(),c = "g")
  5. ax1.legend(["positive","negative"]);
  6. ax1.set_title("y_true");
  8. Xp_pred = tf.boolean_mask(X,tf.squeeze(model(X)>=0.5),axis = 0)
  9. Xn_pred = tf.boolean_mask(X,tf.squeeze(model(X)<0.5),axis = 0)
  11. ax2.scatter(Xp_pred[:,0].numpy(),Xp_pred[:,1].numpy(),c = "r")
  12. ax2.scatter(Xn_pred[:,0].numpy(),Xn_pred[:,1].numpy(),c = "g")
  13. ax2.legend(["positive","negative"]);
  14. ax2.set_title("y_pred");

3-3 High-level API: Demonstration - 图4

Please leave comments in the WeChat official account “Python与算法之美” (Elegance of Python and Algorithms) if you want to communicate with the author about the content. The author will try best to reply given the limited time available.

You are also welcomed to join the group chat with the other readers through replying 加群 (join group) in the WeChat official account.

image.png