5-2 feature_column

Feature column is usually applied in the feature engineering for the structured data, while rarely used for the image or text date.

1. Introduction about how to use feature column

Feature column is used to converting category features into one-hot encoding, or creating bucketing feature from continuous feature, or generating cross features from multiple features, etc.

Before creating feature column, please call the functions in the module tf.feature_column. The nine most frequently used functions in this module are shown in the figure below. All these functions will return a Categorical-Column or a Dense-Column object, but will not return bucketized_column, since the last class is inhereted from the first two classes.

Be careful: all the Categorical-Column class have to be converted into Dense-Column class through indicator_column before input to the model.

5-2 feature_column - 图1

  • numeric_column, the most frequently used function.
  • bucketized_column, generated from numerical column, listing multiple features from a numerical clumn; it is one-hot encoded.
  • categorical_column_with_identity, one-hot encoded, identical to the case that each bucket is one interger.
  • categorical_column_with_vocabulary_list, one-hot encoded; the dictionary is specified by the list.
  • categorical_column_with_vocabulary_file， one-hot encoded; the dictionary is specified by the file.
  • categorical_column_with_hash_bucket, used in the case with a large interger or a large dictionary.
  • indicator_column, generated by Categorical-Column; one-hot encoded.
  • embedding_column, generated by Categorical Column; the embedded vector distributed parameter needs learning/training. The recommended dimension of the embedded vector is the fourth root to the number of categories.
  • crossed_column, consists of arbitrary category column except for categorical_column_with_hash_bucket

2. Demonstration of feature column

Here is a complete example that solves Titanic survival problmen using feature column.

  1. import datetime
  2. import numpy as np
  3. import pandas as pd
  4. from matplotlib import pyplot as plt
  5. import tensorflow as tf
  6. from tensorflow.keras import layers,models
  9. # Printing log
  10. def printlog(info):
  11.     nowtime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
  12.     print("\n"+"=========="*8 + "%s"%nowtime)
  13.     print(info+'...\n\n')
  1. #================================================================================
  2. # 1. Constructing data pipeline
  3. #================================================================================
  4. printlog("step1: prepare dataset...")
  7. dftrain_raw = pd.read_csv("../data/titanic/train.csv")
  8. dftest_raw = pd.read_csv("../data/titanic/test.csv")
  10. dfraw = pd.concat([dftrain_raw,dftest_raw])
  12. def prepare_dfdata(dfraw):
  13.     dfdata = dfraw.copy()
  14.     dfdata.columns = [x.lower() for x in dfdata.columns]
  15.     dfdata = dfdata.rename(columns={'survived':'label'})
  16.     dfdata = dfdata.drop(['passengerid','name'],axis = 1)
  17.     for col,dtype in dict(dfdata.dtypes).items():
  18.         # See if there are missing values.
  19.         if dfdata[col].hasnans:
  20.             # Adding signs to the missing columns
  21.             dfdata[col + '_nan'] = pd.isna(dfdata[col]).astype('int32')
  22.             # Fill
  23.             if dtype not in [np.object,np.str,np.unicode]:
  24.                 dfdata[col].fillna(dfdata[col].mean(),inplace = True)
  25.             else:
  26.                 dfdata[col].fillna('',inplace = True)
  27.     return(dfdata)
  29. dfdata = prepare_dfdata(dfraw)
  30. dftrain = dfdata.iloc[0:len(dftrain_raw),:]
  31. dftest = dfdata.iloc[len(dftrain_raw):,:]
  35. # Importing data from dataframe
  36. def df_to_dataset(df, shuffle=True, batch_size=32):
  37.     dfdata = df.copy()
  38.     if 'label' not in dfdata.columns:
  39.         ds = tf.data.Dataset.from_tensor_slices(dfdata.to_dict(orient = 'list'))
  40.     else: 
  41.         labels = dfdata.pop('label')
  42.         ds = tf.data.Dataset.from_tensor_slices((dfdata.to_dict(orient = 'list'), labels))  
  43.     if shuffle:
  44.         ds = ds.shuffle(buffer_size=len(dfdata))
  45.     ds = ds.batch(batch_size)
  46.     return ds
  48. ds_train = df_to_dataset(dftrain)
  49. ds_test = df_to_dataset(dftest)
  1. #================================================================================
  2. # 2. Defining the feature column
  3. #================================================================================
  4. printlog("step2: make feature columns...")
  6. feature_columns = []
  8. # Numerical column
  9. for col in ['age','fare','parch','sibsp'] + [
  10.     c for c in dfdata.columns if c.endswith('_nan')]:
  11.     feature_columns.append(tf.feature_column.numeric_column(col))
  13. # Bucketized column
  14. age = tf.feature_column.numeric_column('age')
  15. age_buckets = tf.feature_column.bucketized_column(age, 
  16.              boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
  17. feature_columns.append(age_buckets)
  19. # Category column
  20. # NOTE: all the Categorical-Column class have to be converted into Dense-Column class through `indicator_column` before input to the model.
  21. sex = tf.feature_column.indicator_column(
  22.       tf.feature_column.categorical_column_with_vocabulary_list(
  23.       key='sex',vocabulary_list=["male", "female"]))
  24. feature_columns.append(sex)
  26. pclass = tf.feature_column.indicator_column(
  27.       tf.feature_column.categorical_column_with_vocabulary_list(
  28.       key='pclass',vocabulary_list=[1,2,3]))
  29. feature_columns.append(pclass)
  31. ticket = tf.feature_column.indicator_column(
  32.      tf.feature_column.categorical_column_with_hash_bucket('ticket',3))
  33. feature_columns.append(ticket)
  35. embarked = tf.feature_column.indicator_column(
  36.       tf.feature_column.categorical_column_with_vocabulary_list(
  37.       key='embarked',vocabulary_list=['S','C','B']))
  38. feature_columns.append(embarked)
  40. # Embedding column
  41. cabin = tf.feature_column.embedding_column(
  42.     tf.feature_column.categorical_column_with_hash_bucket('cabin',32),2)
  43. feature_columns.append(cabin)
  45. # Crossed column
  46. pclass_cate = tf.feature_column.categorical_column_with_vocabulary_list(
  47.           key='pclass',vocabulary_list=[1,2,3])
  49. crossed_feature = tf.feature_column.indicator_column(
  50.     tf.feature_column.crossed_column([age_buckets, pclass_cate],hash_bucket_size=15))
  52. feature_columns.append(crossed_feature)
  1. #================================================================================
  2. # 3. Defining the model
  3. #================================================================================
  4. printlog("step3: define model...")
  6. tf.keras.backend.clear_session()
  7. model = tf.keras.Sequential([
  8.   layers.DenseFeatures(feature_columns), # Placing the feature into tf.keras.layers.DenseFeatures
  9.   layers.Dense(64, activation='relu'),
  10.   layers.Dense(64, activation='relu'),
  11.   layers.Dense(1, activation='sigmoid')
  12. ])
  1. #================================================================================
  2. # 4. Training the model
  3. #================================================================================
  4. printlog("step4: train model...")
  6. model.compile(optimizer='adam',
  7.               loss='binary_crossentropy',
  8.               metrics=['accuracy'])
  10. history = model.fit(ds_train,
  11.           validation_data=ds_test,
  12.           epochs=10)
  1. #================================================================================
  2. # 5. Evaluating the model
  3. #================================================================================
  4. printlog("step5: eval model...")
  6. model.summary()
  9. %matplotlib inline
  10. %config InlineBackend.figure_format = 'svg'
  12. import matplotlib.pyplot as plt
  14. def plot_metric(history, metric):
  15.     train_metrics = history.history[metric]
  16.     val_metrics = history.history['val_'+metric]
  17.     epochs = range(1, len(train_metrics) + 1)
  18.     plt.plot(epochs, train_metrics, 'bo--')
  19.     plt.plot(epochs, val_metrics, 'ro-')
  20.     plt.title('Training and validation '+ metric)
  21.     plt.xlabel("Epochs")
  22.     plt.ylabel(metric)
  23.     plt.legend(["train_"+metric, 'val_'+metric])
  24.     plt.show()
  26. plot_metric(history,"accuracy")
  1. Model: "sequential"
  2. _________________________________________________________________
  3. Layer (type)                 Output Shape              Param #   
  4. =================================================================
  5. dense_features (DenseFeature multiple                  64        
  6. _________________________________________________________________
  7. dense (Dense)                multiple                  3008      
  8. _________________________________________________________________
  9. dense_1 (Dense)              multiple                  4160      
  10. _________________________________________________________________
  11. dense_2 (Dense)              multiple                  65        
  12. =================================================================
  13. Total params: 7,297
  14. Trainable params: 7,297
  15. Non-trainable params: 0
  16. _________________________________________________________________

5-2 feature_column - 图2

Please leave comments in the WeChat official account “Python与算法之美” (Elegance of Python and Algorithms) if you want to communicate with the author about the content. The author will try best to reply given the limited time available.

You are also welcomed to join the group chat with the other readers through replying 加群 (join group) in the WeChat official account.

image.png