Datasets

機器學習資料集/ 範例一: The digits dataset

http://scikit-learn.org/stable/auto_examples/datasets/plot_digits_last_image.html

這個範例目的是介紹機器學習範例資料集的操作，對於初學者以及授課特別適合使用。

(一)引入函式庫及內建手寫數字資料庫

  1. #這行是在ipython notebook的介面裏專用，如果在其他介面則可以拿掉
  2. %matplotlib inline
  3. from sklearn import datasets
  5. import matplotlib.pyplot as plt
  7. #載入數字資料集
  8. digits = datasets.load_digits()
  10. #畫出第一個圖片
  11. plt.figure(1, figsize=(3, 3))
  12. plt.imshow(digits.images[-1], cmap=plt.cm.gray_r, interpolation='nearest')
  13. plt.show()

png

(二)資料集介紹

digits = datasets.load_digits() 將一個dict型別資料存入digits，我們可以用下面程式碼來觀察裏面資料

  1. for key,value in digits.items() :
  2.     try:
  3.         print (key,value.shape)
  4.     except:
  5.         print (key)
  1. ('images', (1797L, 8L, 8L))
  2. ('data', (1797L, 64L))
  3. ('target_names', (10L,))
  4. DESCR
  5. ('target', (1797L,))
顯示 說明
(‘images’, (1797L, 8L, 8L)) 共有 1797 張影像，影像大小為 8x8
(‘data’, (1797L, 64L)) data 則是將8x8的矩陣攤平成64個元素之一維向量
(‘target_names’, (10L,)) 說明10種分類之對應 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
DESCR 資料之描述
(‘target’, (1797L,)) 記錄1797張影像各自代表那一個數字

接下來我們試著以下面指令來觀察資料檔，每張影像所對照的實際數字存在digits.target變數中

  1. images_and_labels = list(zip(digits.images, digits.target))
  2. for index, (image, label) in enumerate(images_and_labels[:4]):
  3.     plt.subplot(2, 4, index + 1)
  4.     plt.axis('off')
  5.     plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest')
  6.     plt.title('Training: %i' % label)

png

  1. #接著我們嘗試將這個機器學習資料之描述檔顯示出來
  2. print(digits['DESCR'])
  1. Optical Recognition of Handwritten Digits Data Set
  2. ===================================================
  4. Notes
  5. -----
  6. Data Set Characteristics:
  7.     :Number of Instances: 5620
  8.     :Number of Attributes: 64
  9.     :Attribute Information: 8x8 image of integer pixels in the range 0..16.
  10.     :Missing Attribute Values: None
  11.     :Creator: E. Alpaydin (alpaydin '@' boun.edu.tr)
  12.     :Date: July; 1998
  14. This is a copy of the test set of the UCI ML hand-written digits datasets
  15. http://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits
  17. The data set contains images of hand-written digits: 10 classes where
  18. each class refers to a digit.
  20. Preprocessing programs made available by NIST were used to extract
  21. normalized bitmaps of handwritten digits from a preprinted form. From a
  22. total of 43 people, 30 contributed to the training set and different 13
  23. to the test set. 32x32 bitmaps are divided into nonoverlapping blocks of
  24. 4x4 and the number of on pixels are counted in each block. This generates
  25. an input matrix of 8x8 where each element is an integer in the range
  26. 0..16. This reduces dimensionality and gives invariance to small
  27. distortions.
  29. For info on NIST preprocessing routines, see M. D. Garris, J. L. Blue, G.
  30. T. Candela, D. L. Dimmick, J. Geist, P. J. Grother, S. A. Janet, and C.
  31. L. Wilson, NIST Form-Based Handprint Recognition System, NISTIR 5469,
  32. 1994.
  34. References
  35. ----------
  36.   - C. Kaynak (1995) Methods of Combining Multiple Classifiers and Their
  37.     Applications to Handwritten Digit Recognition, MSc Thesis, Institute of
  38.     Graduate Studies in Science and Engineering, Bogazici University.
  39.   - E. Alpaydin, C. Kaynak (1998) Cascading Classifiers, Kybernetika.
  40.   - Ken Tang and Ponnuthurai N. Suganthan and Xi Yao and A. Kai Qin.
  41.     Linear dimensionalityreduction using relevance weighted LDA. School of
  42.     Electrical and Electronic Engineering Nanyang Technological University.
  43.     2005.
  44.   - Claudio Gentile. A New Approximate Maximal Margin Classification
  45.     Algorithm. NIPS. 2000.

這個描述檔說明了這個資料集是在 1998年時建立的，由E. Alpaydin, C. Kaynak ，Department of Computer Engineering Bogazici University, Istanbul Turkey 建立的。數字的筆跡總共來自43個人，一開始取像時為32x32的點陣影像，之後經運算處理形成 8x8影像，其中灰階記錄的範圍則為 0~16的整數。

(三)應用範例介紹

在整個scikit-learn應用範例中，有以下幾個範例是利用了這組手寫辨識資料集。這個資料集的使用最適合機器學習初學者來理解分類法的原理以及其進階應用