计算缺少的数据

Missing values propagate naturally through arithmetic operations between pandas objects.

  1. In [27]: a
  2. Out[27]: 
  3.         one       two
  4. a       NaN  0.501113
  5. c       NaN  0.580967
  6. e  0.057802  0.761948
  7. f -0.443160 -0.974602
  8. h -0.443160 -1.053898
  10. In [28]: b
  11. Out[28]: 
  12.         one       two     three
  13. a       NaN  0.501113 -0.355322
  14. c       NaN  0.580967  0.983801
  15. e  0.057802  0.761948 -0.712964
  16. f -0.443160 -0.974602  1.047704
  17. h       NaN -1.053898 -0.019369
  19. In [29]: a + b
  20. Out[29]: 
  21.         one  three       two
  22. a       NaN    NaN  1.002226
  23. c       NaN    NaN  1.161935
  24. e  0.115604    NaN  1.523896
  25. f -0.886321    NaN -1.949205
  26. h       NaN    NaN -2.107796

The descriptive statistics and computational methods discussed in the data structure overview (and listed here and here) are all written to account for missing data. For example:

  • When summing data, NA (missing) values will be treated as zero.
  • If the data are all NA, the result will be 0.
  • Cumulative methods like cumsum() and cumprod() ignore NA values by default, but preserve them in the resulting arrays. To override this behaviour and include NA values, use skipna=False.
  1. In [30]: df
  2. Out[30]: 
  3.         one       two     three
  4. a       NaN  0.501113 -0.355322
  5. c       NaN  0.580967  0.983801
  6. e  0.057802  0.761948 -0.712964
  7. f -0.443160 -0.974602  1.047704
  8. h       NaN -1.053898 -0.019369
  10. In [31]: df['one'].sum()
  11. Out[31]: -0.38535826528461409
  13. In [32]: df.mean(1)
  14. Out[32]: 
  15. a    0.072895
  16. c    0.782384
  17. e    0.035595
  18. f   -0.123353
  19. h   -0.536633
  20. dtype: float64
  22. In [33]: df.cumsum()
  23. Out[33]: 
  24.         one       two     three
  25. a       NaN  0.501113 -0.355322
  26. c       NaN  1.082080  0.628479
  27. e  0.057802  1.844028 -0.084485
  28. f -0.385358  0.869426  0.963219
  29. h       NaN -0.184472  0.943850
  31. In [34]: df.cumsum(skipna=False)
  32. Out[34]: 
  33.    one       two     three
  34. a  NaN  0.501113 -0.355322
  35. c  NaN  1.082080  0.628479
  36. e  NaN  1.844028 -0.084485
  37. f  NaN  0.869426  0.963219
  38. h  NaN -0.184472  0.943850

Sum/Prod of Empties/Nans

警告

This behavior is now standard as of v0.22.0 and is consistent with the default in numpy; previously sum/prod of all-NA or empty Series/DataFrames would return NaN. See v0.22.0 whatsnew for more.

The sum of an empty or all-NA Series or column of a DataFrame is 0.

  1. In [35]: pd.Series([np.nan]).sum()
  2. Out[35]: 0.0
  4. In [36]: pd.Series([]).sum()
  5. Out[36]: 0.0

The product of an empty or all-NA Series or column of a DataFrame is 1.

  1. In [37]: pd.Series([np.nan]).prod()
  2. Out[37]: 1.0
  4. In [38]: pd.Series([]).prod()
  5. Out[38]: 1.0

NA values in GroupBy

NA groups in GroupBy are automatically excluded. This behavior is consistent with R, for example:

  1. In [39]: df
  2. Out[39]: 
  3.         one       two     three
  4. a       NaN  0.501113 -0.355322
  5. c       NaN  0.580967  0.983801
  6. e  0.057802  0.761948 -0.712964
  7. f -0.443160 -0.974602  1.047704
  8. h       NaN -1.053898 -0.019369
  10. In [40]: df.groupby('one').mean()
  11. Out[40]: 
  12.                 two     three
  13. one                          
  14. -0.443160 -0.974602  1.047704
  15.  0.057802  0.761948 -0.712964

See the groupby section here for more information.