Data Structures
DataArray
xarray.DataArray
is xarray’s implementation of a labeled,multi-dimensional array. It has several key properties:
values
: anumpy.ndarray
holding the array’s valuesdims
: dimension names for each axis (e.g.,('x', 'y', 'z')
)coords
: a dict-like container of arrays (coordinates) that label eachpoint (e.g., 1-dimensional arrays of numbers, datetime objects orstrings)attrs
: anOrderedDict
to hold arbitrary metadata (attributes)
xarray uses dims
and coords
to enable its core metadata aware operations.Dimensions provide names that xarray uses instead of the axis
argument foundin many numpy functions. Coordinates enable fast label based indexing andalignment, building on the functionality of the index
found on a pandasDataFrame
or Series
.
DataArray objects also can have a name
and can hold arbitrary metadata inthe form of their attrs
property (an ordered dictionary). Names andattributes are strictly for users and user-written code: xarray makes no attemptto interpret them, and propagates them only in unambiguous cases (see FAQ,What is your approach to metadata?).
Creating a DataArray
The DataArray
constructor takes:
data
: a multi-dimensional array of values (e.g., a numpy ndarray,Series
,DataFrame
orPanel
)coords
: a list or dictionary of coordinates. If a list, it should be alist of tuples where the first element is the dimension name and the secondelement is the corresponding coordinate array_like object.dims
: a list of dimension names. If omitted andcoords
is a list oftuples, dimension names are taken fromcoords
.attrs
: a dictionary of attributes to add to the instancename
: a string that names the instance
- In [1]: data = np.random.rand(4, 3)
- In [2]: locs = ['IA', 'IL', 'IN']
- In [3]: times = pd.date_range('2000-01-01', periods=4)
- In [4]: foo = xr.DataArray(data, coords=[times, locs], dims=['time', 'space'])
- In [5]: foo
- Out[5]:
- <xarray.DataArray (time: 4, space: 3)>
- array([[0.12697 , 0.966718, 0.260476],
- [0.897237, 0.37675 , 0.336222],
- [0.451376, 0.840255, 0.123102],
- [0.543026, 0.373012, 0.447997]])
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
Only data
is required; all of other arguments will be filledin with default values:
- In [6]: xr.DataArray(data)
- Out[6]:
- <xarray.DataArray (dim_0: 4, dim_1: 3)>
- array([[0.12697 , 0.966718, 0.260476],
- [0.897237, 0.37675 , 0.336222],
- [0.451376, 0.840255, 0.123102],
- [0.543026, 0.373012, 0.447997]])
- Dimensions without coordinates: dim_0, dim_1
As you can see, dimension names are always present in the xarray data model: ifyou do not provide them, defaults of the form dim_N
will be created.However, coordinates are always optional, and dimensions do not have automaticcoordinate labels.
Note
This is different from pandas, where axes always have tick labels, whichdefault to the integers [0, …, n-1]
.
Prior to xarray v0.9, xarray copied this behavior: default coordinates foreach dimension would be created if coordinates were not supplied explicitly.This is no longer the case.
Coordinates can be specified in the following ways:
A list of values with length equal to the number of dimensions, providingcoordinate labels for each dimension. Each value must be of one of thefollowing forms:
A tuple of the form
(dims, data[, attrs])
, which is converted intoarguments forVariable
A pandas object or scalar value, which is converted into a
DataArray
A 1D array or list, which is interpreted as values for a one dimensionalcoordinate variable along the same dimension as it’s name
A dictionary of
{coord_name: coord}
where values are of the same formas the list. Supplying coordinates as a dictionary allows other coordinatesthan those corresponding to dimensions (more on these later). If you supplycoords
as a dictionary, you must explicitly providedims
.
As a list of tuples:
- In [7]: xr.DataArray(data, coords=[('time', times), ('space', locs)])
- Out[7]:
- <xarray.DataArray (time: 4, space: 3)>
- array([[0.12697 , 0.966718, 0.260476],
- [0.897237, 0.37675 , 0.336222],
- [0.451376, 0.840255, 0.123102],
- [0.543026, 0.373012, 0.447997]])
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
As a dictionary:
- In [8]: xr.DataArray(data, coords={'time': times, 'space': locs, 'const': 42,
- ...: 'ranking': ('space', [1, 2, 3])},
- ...: dims=['time', 'space'])
- ...:
- Out[8]:
- <xarray.DataArray (time: 4, space: 3)>
- array([[0.12697 , 0.966718, 0.260476],
- [0.897237, 0.37675 , 0.336222],
- [0.451376, 0.840255, 0.123102],
- [0.543026, 0.373012, 0.447997]])
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
- const int64 42
- ranking (space) int64 1 2 3
As a dictionary with coords across multiple dimensions:
- In [9]: xr.DataArray(data, coords={'time': times, 'space': locs, 'const': 42,
- ...: 'ranking': (('time', 'space'), np.arange(12).reshape(4,3))},
- ...: dims=['time', 'space'])
- ...:
- Out[9]:
- <xarray.DataArray (time: 4, space: 3)>
- array([[0.12697 , 0.966718, 0.260476],
- [0.897237, 0.37675 , 0.336222],
- [0.451376, 0.840255, 0.123102],
- [0.543026, 0.373012, 0.447997]])
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
- const int64 42
- ranking (time, space) int64 0 1 2 3 4 5 6 7 8 9 10 11
If you create a DataArray
by supplying a pandasSeries
, DataFrame
orPanel
, any non-specified arguments in theDataArray
constructor will be filled in from the pandas object:
- In [10]: df = pd.DataFrame({'x': [0, 1], 'y': [2, 3]}, index=['a', 'b'])
- In [11]: df.index.name = 'abc'
- In [12]: df.columns.name = 'xyz'
- In [13]: df
- Out[13]:
- xyz x y
- abc
- a 0 2
- b 1 3
- In [14]: xr.DataArray(df)
- Out[14]:
- <xarray.DataArray (abc: 2, xyz: 2)>
- array([[0, 2],
- [1, 3]])
- Coordinates:
- * abc (abc) object 'a' 'b'
- * xyz (xyz) object 'x' 'y'
DataArray properties
Let’s take a look at the important properties on our array:
- In [15]: foo.values
- Out[15]:
- array([[0.127, 0.967, 0.26 ],
- [0.897, 0.377, 0.336],
- [0.451, 0.84 , 0.123],
- [0.543, 0.373, 0.448]])
- In [16]: foo.dims
- Out[16]: ('time', 'space')
- In [17]: foo.coords
- Out[17]:
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
- In [18]: foo.attrs
- Out[18]: OrderedDict()
- In [19]: print(foo.name)
- None
You can modify values
inplace:
- In [20]: foo.values = 1.0 * foo.values
Note
The array values in a DataArray
have a single(homogeneous) data type. To work with heterogeneous or structured datatypes in xarray, use coordinates, or put separate DataArray
objectsin a single Dataset
(see below).
Now fill in some of that missing metadata:
- In [21]: foo.name = 'foo'
- In [22]: foo.attrs['units'] = 'meters'
- In [23]: foo
- Out[23]:
- <xarray.DataArray 'foo' (time: 4, space: 3)>
- array([[0.12697 , 0.966718, 0.260476],
- [0.897237, 0.37675 , 0.336222],
- [0.451376, 0.840255, 0.123102],
- [0.543026, 0.373012, 0.447997]])
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
- Attributes:
- units: meters
The rename()
method is another option, returning anew data array:
- In [24]: foo.rename('bar')
- Out[24]:
- <xarray.DataArray 'bar' (time: 4, space: 3)>
- array([[0.12697 , 0.966718, 0.260476],
- [0.897237, 0.37675 , 0.336222],
- [0.451376, 0.840255, 0.123102],
- [0.543026, 0.373012, 0.447997]])
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
- Attributes:
- units: meters
DataArray Coordinates
The coords
property is dict
like. Individual coordinates can beaccessed from the coordinates by name, or even by indexing the data arrayitself:
- In [25]: foo.coords['time']
- Out[25]:
- <xarray.DataArray 'time' (time: 4)>
- array(['2000-01-01T00:00:00.000000000', '2000-01-02T00:00:00.000000000',
- '2000-01-03T00:00:00.000000000', '2000-01-04T00:00:00.000000000'],
- dtype='datetime64[ns]')
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- In [26]: foo['time']
- Out[26]:
- <xarray.DataArray 'time' (time: 4)>
- array(['2000-01-01T00:00:00.000000000', '2000-01-02T00:00:00.000000000',
- '2000-01-03T00:00:00.000000000', '2000-01-04T00:00:00.000000000'],
- dtype='datetime64[ns]')
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
These are also DataArray
objects, which contain tick-labelsfor each dimension.
Coordinates can also be set or removed by using the dictionary like syntax:
- In [27]: foo['ranking'] = ('space', [1, 2, 3])
- In [28]: foo.coords
- Out[28]:
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
- ranking (space) int64 1 2 3
- In [29]: del foo['ranking']
- In [30]: foo.coords
- Out[30]:
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
For more details, see Coordinates below.
Dataset
xarray.Dataset
is xarray’s multi-dimensional equivalent of aDataFrame
. It is a dict-likecontainer of labeled arrays (DataArray
objects) with aligneddimensions. It is designed as an in-memory representation of the data modelfrom the netCDF file format.
In addition to the dict-like interface of the dataset itself, which can be usedto access any variable in a dataset, datasets have four key properties:
dims
: a dictionary mapping from dimension names to the fixed length ofeach dimension (e.g.,{'x': 6, 'y': 6, 'time': 8}
)data_vars
: a dict-like container of DataArrays corresponding to variablescoords
: another dict-like container of DataArrays intended to label pointsused indata_vars
(e.g., arrays of numbers, datetime objects or strings)attrs
: anOrderedDict
to hold arbitrary metadata
The distinction between whether a variables falls in data or coordinates(borrowed from CF conventions) is mostly semantic, and you can probably getaway with ignoring it if you like: dictionary like access on a dataset willsupply variables found in either category. However, xarray does make use of thedistinction for indexing and computations. Coordinates indicateconstant/fixed/independent quantities, unlike the varying/measured/dependentquantities that belong in data.
Here is an example of how we might structure a dataset for a weather forecast:In this example, it would be natural to call temperature
andprecipitation
“data variables” and all the other arrays “coordinatevariables” because they label the points along the dimensions. (see 1 formore background on this example).
Creating a Dataset
To make an Dataset
from scratch, supply dictionaries for anyvariables (data_vars
), coordinates (coords
) and attributes (attrs
).
data_vars
should be a dictionary with each key as the name of the variableand each value as one of:coords
should be a dictionary of the same form asdata_vars
.attrs
should be a dictionary.
Let’s create some fake data for the example we show above:
- In [31]: temp = 15 + 8 * np.random.randn(2, 2, 3)
- In [32]: precip = 10 * np.random.rand(2, 2, 3)
- In [33]: lon = [[-99.83, -99.32], [-99.79, -99.23]]
- In [34]: lat = [[42.25, 42.21], [42.63, 42.59]]
- # for real use cases, its good practice to supply array attributes such as
- # units, but we won't bother here for the sake of brevity
- In [35]: ds = xr.Dataset({'temperature': (['x', 'y', 'time'], temp),
- ....: 'precipitation': (['x', 'y', 'time'], precip)},
- ....: coords={'lon': (['x', 'y'], lon),
- ....: 'lat': (['x', 'y'], lat),
- ....: 'time': pd.date_range('2014-09-06', periods=3),
- ....: 'reference_time': pd.Timestamp('2014-09-05')})
- ....:
- In [36]: ds
- Out[36]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- lat (x, y) float64 42.25 42.21 42.63 42.59
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- Dimensions without coordinates: x, y
- Data variables:
- temperature (x, y, time) float64 11.04 23.57 20.77 ... 6.301 9.61 15.91
- precipitation (x, y, time) float64 5.904 2.453 3.404 ... 3.435 1.709 3.947
Here we pass xarray.DataArray
objects or a pandas object as valuesin the dictionary:
- In [37]: xr.Dataset({'bar': foo})
- Out[37]:
- <xarray.Dataset>
- Dimensions: (space: 3, time: 4)
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) <U2 'IA' 'IL' 'IN'
- Data variables:
- bar (time, space) float64 0.127 0.9667 0.2605 ... 0.543 0.373 0.448
- In [38]: xr.Dataset({'bar': foo.to_pandas()})
- Out[38]:
- <xarray.Dataset>
- Dimensions: (space: 3, time: 4)
- Coordinates:
- * time (time) datetime64[ns] 2000-01-01 2000-01-02 2000-01-03 2000-01-04
- * space (space) object 'IA' 'IL' 'IN'
- Data variables:
- bar (time, space) float64 0.127 0.9667 0.2605 ... 0.543 0.373 0.448
Where a pandas object is supplied as a value, the names of its indexes are used as dimensionnames, and its data is aligned to any existing dimensions.
You can also create an dataset from:
A
pandas.DataFrame
orpandas.Panel
along its columns and itemsrespectively, by passing it into theDataset
directlyA
pandas.DataFrame
withDataset.from_dataframe
,which will additionally handle MultiIndexes See Working with pandasA netCDF file on disk with
open_dataset()
. See Reading and writing files.
Dataset contents
Dataset
implements the Python mapping interface, withvalues given by xarray.DataArray
objects:
- In [39]: 'temperature' in ds
- Out[39]: True
- In [40]: ds['temperature']
- Out[40]:
- <xarray.DataArray 'temperature' (x: 2, y: 2, time: 3)>
- array([[[11.040566, 23.57443 , 20.772441],
- [ 9.345831, 6.6834 , 17.174879]],
- [[11.600221, 19.536163, 17.209856],
- [ 6.300794, 9.610482, 15.909187]]])
- Coordinates:
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- lat (x, y) float64 42.25 42.21 42.63 42.59
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- Dimensions without coordinates: x, y
Valid keys include each listed coordinate and data variable.
Data and coordinate variables are also contained separately in thedata_vars
and coords
dictionary-like attributes:
- In [41]: ds.data_vars
- Out[41]:
- Data variables:
- temperature (x, y, time) float64 11.04 23.57 20.77 ... 6.301 9.61 15.91
- precipitation (x, y, time) float64 5.904 2.453 3.404 ... 3.435 1.709 3.947
- In [42]: ds.coords
- Out[42]:
- Coordinates:
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- lat (x, y) float64 42.25 42.21 42.63 42.59
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
Finally, like data arrays, datasets also store arbitrary metadata in the formof attributes:
- In [43]: ds.attrs
- Out[43]: OrderedDict()
- In [44]: ds.attrs['title'] = 'example attribute'
- In [45]: ds
- Out[45]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- lat (x, y) float64 42.25 42.21 42.63 42.59
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- Dimensions without coordinates: x, y
- Data variables:
- temperature (x, y, time) float64 11.04 23.57 20.77 ... 6.301 9.61 15.91
- precipitation (x, y, time) float64 5.904 2.453 3.404 ... 3.435 1.709 3.947
- Attributes:
- title: example attribute
xarray does not enforce any restrictions on attributes, but serialization tosome file formats may fail if you use objects that are not strings, numbersor numpy.ndarray
objects.
As a useful shortcut, you can use attribute style access for reading (but notsetting) variables and attributes:
- In [46]: ds.temperature
- Out[46]:
- <xarray.DataArray 'temperature' (x: 2, y: 2, time: 3)>
- array([[[11.040566, 23.57443 , 20.772441],
- [ 9.345831, 6.6834 , 17.174879]],
- [[11.600221, 19.536163, 17.209856],
- [ 6.300794, 9.610482, 15.909187]]])
- Coordinates:
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- lat (x, y) float64 42.25 42.21 42.63 42.59
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- Dimensions without coordinates: x, y
This is particularly useful in an exploratory context, because you cantab-complete these variable names with tools like IPython.
Dictionary like methods
We can update a dataset in-place using Python’s standard dictionary syntax. Forexample, to create this example dataset from scratch, we could have written:
- In [47]: ds = xr.Dataset()
- In [48]: ds['temperature'] = (('x', 'y', 'time'), temp)
- In [49]: ds['temperature_double'] = (('x', 'y', 'time'), temp * 2 )
- In [50]: ds['precipitation'] = (('x', 'y', 'time'), precip)
- In [51]: ds.coords['lat'] = (('x', 'y'), lat)
- In [52]: ds.coords['lon'] = (('x', 'y'), lon)
- In [53]: ds.coords['time'] = pd.date_range('2014-09-06', periods=3)
- In [54]: ds.coords['reference_time'] = pd.Timestamp('2014-09-05')
To change the variables in a Dataset
, you can use all the standard dictionarymethods, including values
, items
, delitem
, get
andupdate()
. Note that assigning a DataArray
or pandasobject to a Dataset
variable using setitem
or update
willautomatically align the array(s) to the originaldataset’s indexes.
You can copy a Dataset
by calling the copy()
method. By default, the copy is shallow, so only the container will be copied:the arrays in the Dataset
will still be stored in the same underlyingnumpy.ndarray
objects. You can copy all data by callingds.copy(deep=True)
.
Transforming datasets
In addition to dictionary-like methods (described above), xarray has additionalmethods (like pandas) for transforming datasets into new objects.
For removing variables, you can select and drop an explicit list ofvariables by indexing with a list of names or using thedrop()
methods to return a new Dataset
. Theseoperations keep around coordinates:
- In [55]: ds[['temperature']]
- Out[55]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- reference_time datetime64[ns] 2014-09-05
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- lat (x, y) float64 42.25 42.21 42.63 42.59
- Dimensions without coordinates: x, y
- Data variables:
- temperature (x, y, time) float64 11.04 23.57 20.77 ... 6.301 9.61 15.91
- In [56]: ds[['temperature', 'temperature_double']]
- Out[56]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- reference_time datetime64[ns] 2014-09-05
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- lat (x, y) float64 42.25 42.21 42.63 42.59
- Dimensions without coordinates: x, y
- Data variables:
- temperature (x, y, time) float64 11.04 23.57 20.77 ... 9.61 15.91
- temperature_double (x, y, time) float64 22.08 47.15 41.54 ... 19.22 31.82
- In [57]: ds.drop('temperature')
- Out[57]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- lat (x, y) float64 42.25 42.21 42.63 42.59
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- Dimensions without coordinates: x, y
- Data variables:
- temperature_double (x, y, time) float64 22.08 47.15 41.54 ... 19.22 31.82
- precipitation (x, y, time) float64 5.904 2.453 3.404 ... 1.709 3.947
To remove a dimension, you can use drop_dims()
method.Any variables using that dimension are dropped:
- In [58]: ds.drop_dims('time')
- Out[58]:
- <xarray.Dataset>
- Dimensions: (x: 2, y: 2)
- Coordinates:
- lat (x, y) float64 42.25 42.21 42.63 42.59
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- reference_time datetime64[ns] 2014-09-05
- Dimensions without coordinates: x, y
- Data variables:
- *empty*
As an alternate to dictionary-like modifications, you can useassign()
and assign_coords()
.These methods return a new dataset with additional (or replaced) or values:
- In [59]: ds.assign(temperature2 = 2 * ds.temperature)
- Out[59]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- lat (x, y) float64 42.25 42.21 42.63 42.59
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- Dimensions without coordinates: x, y
- Data variables:
- temperature (x, y, time) float64 11.04 23.57 20.77 ... 9.61 15.91
- temperature_double (x, y, time) float64 22.08 47.15 41.54 ... 19.22 31.82
- precipitation (x, y, time) float64 5.904 2.453 3.404 ... 1.709 3.947
- temperature2 (x, y, time) float64 22.08 47.15 41.54 ... 19.22 31.82
There is also the pipe()
method that allows you to usea method call with an external function (e.g., ds.pipe(func)
) instead ofsimply calling it (e.g., func(ds)
). This allows you to write pipelines fortransforming you data (using “method chaining”) instead of writing hard tofollow nested function calls:
- # these lines are equivalent, but with pipe we can make the logic flow
- # entirely from left to right
- In [60]: plt.plot((2 * ds.temperature.sel(x=0)).mean('y'))
- Out[60]: [<matplotlib.lines.Line2D at 0x7f3425bb25c0>]
- In [61]: (ds.temperature
- ....: .sel(x=0)
- ....: .pipe(lambda x: 2 * x)
- ....: .mean('y')
- ....: .pipe(plt.plot))
- ....:
- Out[61]: [<matplotlib.lines.Line2D at 0x7f3425bb2a20>]
Both pipe
and assign
replicate the pandas methods of the same names(DataFrame.pipe
andDataFrame.assign
).
With xarray, there is no performance penalty for creating new datasets, even ifvariables are lazily loaded from a file on disk. Creating new objects insteadof mutating existing objects often results in easier to understand code, so weencourage using this approach.
Renaming variables
Another useful option is the rename()
method to renamedataset variables:
- In [62]: ds.rename({'temperature': 'temp', 'precipitation': 'precip'})
- Out[62]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- lat (x, y) float64 42.25 42.21 42.63 42.59
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- Dimensions without coordinates: x, y
- Data variables:
- temp (x, y, time) float64 11.04 23.57 20.77 ... 9.61 15.91
- temperature_double (x, y, time) float64 22.08 47.15 41.54 ... 19.22 31.82
- precip (x, y, time) float64 5.904 2.453 3.404 ... 1.709 3.947
The related swap_dims()
method allows you do to swapdimension and non-dimension variables:
- In [63]: ds.coords['day'] = ('time', [6, 7, 8])
- In [64]: ds.swap_dims({'time': 'day'})
- Out[64]:
- <xarray.Dataset>
- Dimensions: (day: 3, x: 2, y: 2)
- Coordinates:
- lat (x, y) float64 42.25 42.21 42.63 42.59
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- time (day) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- * day (day) int64 6 7 8
- Dimensions without coordinates: x, y
- Data variables:
- temperature (x, y, day) float64 11.04 23.57 20.77 ... 9.61 15.91
- temperature_double (x, y, day) float64 22.08 47.15 41.54 ... 19.22 31.82
- precipitation (x, y, day) float64 5.904 2.453 3.404 ... 1.709 3.947
Coordinates
Coordinates are ancillary variables stored for DataArray
and Dataset
objects in the coords
attribute:
- In [65]: ds.coords
- Out[65]:
- Coordinates:
- lat (x, y) float64 42.25 42.21 42.63 42.59
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- day (time) int64 6 7 8
Unlike attributes, xarray does interpret and persist coordinates inoperations that transform xarray objects. There are two types of coordinatesin xarray:
dimension coordinates are one dimensional coordinates with a name equalto their sole dimension (marked by
*
when printing a dataset or dataarray). They are used for label based indexing and alignment,like theindex
found on a pandasDataFrame
orSeries
. Indeed, these “dimension” coordinates use apandas.Index
internally to store their values.non-dimension coordinates are variables that contain coordinatedata, but are not a dimension coordinate. They can be multidimensional(see Working with Multidimensional Coordinates), and there is no relationship between thename of a non-dimension coordinate and the name(s) of its dimension(s).Non-dimension coordinates can be useful for indexing or plotting; otherwise,xarray does not make any direct use of the values associated with them.They are not used for alignment or automatic indexing, nor are they requiredto match when doing arithmetic(see Coordinates).
Note
xarray’s terminology differs from the CF terminology, where the“dimension coordinates” are called “coordinate variables”, and the“non-dimension coordinates” are called “auxiliary coordinate variables”(see GH1295 for more details).
Modifying coordinates
To entirely add or remove coordinate arrays, you can use dictionary likesyntax, as shown above.
To convert back and forth between data and coordinates, you can use theset_coords()
andreset_coords()
methods:
- In [66]: ds.reset_coords()
- Out[66]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- Dimensions without coordinates: x, y
- Data variables:
- temperature (x, y, time) float64 11.04 23.57 20.77 ... 9.61 15.91
- temperature_double (x, y, time) float64 22.08 47.15 41.54 ... 19.22 31.82
- precipitation (x, y, time) float64 5.904 2.453 3.404 ... 1.709 3.947
- lat (x, y) float64 42.25 42.21 42.63 42.59
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- reference_time datetime64[ns] 2014-09-05
- day (time) int64 6 7 8
- In [67]: ds.set_coords(['temperature', 'precipitation'])
- Out[67]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- temperature (x, y, time) float64 11.04 23.57 20.77 ... 9.61 15.91
- precipitation (x, y, time) float64 5.904 2.453 3.404 ... 1.709 3.947
- lat (x, y) float64 42.25 42.21 42.63 42.59
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- day (time) int64 6 7 8
- Dimensions without coordinates: x, y
- Data variables:
- temperature_double (x, y, time) float64 22.08 47.15 41.54 ... 19.22 31.82
- In [68]: ds['temperature'].reset_coords(drop=True)
- Out[68]:
- <xarray.DataArray 'temperature' (x: 2, y: 2, time: 3)>
- array([[[11.040566, 23.57443 , 20.772441],
- [ 9.345831, 6.6834 , 17.174879]],
- [[11.600221, 19.536163, 17.209856],
- [ 6.300794, 9.610482, 15.909187]]])
- Coordinates:
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- Dimensions without coordinates: x, y
Notice that these operations skip coordinates with names given by dimensions,as used for indexing. This mostly because we are not entirely sure how todesign the interface around the fact that xarray cannot store a coordinate andvariable with the name but different values in the same dictionary. But we dorecognize that supporting something like this would be useful.
Coordinates methods
Coordinates
objects also have a few useful methods, mostly for convertingthem into dataset objects:
- In [69]: ds.coords.to_dataset()
- Out[69]:
- <xarray.Dataset>
- Dimensions: (time: 3, x: 2, y: 2)
- Coordinates:
- lat (x, y) float64 42.25 42.21 42.63 42.59
- reference_time datetime64[ns] 2014-09-05
- lon (x, y) float64 -99.83 -99.32 -99.79 -99.23
- day (time) int64 6 7 8
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- Dimensions without coordinates: x, y
- Data variables:
- *empty*
The merge method is particularly interesting, because it implements the samelogic used for merging coordinates in arithmetic operations(see Computation):
- In [70]: alt = xr.Dataset(coords={'z': [10], 'lat': 0, 'lon': 0})
- In [71]: ds.coords.merge(alt.coords)
- Out[71]:
- <xarray.Dataset>
- Dimensions: (time: 3, z: 1)
- Coordinates:
- * time (time) datetime64[ns] 2014-09-06 2014-09-07 2014-09-08
- reference_time datetime64[ns] 2014-09-05
- day (time) int64 6 7 8
- * z (z) int64 10
- Data variables:
- *empty*
The coords.merge
method may be useful if you want to implement your ownbinary operations that act on xarray objects. In the future, we hope to writemore helper functions so that you can easily make your functions act likexarray’s built-in arithmetic.
Indexes
To convert a coordinate (or any DataArray
) into an actualpandas.Index
, use the to_index()
method:
- In [72]: ds['time'].to_index()
- Out[72]: DatetimeIndex(['2014-09-06', '2014-09-07', '2014-09-08'], dtype='datetime64[ns]', name='time', freq='D')
A useful shortcut is the indexes
property (on both DataArray
andDataset
), which lazily constructs a dictionary whose keys are given by eachdimension and whose the values are Index
objects:
- In [73]: ds.indexes
- Out[73]: time: DatetimeIndex(['2014-09-06', '2014-09-07', '2014-09-08'], dtype='datetime64[ns]', name='time', freq='D')
MultiIndex coordinates
Xarray supports labeling coordinate values with a pandas.MultiIndex
:
- In [74]: midx = pd.MultiIndex.from_arrays([['R', 'R', 'V', 'V'], [.1, .2, .7, .9]],
- ....: names=('band', 'wn'))
- ....:
- In [75]: mda = xr.DataArray(np.random.rand(4), coords={'spec': midx}, dims='spec')
- In [76]: mda
- Out[76]:
- <xarray.DataArray (spec: 4)>
- array([0.641666, 0.274592, 0.462354, 0.871372])
- Coordinates:
- * spec (spec) MultiIndex
- - band (spec) object 'R' 'R' 'V' 'V'
- - wn (spec) float64 0.1 0.2 0.7 0.9
For convenience multi-index levels are directly accessible as “virtual” or“derived” coordinates (marked by -
when printing a dataset or data array):
- In [77]: mda['band']
- Out[77]:
- <xarray.DataArray 'band' (spec: 4)>
- array(['R', 'R', 'V', 'V'], dtype=object)
- Coordinates:
- * spec (spec) MultiIndex
- - band (spec) object 'R' 'R' 'V' 'V'
- - wn (spec) float64 0.1 0.2 0.7 0.9
- In [78]: mda.wn
- Out[78]:
- <xarray.DataArray 'wn' (spec: 4)>
- array([0.1, 0.2, 0.7, 0.9])
- Coordinates:
- * spec (spec) MultiIndex
- - band (spec) object 'R' 'R' 'V' 'V'
- - wn (spec) float64 0.1 0.2 0.7 0.9
Indexing with multi-index levels is also possible using the sel
method(see Multi-level indexing).
Unlike other coordinates, “virtual” level coordinates are not stored inthe coords
attribute of DataArray
and Dataset
objects(although they are shown when printing the coords
attribute).Consequently, most of the coordinates related methods don’t apply for them.It also can’t be used to replace one particular level.
Because in a DataArray
or Dataset
object each multi-index level isaccessible as a “virtual” coordinate, its name must not conflict with the namesof the other levels, coordinates and data variables of the same object.Even though Xarray set default names for multi-indexes with unnamed levels,it is recommended that you explicitly set the names of the levels.
- 1
- Latitude and longitude are 2D arrays because the dataset usesprojected coordinates.
reference_time
refers to the reference timeat which the forecast was made, rather thantime
which is the valid timefor which the forecast applies.