5.3 Column Transformation

Whereas the @select macro variants performs column selection, the @transform macro variants do not perform any column selection. It can either overwrite existent columns or create new columns that will be added to the right of our DataFrame.

For example, the previous operation on :grade can be invoked as a transformation with:

@rtransform df :grade_100 = :grade * 10

As you can see, @transform does not perform column selection, and the :grade_100 column is created as a new column and added to the right of our DataFrame.

DataFramesMeta.jl macros also support begin ... end statements. For example, suppose that you are creating two columns in a @transform macro:

@rtransform df :grade_100 = :grade * 10 :grade_5 = :grade / 2

It can be cumbersome and difficult to read the performed transformations. To facilitate that, we can use begin ... end statements and put one transformation per line:

@rtransform df begin
    :grade_100 = :grade * 10
    :grade_5 = :grade / 2
end

We can also use other columns in our transformations, which makes DataFramesMeta.jl more appealing than DataFrames.jl due to the easier syntax.

First, let’s revisit the leftjoined DataFrame from Chapter 4:

leftjoined = leftjoin(grades_2020(), grades_2021(); on=:name)

Additionally, we’ll replace the missing values with 5 (Section 4.9, also note the ! in in-place variant @rtransform!):

@rtransform! leftjoined :grade_2021 = coalesce(:grade_2021, 5)

This is how you calculate the mean of grades in both years using DataFramesMeta.jl:

@rtransform leftjoined :mean_grades = (:grade_2020 + :grade_2021) / 2

This is how you would perform it in DataFrames.jl:

transform(leftjoined, [:grade_2020, :grade_2021] => ByRow((x, y) -> (x + y) / 2) => :mean_grades)

As you can see, the case for easier syntax is not hard to argue for DataFramesMeta.jl.

Support this project
CC BY-NC-SA 4.0 Jose Storopoli, Rik Huijzer, Lazaro Alonso