datawizard

Magic potions to clean and transform your data 🧙

output: github_document

`datawizard`: Easy Data Wrangling and Statistical Transformations

knitr::opts_chunk$set(
  collapse = TRUE,
  dpi = 300,
  out.width = "100%",
  fig.path = "man/figures/",
  comment = "#>"
)

set.seed(333)
library(datawizard)

{datawizard} is a lightweight package to easily manipulate, clean, transform, and prepare your data for analysis. It is part of the easystats ecosystem, a suite of R packages to deal with your entire statistical analysis, from cleaning the data to reporting the results.

It covers two aspects of data preparation:

Data manipulation: {datawizard} offers a very similar set of functions to that of the tidyverse packages, such as a {dplyr} and {tidyr}, to select, filter and reshape data, with a few key differences. 1) All data manipulation functions start with the prefix data_* (which makes them easy to identify). 2) Although most functions can be used exactly as their tidyverse equivalents, they are also string-friendly (which makes them easy to program with and use inside functions). Finally, {datawizard} is super lightweight (no dependencies, similar to poorman), which makes it awesome for developers to use in their packages.
Statistical transformations: {datawizard} also has powerful functions to easily apply common data transformations, including standardization, normalization, rescaling, rank-transformation, scale reversing, recoding, binning, etc.

Installation

Type	Source	Command
Release	CRAN	`install.packages("datawizard")`
Development	r-universe	`install.packages("datawizard", repos = "https://easystats.r-universe.dev")`
Development	GitHub	`remotes::install_github("easystats/datawizard")`

Tip

Instead of library(datawizard), use library(easystats).
This will make all features of the easystats-ecosystem available.

To stay updated, use easystats::install_latest().

Citation

To cite the package, run the following command:

citation("datawizard")

Features

Most courses and tutorials about statistical modeling assume that you are working with a clean and tidy dataset. In practice, however, a major part of doing statistical modeling is preparing your data–cleaning up values, creating new columns, reshaping the dataset, or transforming some variables. {datawizard} provides easy to use tools to perform these common, critical, and sometimes tedious data preparation tasks.

Data wrangling

Select, filter and remove variables

The package provides helpers to filter rows meeting certain conditions…

data_match(mtcars, data.frame(vs = 0, am = 1))

… or logical expressions:

data_filter(mtcars, vs == 0 & am == 1)

Finding columns in a data frame, or retrieving the data of selected columns, can be achieved using find_columns() or get_columns():

# find column names matching a pattern
find_columns(iris, starts_with("Sepal"))

# return data columns matching a pattern
get_columns(iris, starts_with("Sepal")) |> head()

It is also possible to extract one or more variables:

# single variable
data_extract(mtcars, "gear")

# more variables
head(data_extract(iris, ends_with("Width")))

Due to the consistent API, removing variables is just as simple:

head(data_remove(iris, starts_with("Sepal")))

Reorder or rename

head(data_relocate(iris, select = "Species", before = "Sepal.Length"))

head(data_rename(iris, c("Sepal.Length", "Sepal.Width"), c("length", "width")))

Merge

x <- data.frame(a = 1:3, b = c("a", "b", "c"), c = 5:7, id = 1:3)
y <- data.frame(c = 6:8, d = c("f", "g", "h"), e = 100:102, id = 2:4)

x
y

data_merge(x, y, join = "full")

data_merge(x, y, join = "left")

data_merge(x, y, join = "right")

data_merge(x, y, join = "semi", by = "c")

data_merge(x, y, join = "anti", by = "c")

data_merge(x, y, join = "inner")

data_merge(x, y, join = "bind")

Reshape

A common data wrangling task is to reshape data.

Either to go from wide/Cartesian to long/tidy format

wide_data <- data.frame(replicate(5, rnorm(10)))

head(data_to_long(wide_data))

or the other way

long_data <- data_to_long(wide_data, rows_to = "Row_ID") # Save row number

data_to_wide(long_data,
  names_from = "name",
  values_from = "value",
  id_cols = "Row_ID"
)

Empty rows and columns

tmp <- data.frame(
  a = c(1, 2, 3, NA, 5),
  b = c(1, NA, 3, NA, 5),
  c = c(NA, NA, NA, NA, NA),
  d = c(1, NA, 3, NA, 5)
)

tmp

# indices of empty columns or rows
empty_columns(tmp)
empty_rows(tmp)

# remove empty columns or rows
remove_empty_columns(tmp)
remove_empty_rows(tmp)

# remove empty columns and rows
remove_empty(tmp)

Recode or cut dataframe

set.seed(123)
x <- sample(1:10, size = 50, replace = TRUE)

table(x)

# cut into 3 groups, based on distribution (quantiles)
table(categorize(x, split = "quantile", n_groups = 3))

Data Transformations

The packages also contains multiple functions to help transform data.

Standardize

For example, to standardize (z-score) data:

# before
summary(swiss)

# after
summary(standardize(swiss))

Winsorize

To winsorize data:

# before
anscombe

# after
winsorize(anscombe)

Center

To grand-mean center data

center(anscombe)

Ranktransform

To rank-transform data:

# before
head(trees)

# after
head(ranktransform(trees))

Rescale

To rescale a numeric variable to a new range:

change_scale(c(0, 1, 5, -5, -2))

Rotate or transpose

x <- mtcars[1:3, 1:4]

x

data_rotate(x)

Data properties

datawizard provides a way to provide comprehensive descriptive summary for all variables in a dataframe:

data(iris)
describe_distribution(iris)

Or even just a variable

describe_distribution(mtcars$wt)

There are also some additional data properties that can be computed using this package.

x <- (-10:10)^3 + rnorm(21, 0, 100)
smoothness(x, method = "diff")

Function design and pipe-workflow

The design of the {datawizard} functions follows a design principle that makes it easy for user to understand and remember how functions work:

the first argument is the data
for methods that work on data frames, two arguments are following to select and exclude variables
the following arguments are arguments related to the specific tasks of the functions

Most important, functions that accept data frames usually have this as their first argument, and also return a (modified) data frame again. Thus, {datawizard} integrates smoothly into a “pipe-workflow”.

iris |>
  # all rows where Species is "versicolor" or "virginica"
  data_filter(Species %in% c("versicolor", "virginica")) |>
  # select only columns with "." in names (i.e. drop Species)
  get_columns(contains("\\.")) |>
  # move columns that ends with "Length" to start of data frame
  data_relocate(ends_with("Length")) |>
  # remove fourth column
  data_remove(4) |>
  head()

Contributing and Support

In case you want to file an issue or contribute in another way to the package, please follow this guide. For questions about the functionality, you may either contact us via email or also file an issue.

Code of Conduct

Please note that this project is released with a
Contributor Code of Conduct. By participating in this project you agree to abide by its terms.