Chapter 17 R objects and indexing

R objects (beyond data.frames) and indexing.

“Rigor and clarity are not synonymous” – Larry Wasserman

“Never hesitate to sacrifice truth for clarity.” – Greg Wilson’s dad

17.1 Vectors are everywhere

Your garden variety R object is a vector. A single piece of info that you regard as a scalar is just a vector of length 1 and R will cheerfully let you add stuff to it. Square brackets are used for isolating elements of a vector for inspection, modification, etc. This is often called indexing. Go through the following code carefully, as it’s really rather surprising. BTW, indexing begins at 1 in R, unlike many other languages that index from 0.

R is built to work with vectors. Many operations are vectorized, i.e. by default they will happen component-wise when given a vector as input. Novices often don’t internalize or exploit this and they write lots of unnecessary for loops.

When reading function documentation, keep your eyes peeled for arguments that can be vectors. You’ll be surprised how common they are. For example, the mean and standard deviation of random normal variates can be provided as vectors.

This could be awesome in some settings, but dangerous in others, i.e. if you exploit this by mistake and get no warning. This is one of the reasons it’s so important to keep close tabs on your R objects: are they what you expect in terms of their flavor and length or dimensions? Check early and check often.

Notice that R also recycles vectors, if they are not the necessary length. You will get a warning if R suspects recycling is unintended, i.e. when one length is not an integer multiple of another, but recycling is silent if it seems like you know what you’re doing. Can be a beautiful thing when you’re doing this deliberately, but devastating when you don’t.

Question: is there a way to turn recycling off? Not that I know of.

The combine function c() is your go-to function for making vectors.

Plain vanilla R objects are called “atomic vectors” and an absolute requirement is that all the bits of info they hold are of the same flavor, i.e. all numeric or logical or character. If that’s not already true upon creation, the elements will be coerced to the same flavor, using a “lowest common denominator” approach (usually character). This is another stellar opportunity for you to create an object of one flavor without meaning to do so and to remain ignorant of that for a long time. Check early, check often.

The most important atomic vector types are:

  • logical: TRUE’s AND FALSE’s, easily coerced into 1’s and 0’s
  • numeric: numbers and, yes, integers and double-precision floating point numbers are different but you can live happily for a long time without worrying about this
  • character

Let’s create some simple vectors for more demos below.

Use str() and any other functions you wish to inspect these objects, such as length(), mode(), class(), is.numeric(), is.logical(), etc. Like the family of functions, there are also functions you can experiment with.

17.2 Indexing a vector

We’ve said, and even seen, that square brackets are used to index a vector. There is great flexibility in what one can put inside the square brackets and it’s worth understanding the many options. They are all useful, just in different contexts.

Most common, useful ways to index a vector:

  • logical vector: keep elements associated with TRUE’s, ditch the FALSE’s
  • vector of positive integers: specifying the keepers
  • vector of negative integers: specifying the losers
  • character vector: naming the keepers

17.3 lists hold just about anything

Lists are basically über-vectors in R. It’s like a vector, but with no requirement that the elements be of the same flavor. In data analysis, you won’t make lists very often, at least not consciously, but you should still know about them. Why?

  • data.frames are lists! They are a special case where each element is an atomic vector, all having the same length.
  • many functions will return lists to you and you will want to extract goodies from them, such as the p-value for a hypothesis test or the estimated error variance in a regression model

Here we repeat an assignment from above, using list() instead of c() to combine things and you’ll notice that the different flavors of the constituent parts are retained this time.

List components can also have names. You can create or change names after a list already exists or this can be integrated into the initial assignment.

Indexing a list is similar to indexing a vector but it is necessarily more complex. The fundamental issue is this: if you request a single element from the list, do you want a list of length 1 containing only that element or do you want the element itself? For the former (desired return value is a list), we use single square brackets, [ and ], just like indexing a vector. For the latter (desired return value is a single element), we use a dollar sign $, which you’ve already used to get one variable from a data.frame, or double square brackets, [[ and ]].

The “pepper shaker photos” in R for Data Science are a splendid visual explanation of the different ways to get stuff out of a list. Highly recommended.

Warning: the rest of this section might make your eyes glaze over. Skip to the next section if you need to; come back later when some list is ruining your day.

A slightly more complicated list will make our demos more educational. Now we really see that the elements can differ in flavor and length.

Here’s are ways to get a single list element:

A case when one must use the double bracket approach, as opposed to the dollar sign, is when the indexing object itself is an R object; we show that above.

What if you want more than one element? You must index vector-style with single square brackets. Note that the return value will always be a list, unlike the return value from double square brackets, even if you only request 1 element.

17.4 Creating a data.frame explicitly

In data analysis, we often import data into data.frame via read.table(). But one can also construct a data.frame directly using data.frame().

Sidebar: What is I(), used when creating the variable \(y\) in the above data.frame? Short version: it tells R to do something quite literally. Here we are protecting the letters from being coerced to factor. We are ensuring we get a character vector. Note we let character-to-factor conversion happen in creating the \(v\) variable above. More about (foiling) R’s determination to convert character data to factor can be found here.

data.frames really are lists! Double square brackets can be used to get individual variables. Single square brackets can be used to get one or more variables, returned as a data.frame (though subset(..., select = ...)) is how I would more likely do in a data analysis).

Question: How do I make a data.frame from a list? It is an absolute requirement that the constituent vectors have the same length, although they can be of different flavors. Assuming you meet that requirement, use to convert.

You will encounter weirder situations in which you want to make a data.frame out of a list and there are many tricks. Ask me and we’ll beef up this section.

17.5 Indexing arrays, e.g. matrices

Though data.frames are recommended as the default receptacle for rectangular data, there are times when one will store rectangular data as a matrix instead. A matrix is a generalization of an atomic vector and the requirement that all the elements be of the same flavor still holds. General arrays are available in R, where a matrix is an important special case having dimension 2.

Let’s make a simple matrix and give it decent row and column names, which we know is a good practice. You’ll see familiar or self-explanatory functions below for getting to know a matrix.

Indexing a matrix is very similar to indexing a vector or a list: use square brackets and index with logical, integer numeric (positive or negative), or character vectors. Combine those approaches if you like! The main new wrinkle is the use of a comma , to distinguish rows and columns. The \(i,j\)-th element is the element at the intersection of row \(i\) and column \(j\) and is obtained with jMat[i, j]. Request an entire row or an entire column by simply leaving the associated index empty. The drop = argument controls whether the return value should be an atomic vector (drop = TRUE) or a matrix with a single row or column (drop = FALSE). Notice how row and column names persist and can help you stay oriented.

Under the hood, of course, matrices are just vectors with some extra facilities for indexing. R is a column-major order language, in contrast to C and Python which use row-major order. What this means is that in the underlying vector storage of a matrix, the columns are stacked up one after the other. Matrices can be indexed exactly like a vector, i.e. with no comma \(i,j\) business, like so:

How to understand this: start counting in the upper left corner, move down the column, continue from the top of column 2 and you’ll land on the element “x32” when you get to 7.

If you have meaningful, systematic row or column names, there are many possibilities for indexing via regular expressions. Maybe we will talk about grep later….

Note also that one can put an indexed matrix on the receiving end of an assignment operation and, as long as your replacement values have valid shape or extent, you can change the matrix.

Note that R can also work with vectors and matrices in the proper mathematical sense, i.e. perform matrix algebra. That is a separate topic. To get you started, read the help on %*% for matrix multiplication….

17.6 Creating arrays, e.g. matrices

There are three main ways to create a matrix. It goes without saying that the inputs must comply with the requirement that all matrix elements are the same flavor. If that’s not true, you risk an error or, worse, silent conversion to character.

  • Filling a matrix with a vector
  • Glueing vectors together as rows or columns
  • Conversion of a data.frame

Let’s demonstrate. Here we fill a matrix with a vector, explore filling by rows and giving row and columns at creation. Notice that recycling happens here too, so if the input vector is not large enough, R will recycle it.

Here we create a matrix by glueing vectors together. Watch the vector names propagate as row or column names.

Here we create a matrix from a data.frame.

Here we create a matrix from a data.frame, but experience the “silently convert everything to character” fail. As an added bonus, I’m also allowing the “convert character to factor” thing to happen when we create the data.frame initially. Let this be a reminder to take control of your objects!

17.7 Putting it all together…implications for data.frames

This behind the scenes tour is still aimed at making you a better data analyst. Hopefully the slog through vectors, matrices, and lists will be redeemed by greater prowess at manipulating data.frames. Why should this be true?

  • a data.frame is a list
  • the list elements are the variables and they are atomic vectors
  • data.frames are rectangular, like their matrix friends, so your intuition – and even some syntax – can be borrowed from the matrix world

A data.frame is a list that quacks like a matrix.

Reviewing list-style indexing of a data.frame:

Reviewing vector-style indexing of a data.frame:

Demonstrating matrix-style indexing of a data.frame:

17.8 Table of atomic R object flavors

This table will be hideous unless Pandoc is used to compile.

“flavor” type reported by typeof() mode() class()
character character character character
logical logical logical logical
numeric integer or double numeric integer or double
factor integer numeric factor

This should be legible no matter what.

| "flavor"  | type reported | mode()    | class()   |
|           | by typeof()   |           |           |
| character | character     | character | character |
| logical   | logical       | logical   | logical   |
| numeric   | integer       | numeric   | integer   |
|           | or double     |           | or double |
| factor    | integer       | numeric   | factor    |

Thinking about objects according to the flavors above will work fairly well for most purposes most of the time, at least when you’re first getting started. Notice that most rows in the table are quite homogeneous, i.e. a logical vector is a logical vector is a logical vector. But the row pertaining to factors is an exception, which highlights the special nature of factors. (for more, go here).