10  Miscellaneous

10.1 Search for patterns with grep and grepl

Oftentimes you may need to filter a data set based on a partial character string that is beyond the scope of comparison operators.

R provides such functions (grep and grepl) that match character patterns in specified vector. While both of these functions find patterns, they return different output types based on those patterns.

  • grep returns numeric values that correspond to the indexed locations of the patterns
  • grepl returns a logical vector in which ‘TRUE’ represents a pattern match.

In our growth data, we only want to print measurements of our controls (i.e. the mock treatments). Our table has many rows, so we only look at the first few results with head().

#find the row index values with mock treatments using grep
head(grep("mock", growth_data$Condition))
[1] 1 2 3 4 5 6

To see how grepl works, let’s first try it on a short vector with our three conditions. Both strains (129E and 13C) contain a 1, the mock treatment does not:

#use grepl to search for conditions that contain a 1, returning FALSE/TRUE statements for each element
#with "TRUE" representing matched patterns
grepl("1", c("mock", "129E", "13C"))
[1] FALSE  TRUE  TRUE

On our dataframe, grepl gives one TRUE or FALSE for each of the 594 rows. Instead of printing them all, we can count the TRUE values with sum(), i.e. the number of plants treated with a strain:

#count the rows where the condition contains a 1
sum(grepl("1", growth_data$Condition))
[1] 385

Now lets use this to actually filter our data table for a pattern.

10.1.1 Filter using grep

#use grep
filter_for_value <- timecourse[grep("mock", timecourse$Condition),]

#check the first rows of our data
head(filter_for_value)
#check the dimensions of the original dataframe
dim(timecourse)
[1] 2323    4
#check the dimensions of our new dataframe
dim(filter_for_value)
[1] 1156    4

10.1.2 Filter using grepl

filter_for_value <- timecourse[grepl("moc", timecourse$Condition),]

#check the first rows of our data
head(filter_for_value)
#check the dimensions of the original dataframe
dim(timecourse)
[1] 2323    4
#check the dimensions of our new dataframe
dim(filter_for_value)
[1] 1156    4

10.1.3 Discard matching rows

#filter data set based on values that do not match the specified pattern (by using the ! symbol)
filter_for_not_a_value <- timecourse[!grepl("mock", timecourse$Condition),]

#view dataframe
head(filter_for_not_a_value)
#check the dimensions of the original dataframe
dim(timecourse)
[1] 2323    4
#check the dimensions of our new dataframe
dim(filter_for_not_a_value)
[1] 1167    4

Other comments:

  • Using regular expressions (programming symbol pattern) will increase their functionality
  • Specified patterns are case sensitive (‘t’ does not equal ‘T’)
  • Any matching pattern will be returned despite the context in which that pattern is located (i.e., grep(‘the’, data) with return matches for ‘the’, ‘theme’, ‘heather’, ‘breathe’, and so on. This is where regular expressions are useful for specifying where in a string the pattern should appear.

10.1.4 Using regular expressions

A regular expression (regex or regexp for short) is a special text string for describing a search pattern. You can think of regular expressions as wildcards on steroids. You are probably familiar with wildcard notations such as *.txt to find all text files in a file manager.

As an example, let’s grep the strains that have a number with 3 digits.

If you want to know more, see this cheat sheet here

filter_3digits <- growth_data[grepl("[0-9]{3}", growth_data$Condition),]

#check the structure of our data
head(filter_3digits)
#check the dimensions of the original dataframe
dim(growth_data)
[1] 594   5
#check the dimensions of our new dataframe
dim(filter_3digits)
[1] 210   5

Here, [0-9] searches for every number from 0-9 and with {3} we look for three numbers in a row. 129E matches, but 13C does not, because it only has two numbers.

10.2 Search and replace in markdown

If we want to search for the end of the line to, for example, add two spaces (and thus create a linebreak), we can:

  • find `(.)$ with Regex selected
  • and replace with \1your_string