2. Downloading and loading data
A public clinical dataset, checked before analysis
For function and argument descriptions, open Reference tables in the Lesson companion.
A data frame stores variables in columns and observations in rows. We’ll read the prepared CSVs from the Moorfields study by Fasler and colleagues (2019), then check what one row represents. The data notes document their preparation.
Read the baseline CSV
read.csv() imports the file; <- assigns the result to cohort. na.strings treats empty fields and "NA" as missing. head(..., 3) shows three rows of the selected columns. Change 3 to 10 and compare.
Each row is one patient with an exact day-zero visual acuity measurement. The browser already has this file, so no upload is needed.
| Column | Meaning |
|---|---|
patient_id |
Published deidentified ID, used to check or join rows |
age |
Recorded baseline age in years, coarsened for privacy |
sex |
Recorded sex, Female or Male |
baseline_va |
ETDRS letter score; higher means better visual acuity |
baseline_vision |
Poor (≤35), Intermediate (>35 and <70), or Good (≥70 letters) |
In the source, ages at or below 65 are recorded as 60, and ages over 90 as 99. These clusters reflect coarsening, not exact patient ages. We therefore label summaries recorded baseline age.
For local R, download moorfields_baseline.csv and moorfields_followup.csv into your Project’s data/ folder.
Where R looks for files
The working directory is the starting folder for a relative path such as "data/moorfields_baseline.csv".
The last command should return TRUE. Change .csv to .cvs: what happens? In the browser, these files are in a temporary file system. Locally, open the Project so R uses its folder.
Check the size and the unit of observation
The prepared cohort has 3,355 rows and five columns, with no repeated IDs. The original data contain 3,357 patients; two lack an exact day-zero measurement. Repeated visit rows would give frequent attenders extra weight in a patient summary.
Inspect types, missing values, and ranges
str() shows each column’s type. A column expected to be numeric but read as text needs investigation before analysis.
This prepared file has no missing values. That doesn’t make the original records or later visits complete. Visual acuity is a letter-chart score, not a percentage; check its range against the data notes.
Question: which command above counts missing values, and which counts patients in each vision group? Try summary(cohort$age) as another check.
Set the category order
A factor stores categories in a chosen order. Without these levels, text would sort alphabetically, with Good before Poor. $ selects one column.
A second file: one row per patient and visit
Here a patient can have a baseline row and selected year-one and year-two rows. Annual observations are the nearest valid visits to 52 and 104 weeks, within eight weeks on either side. A missing visit contributes no row; it isn’t a zero acuity score.
This file illustrates repeated observations. Table 1 uses the one-row-per-patient baseline file; part 4 switches to the Ghana sepsis data.
Try it: count the vision groups
Fill the blank with the grouping column’s name.
Use baseline_vision. $ selects a column.
table(cohort$baseline_vision, useNA = "ifany")There are 451 patients in Poor, 2,071 in Intermediate, and 833 in Good.
Useful read.csv() arguments
nrows limits the import to five rows; colClasses keeps the ID as text. This preserves leading zeros such as "00123". Change 5 to 20, then check dim(preview).
Common errors when loading data
- File not found: use
getwd()andlist.files("data"); check spelling and hidden extensions such asresults.csv.csv. '\U' used without hex digits: replace Windows backslashes ("C:\Users\...") with forward slashes ("C:/Users/...").- Everything in one column: check the separator; try
read.csv2()below. - Numbers read as text: inspect entries with
unique()ortable(); values such as"<5","n/a", or spaces may need cleaning orna.strings. - Garbled accents: try
fileEncoding = "UTF-8"or"latin1"for older exports. - Excel dates read as numbers such as 45231: readxl usually converts them; otherwise use
as.Date(x, origin = "1899-12-30").
Optional: other file formats
The following examples write a copy of cohort, then read it back. Run Set the category order first. With your own files, start at the reading step. The CSVs already bundled with this page are sufficient for the seminar.
Built-in practice datasets
R also includes mtcars and iris for practice.
Semicolon and tab-separated files
The unmodified Moorfields source uses semicolons. Our prepared CSVs use commas. read.csv2() expects semicolons with decimal commas; read.delim() reads tab-separated files. A different separator can be set with read.csv(file, sep = "|").
readLines() shows the raw separators. Compare it with the imported rows.
Try it: read the semicolon file
The exercise setup writes cohort_semicolon.csv for you. Fill the blank with the reader for that separator.
The semicolon reader’s name has one extra character after read.csv.
semicolon <- read.csv2("cohort_semicolon.csv")
dim(semicolon)read.csv2() uses sep = ";" and dec = "," by default. Those arguments also work with read.csv().
R’s own format: .rds
RDS retains the object and its factor levels. CSV returns the category as text, so its order must be set again. Use RDS between R scripts and CSV when sharing with other software.
Install the extra packages
Locally, use install.packages(c("readr", "readxl", "writexl", "haven")) once. The cell above installs browser builds. If it fails here, continue with the CSV examples or use local R.
A faster CSV reader: readr
readr::read_csv() returns a tibble, a compact data frame, and prints guessed types. show_col_types = FALSE hides that report. It uses na instead of na.strings and preserves names with spaces, which need backticks when used in code, such as `Visual acuity`.
Excel workbooks
excel_sheets() lists available sheets; sheet selects one. Try its name instead of 1, or add range = "A1:C6". Merged headers, notes, and colours aren’t a clean data structure; select a rectangular range or request a CSV export.
SPSS, Stata, and SAS files
SPSS stores numbers with value labels, such as 1 = Poor. haven preserves these labels; as_factor() turns them into R categories. SAS transport files (.xpt) are often used in clinical-trial submissions.
From a web address
url <- "https://example.org/study/data.csv" # replace with the real address
remote <- read.csv(url)
# Keep a dated local copy, so the analysis doesn't change if the online file does
local_file <- paste0("data/download_", Sys.Date(), ".csv")
download.file(url, local_file, mode = "wb")
remote <- read.csv(local_file)Replace the example URL with a real address. A dated local copy prevents later online changes from silently changing your analysis. mode = "wb" keeps downloaded bytes unchanged, including Excel files on Windows. Browser downloads also depend on the host allowing cross-origin requests.
Other sources
The data import cheatsheet summarizes readr and readxl.