Electronic Assessment Materials for Intro Stat

ePort is an R package to generate electronic reports for online homework assignments and vocabulary quizzes. The reports provide statistical summary and analysis on the scores at different hierarchies. The reports can be created immediately after every homework deadline during the teaching semester, which asists the instructors in assessing and improving their teaching in a fast response cycle. This page introduces the usage of the package. We are currently preparing ePort for submission to the CRAN, with an expected submission date in early 2016. Updates will be posted here when the package is approved and available for download.

Input

Homework/test data file(s)
Answer key
Learning objective(s)

Homework/test data file(s) should be downloaded from Blackboard Learn. The download option must be by question and student, and in the .csv format.

The answer key should be exported as a word document from Respondus. Then the document must be saved as an .htm file. This step is to extract and save the images from the answer key, which will be used later in the reports.

The learning adjective(s) must be prepared in a .txt file with the format like the example below.

objectives_example.txt

A. Identify the context of the data by answering the 5 W questions.
B. Identify the "who" and "what" questions of data based on the structure of the data.
C. Determine the structure of the data table based on given information.
D. ...

Generate Report

When the input files are ready, we can run the following code to create an individual homework report of Topic 1, Section AB:

# Get the path for the answer key 
key_hml = "~/Dropbox/NSF Grant 2013-2015/Semesters/Spring 2015/Database Questions/Topic01.Questions.htm"

# Generate a clean answer key with paths to the plots in questions 
refine_key(key_htm)

# Get the path/name of the new answer key 
key_txt = gsub("htm$", "txt", key_htm)

# Get the path for a data file
datapath = "~/Dropbox/NSF Grant 2013-2015/Semesters/Spring 2015/Data Files/Topic01.AB.csv"

# Get the path for the learning objective
LOpath = "~/Dropbox/NSF Grant 2013-2015/Semesters/Spring 2015/Topic Outcomes/Topic01.Outcomes.txt"

# Run a short report
report_routine(key_txt,datapath,rewrite=FALSE,LOfile=LOpath,
               knitfile="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst/Rnw/hw-individual-short.Rnw",
               knito="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst")

# Run a full report
report_routine(key_txt,datapath,rewrite=FALSE,LOfile=LOpath,
               knitfile="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst/Rnw/hw-individual.Rnw",
               knito="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst")

At this point the .tex files and graphs have been created and are in the folder "~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst". Running

pdflatex Stat101hwk_Topic01_AB-short.tex

and

pdflatex Stat101hwk_Topic01_AB.tex

will generate the .pdf reports.

Notes

refine_key(key_htm) is used to clean the html code in the answer key and create a .txt key. This line only needs to run ONCE.
Before running any short or full reports, the data files must be checked - sometimes the downloaded files have the quote sign problem that stops R from reading the files. If that problem exists, the following line must be executed to rewrite the data files. Note that it can only be run ONCE: for (i in namelist) rewrite_data(i)

Generate Reports in Batches for a Topic

When there are a few sections for the same course, it is inconvenient to change the path for the data files sequentially. To save time, we can generate reports in batches as follows.

# Get a vector of data files
datapath = "~/Dropbox/NSF Grant 2013-2015/Semesters/Spring 2015/Data Files"
namelist = list.files(path=datapath,full.names=TRUE)
namelist = namelist[grep('Topic01\\.',basename(namelist))]

# Run the short reports in batches (by loop)
for (i in namelist) 
  report_routine(key_txt, datafile=i, rewrite=FALSE, LOfile=LOpath,
                 knitfile="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst/Rnw/hw-individual-short.Rnw",
                 knito="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst")

# Run the full reports in batches (by loop)
for (i in namelist) 
  report_routine(key_txt, datafile=i, rewrite=FALSE, LOfile=LOpath,
                 knitfile="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst/Rnw/hw-individual.Rnw",
                 knito="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst")

# Run `pdflatex` for all the .tex files

Report Across Sections for a Topic

To compare different sections (or instructors), we can run the report over sections for one topic. The input files key_txt, namelist, and LOpath should be the same as above (reports in batches for a topic).

# Run a short report
report_routine(key_txt, namelist, LOfile=LOpath, 
               knitfile="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst/Rnw/hw-section-short.Rnw",
               knito="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst")

# Run the full report
report_routine(key_txt, namelist, LOfile=LOpath, 
               knitfile="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst/Rnw/hw-section.Rnw",
               knito="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst")

Merge Reports Across Topics

After teaching a unit or module, an instructor may want to summarise students' performance over the topics in the unit. When the semester is over, it is also important to track students' studying pattern for the instructors or course coordinator.

# Merge the desired data files together

# Collect all the data files under the path
tabfiles = set_dir(datapath)

# If only one unit of data files is required, or some sections
# are not desired, the following line will take a subset
tabfiles = tabfiles[tabfiles$section!='201' & tabfiles$topic %in% paste0('Topic0',1:5),]

# Merge the data files, it may take some time
mgdata = merge_data(tabfiles)

# Run the sectional reports in batches
for (sctn in unique(tabfiles$section)) {
  merged = subset_data(mgdata,tabfiles,choice=sctn)
  knit("Rnw/hw-topic.Rnw",output=paste0('Stat101hwk_Unit1_Section',sctn,'.tex'))
}

# Run one report for all sections
merged = subset_data(mgdata,tabfiles)
knit("Rnw/hw-topic-section.Rnw",output='Stat101hwk_Unit1_allSections.tex')

Reports for the Unit Tests

This part is quite similar as the procedure for generating the homework reports for one topic. Three components of the inputs are still needed, and the code to run the reports are almost the same.

The only thing should be awared of is the path for learning objectives. Since a unit will cover the materials from multiple topics, the corresponding learning objective files should be appointed.

# Set up the input path
key_htm = "~/Dropbox/NSF Grant 2013-2015/Semesters/Fall 2014/Unit Assessment Questions/Unit1Assessment.htm"
refine_key(key_htm)
key_txt = gsub("htm$","txt",key_htm)

datapath = "~/Dropbox/NSF Grant 2013-2015/Semesters/Fall 2014/Unit Assessment Data Files"
topic = gsub('Assessment.txt','',gsub('Unit','',basename(key_txt)))
namelist = list.files(path=datapath,full.names=TRUE)
namelist = namelist[grep(paste('Unit',topic,'\\.',sep=''),basename(namelist))]

# choose the corresponding topics
LOpath = list.files(path="~/Dropbox/NSF Grant 2013-2015/Semesters/Fall 2014/Topic Outcomes",
                    pattern='\\.txt$',full.names=TRUE)[1:4]

# Run the individual unit reports in batches
for (i in namelist) 
  report_routine(key_txt, datafile=i, rewrite=FALSE, LOfile=LOpath, 
                 knitfile="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst/Rnw/unit-individual.Rnw",
                 knito="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst")

# Run a report across sections
report_routine(key_txt, namelist, LOfile=LOpath, 
               knitfile="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst/Rnw/unit-section.Rnw",
               knito="~/Dropbox/NSF Grant 2013-2015/Reports/Code/ePort/inst")

Code to Solve Some Common Problems

Merge data files

datapath = "~/Dropbox/NSF Grant 2013-2015/Semesters/Fall 2014/Data Files/Ch9-Ch11"
namelist = list.files(path=datapath,full.names=TRUE)
for (i in namelist) rewrite_data(i) # May not need this rewriting step!
for(i in c('AB','CD','EF','GH','JK','LM')){
  tmp = namelist[grep(i,basename(namelist))]
  combine_files(tmp[2],tmp[1],paste("Topic11",i,"csv",sep='.'))
}

Split data files

datapath = "~/Dropbox/NSF Grant 2013-2015/Semesters/Spring 2015/Data Files/Ch3"
namelist = list.files(path=datapath,full.names=TRUE)
for (i in namelist) rewrite_data(i) # May not need this rewriting step!
for(i in c('AB','CD','EF','GH','JKQ','LM')){
  tmp = namelist[grep(i,basename(namelist))]
  split_file(tmp, 29, "ID")
}

De-identify students

datapath = "~/Dropbox/NSF Grant 2013-2015/Semesters/Fall 2013/Data Files (de-identified)"
a = getNameList(datapath, section=NULL, semester="13f", secblind=TRUE, save=TRUE)
encodeName(datapath, dict=paste(datapath,"nameCode.csv",sep='/'))

Merge sections

key_txt = "~/Dropbox/NSF Grant 2013-2015/Semesters/Spring 2015/Database Questions/Topic05.Questions.txt"
datapath = "~/Dropbox/NSF Grant 2013-2015/Semesters/Spring 2015/Data Files"
topic = "05"
namelist = list.files(path=datapath,full.names=TRUE)
namelist = namelist[grep(paste('Topic',topic,'\\.',sep=''),basename(namelist))]

answerkey = convertkey(key_txt)
instructor_scores = merge_section(namelist, answerkey,skip=NULL)
rownames(instructor_scores$score) = instructor_scores$score$student
tmp1 = instructor_scores$Qscore
l = ncol(tmp1)
tmp1$TotalScore = rowSums(tmp1, na.rm=TRUE)
tmp1$Section = instructor_scores$score[rownames(tmp1),'Section']
tmp1 = tmp1[,c(l+2,l+1,1:l)]
tmp2 = instructor_scores$Qanswer
tmp2$Section = instructor_scores$score[rownames(tmp2),'Section']
tmp2 = tmp2[,c(l+1,1:l)]
write.csv(tmp1,file='Topic3.score.csv')
write.csv(tmp2,file='Topic3.answer.csv')

ePort | Generate Reports for Intro Stat Students and Instructors