# Data Management – Script for Compiling qPCR Data

Over the last few weeks, I’ve wrestled with tracking down data (primarily qPCR data) from the litany of projects the Friedman Lab has had over the last decade or so. I’ve also noticed that it’s increasingly difficult for me to track down my own data from individual projects where data is not generated continuously over time, but in chunks. Tracking down 6 different qPCR runs that were conducted over the course of a year is tedious.

In order to save myself, and others who might need/want to review my data, a lot of time in the future, I decided to write a script that will allow me to compile all of my qPCR data into a single, massive CSV file. Accessing this CSV file via spreadsheet (Excel/Calc/Sheets) or database (SQL) means I’ll always be able to quickly search for all the related data I need, since it will reside in a single file!

The script is written in bash, called qpcr_aggregation.sh, and is currently hosted here: https://github.com/kubu4/Scripts/blob/master/bash/qpcr_aggregation.sh

Basically, the script does the following:

• Replace the spaces in the BioRad filenames with underscores (used to simplify parsing of the filename for use in downstream steps in the script)
• Replace the header row to accommodate two new fields: qPCR_filename and qPCR_date
• Add qPCR_filename and qPCR_date to each file.
• Concatenate all the files into a single “master” CSV file.

Now, the easy part – exporting the data from hundreds of qPCR files…

# Automatic Notebook Backups – wget Script & Synology Task Scheduler

## UPDATE 20150714 – READ ENTIRE POST

I’ve been tweaking a shell script (notebook_backups.sh) to use the shell program wget to retrieve fully functional HTML versions of our online notebooks for offline viewing. I had been planning on setting up a cron job to automatically run this script on our Synology server (Eagle) at a set day/time. However, I came across the Task Scheduler that’s built right into the Synology GUI! So, I set up the Task Scheduler to run the notebook_backups.sh script every Sunday. See screenshots below.