MBG wiki: Editing Backing-up molecular dynamics simulation data

As yet there is no DVD writter available, so you will have to use CDs. Plenty of CDs. The basic idea is that you should back-up everything : the system preparation files (usually in your home directory), the NAMD and SGE files, the trajectories, the restart files, the LOG files, everything. You will also have to prepare and back-up some partially processed trajectory files (for example, a solute-only trajectory file). The name of the game is that you can only fit 700 MB in one CD, which means that at least for the trajectory you will have to divide the file(s) in pieces small enough to fit a CD.

What the previous paragraph imply then, is that you will have to be awake (have a cup of coffee _now_) because if you mess it up and then delete the files, there is no turning back.

Ok, now that you had your cup of coffee :

* Make an easy start : go to your home directory, find the directory you've used for preparing the system (let's say it's name is *prepfiles*) and back-it up :
<code>
tar cvf preparation.tar prepfiles/
WriteCD preparation.tar
mount /mnt/cdrom/ 
diff /mnt/cdrom/preparation.tar ./
               
                           [if diff fails, your backup failed]

umount /mnt/cdrom/
ls -alFh preparation.tar
rm preparation.tar
</code>
Write down the size of the file you've written so that you know how much empty space you have on that CD. Mark the CD /in full/. /In full/ means something in the spirit of : 
<code>
MD of A31P Rop mutant in complex with RNA stem-loop (sequence ...)
preparation.tar : all files from system preparation, 2.8 MBytes
23rd June 2006, MyName
</code>

* *Go to an idle machine for the back-ups that follow*. For example PC8.

* Now, it is time to back-up the full trajectory files. Go to your area in /work. You should be seeing the various directories (heat, equi, restart1, restart2, ...). Each of these contains a trajectory file in their output/ subdirectory. The first thing is to find how many frames you can fit in one CD . Find a big trajectory file (let's say the one in restart2/) and calculate number of bytes per frame :
<code>
cd restart2/output/
catdcd -num equi_out.dcd
ls -lF equi_out.dcd
</code>
Use a calculator (xcalc) to divide the size of the DCD with the number of frames reported by catdcd. Calculate the (number of frames per CD) : 
<code>
[700 * 1024 * 1024] / [size of DCD / number of frames]
</code>
Beautify the number : if it is 1236.37, you can safely fit 1200 frames per CD. Now, go for it :
<code>
cd heat/output/
ls -lF heat_out.dcd
</code>

* If it is less than 700 MBytes, just write it :
<code>
cp heat_out.dcd ~
cd
catdcd -num heat_out.dcd
mv heat_out.dcd A31P_RNA_complex_Heat_1_730.dcd
WriteCD A31P_RNA_complex_Heat_1_730.dcd
mount /mnt/cdrom/ 
diff /mnt/cdrom/A31P_RNA_complex_Heat_1_730.dcd ./
               
                           [if diff fails, your backup failed]

umount /mnt/cdrom/
rm A31P_RNA_complex_Heat_1_730.dcd
</code>
Mark the CD :
<code>
MD of A31P Rop mutant in complex with RNA stem-loop (sequence ...)
Heating phase trajectory including waters and ions (full system)
Frames 1 to 730 (complete heating phase)
23rd June 2006, MyName
</code>

* If it is a big file you will have to write it in pieces as shown below. <b>PLEASE NOTE : if you have made restarts during your simulation, then there will be few frames at the end of each restart that you will have to _delete_. The reason is that the restart files written by NAMD have a period of 1000 steps, whereas the period for writing to the DCD is 200 steps</b>
<code>
cd restart2/output/
catdcd -o ~/A31P_RNA_complex_Rest2_1_1200.dcd -last 1200 equi_out.dcd
cd
ls -alFh A31P_RNA_complex_Rest2_1_1200.dcd       (is it less than 700 MB ?)
WriteCD A31P_RNA_complex_Rest2_1_1200.dcd
mount /mnt/cdrom/ 
diff /mnt/cdrom/A31P_RNA_complex_Rest2_1_1200.dcd ./
               
                           [if diff fails, your backup failed]

umount /mnt/cdrom/
rm A31P_RNA_complex_Rest2_1_1200.dcd

MARK THE CD : 
MD of A31P Rop mutant in complex with RNA stem-loop (sequence ...)
Second restart trajectory including waters and ions (full system)
Frames 1 to 1200 of second restart (Frames 3451-4650 of whole simulation)
23rd June 2006, MyName

REPEAT:

cd restart2/output/
catdcd -o ~/A31P_RNA_complex_Rest2_1201_2400.dcd -first 1201 -last 2400 equi_out.dcd
cd
ls -alFh A31P_RNA_complex_Rest2_1201_2400.dcd       (is it less than 700 MB ?)
WriteCD A31P_RNA_complex_Rest2_1201_2400.dcd
mount /mnt/cdrom/ 
diff /mnt/cdrom/A31P_RNA_complex_Rest2_1201_2400.dcd ./
               
                           [if diff fails, your backup failed]

umount /mnt/cdrom/
rm A31P_RNA_complex_Rest2_1201_2400.dcd

MARK THE CD : 
MD of A31P Rop mutant in complex with RNA stem-loop (sequence ...)
Second restart trajectory including waters and ions (full system)
Frames 1201 to 2400 of second restart (Frames 4651-5850 of whole simulation)
23rd June 2006, MyName

REPEAT ...

</code>
You should be careful with the last set of frames : the number of frames in the last set will not be exactly 1200, but something like 735 (depending on the time step that the job stopped). Try to get the arithmetic correct, especially the number of frames for the whole simulation. Check the numbers again. Once more.

* Repeat until you have saved all trajectory frames.

* Before deleting the DCD files from the server disks, you should prepare a solute-only trajectory and back-it-up (solute-only means without the waters or ions). To do this you can use /catdcd/ or /carma/ or ... The following shows how you can do the trick with /carma/ :
<code>
cd

carma -v -fit -atmid "ALLID" -segid " A   " -segid " B   " /work/mydir/heat/output/heat_out.dcd /work/mydir/heat/ionized.psf
rm carma.fit-rms.dat
mv carma.fitted.dcd 1.dcd

carma -v -fit -atmid "ALLID" -segid " A   " -segid " B   " /work/mydir/equi/output/equi_out.dcd /work/mydir/heat/ionized.psf
rm carma.fit-rms.dat
mv carma.fitted.dcd 2.dcd

carma -v -fit -atmid "ALLID" -segid " A   " -segid " B   " /work/mydir/restart1/output/equi_out.dcd /work/mydir/heat/ionized.psf
rm carma.fit-rms.dat
mv carma.fitted.dcd 3.dcd

carma -v -fit -atmid "ALLID" -segid " A   " -segid " B   " /work/mydir/restart2/output/equi_out.dcd /work/mydir/heat/ionized.psf
rm carma.fit-rms.dat
mv carma.fitted.dcd 4.dcd

...

catdcd -o solute.dcd 1.dcd 2.dcd 3.dcd ...

catdcd -num solute.dcd

</code>
The segids (A and B) should be changed to match the segids of your solute. Make sure that the total number of frames (reported by the last /catdcd/ command) is indeed the one you would expect for your simulation.

Summary:

This change is a minor edit.

Username:

Replace this text with a file.