Before the actual exercise, open a view to the Allas service in your browser using the cPouta web interface.
Object Store > Containers
Note that you need to reload the view in order to see changes
ssh <username>@puhti.csc.fi # replace <username> with your CSC username
csc-workspaces
/scratch
directory of your projectcd /scratch/<project> # replace <project> with your CSC project, e.g. project_2001234
mkdir -p $USER
cd $USER
wget
wget https://a3s.fi/course_12.11.2019/pythium.tgz
tar -xzvf pythium.tgz
tree pythium
module load allas
allas-conf
rclone
rclone
:rclone -P copyto pythium allas:$USER-genomes-rc/
rclone lsd allas:
rclone ls allas:$USER-genomes-rc/
rclone lsl allas:$USER-genomes-rc/
rclone lsf allas:$USER-genomes-rc/
$USER-genomes-rc
bucket and download one of the uploaded fasta files to your local computer💡 You can read more about moving files at Docs CSC: Copying files using scp and Moving data with rclone
a-put
<project number>
with your CSC project number, e.g. 2001234):a-put pythium
a-list
a-list <project number>-puhti-SCRATCH
a-info <project number>-puhti-SCRATCH/$USER/pythium.tar
<project number>
with your CSC project number, e.g. 2001234):a-put pythium/*
a-list <project number>-puhti-SCRATCH
a-check pythium/*
a-info <project number>-puhti-SCRATCH/$USER/pythium/pythium_vexans.tar
<project number>
with your project number, e.g. 2001234):a-put pythium/* -b <project number>-$USER-genomes-ap
a-list <project number>-$USER-genomes-ap
a-put
commands above?<project number>-$USER-genomes-ap
bucket with commands:a-list <project number>-$USER-genomes-ap
rclone ls allas:<project number>-$USER-genomes-ap
<project number>
with your project number, e.g. 2001234):a-info <project number>-$USER-genomes-ap/pythium_vexans.tar
rclone cat allas:<project number>-$USER-genomes-ap/pythium_vexans.tar_ameta
a-flip pythium/pythium_vexans/pythium_vexans.fasta
a-flip
produced with your browserallas-backup
allas-backup -help
allas-backup pythium
allas-backup list
pythium
directory is now stored in many ways in Allas, so we can remove the data from Puhti and log out:rm -r pythium
exit
/scratch
:ssh <username>@puhti.csc.fi # replace <username> with your CSC username
cd /scratch/<project>/$USER # replace `<project>` with your CSC project, e.g. project_2001234
csc-workspaces
module load allas
allas-conf
a-list
rclone lsd allas:
# replace <project number> with your project number, e.g. 2001234
a-list <project number>-$USER-genomes-ap
rclone ls allas:<project number>-$USER-genomes-ap
a-find pythium_vexans.fasta
a-find -a pythium_vexans.fasta
rclone
mkdir rclone_dir
cd rclone_dir/
mkdir all
rclone ls allas:<project number>-$USER-genomes-ap
rclone copyto -P allas:<project number>-$USER-genomes-ap all/
ls all
mkdir vexans
rclone copyto allas:$USER-genomes-rc/pythium_vexans vexans/
ls vexans
rclone copyto allas:$USER-genomes-rc/pythium_vexans/pythium_vexans.fasta ./vexans.fasta
ls
a-get
$USER
directory under your project’s /scratch
on Puhti (The pwd
command should print /scratch/<project/$USER
):cd ..
pwd
mkdir a_dir
cd a_dir/
all
and move there:mkdir all
cd all
SCRATCH
bucket (replace <project number>
with your project number, e.g. 2001234):a-list <project number>-puhti-SCRATCH
a-list <project number>-puhti-SCRATCH/$USER
pythium_vexans.fasta
in your Puhti SCRATCH
bucket:a-find pythium_vexans.fasta -b <project number>-puhti-SCRATCH # replace <project number> with your project number, e.g. 2001234
a-get <project number>-puhti-SCRATCH/$USER/pythium.tar # replace <project number> with your project number, e.g. 2001234
ls -l
ls -R
cd ..
a-get <project number>-puhti-SCRATCH/$USER/pythium/pythium_vexans.tar # replace <project number> with your project number, e.g. 2001234
ls -l pythium/
ls -l pythium/pythium_vexans/
allas-backup
cd ..
mkdir a_backup
cd a_backup/
allas-backup list
allas-backup list | grep $USER
allas-backup restore
to download the data:allas-backup restore <id string> # replace <id string> with the ID of your backup snapshot
ls -l
ls -l pythium