{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "# First steps on Linux - On Your Own\n", "\n", "The program is :\n", "* using linux commands seen previously and new linux commands\n", "* Counting number of reads, of files\n", "* Plot some basics stats\n", "* Recover sequences of interest from a GFF (capture seq)\n", "* Compare two files (BED)\n", "* Bonus: create a mini workflow with pipe" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# where are you? - `pwd`\n", "\n", "Check where you are in the linux filetree\n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "slideshow": { "slide_type": "slide" } }, "outputs": [], "source": [ "PUT YOUR COMMAND HERE" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# List the content of the directory SV_DATA - `ls`\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND HERE" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# List the content of the directory SHORT_READS - `ls -l`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND HERE" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__EXERCICES:__\n", "\n", "* 1 - List the content of the directory REF by using its absolute path in first then its relative path - `ls` command\n", "* 2 - List the content of the directory LONG_READS with the `ls` command and the option `-R`\n", "* 3 - List the content of the directory ifbdata/public/swissprot with the `ls` command and the option `-al` or `-a -l`\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Working with the wildcard `*`\n", "\n", "\n", "#### What is a wildcard ?\n", "\n", "* It is a character that can be used as a substitute for any of a class of characters in a search\n", "\n", "* Can be used with numerous linux commands \n", "\n", "#### How to use the wildcard `*`\n", "\n", "* \\* can represent zero characters, all single characters or any string\n", "\n", "###### Example\n", "`ls *fastq` will display __only the files ended by fastq__ in the directory where I am working1\n", "\n", "#### Try the following commands using (or not) the wildcad *" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ls /home/jovyan/SV_DATA/REF" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "ls /home/jovyan/SV_DATA/REF/*fna" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "__EXERCICES__\n", "\n", "* 4 - List the content of the directory SHORT_READS. - `ls command`\n", "* 5 - List only the files starting by B (in the directory SHORT_READS ) - `ls command & *`\n", "* 6 - List only the files with the word R1 (in the directory SHORT_READS ) - `ls command & *`\n", "\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Moving into file system using `cd` and `ls` command \n", "\n", "\n", "\n", "#### EXERCICES\n", "\n", "* 7 - Go to the directory _REF_ and check in the prompt you have changed correctly your working directory `pwd`\n", "* 8 - List the dir content with `ls`\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " #### Let's use the command `cd ..`\n", "\n", "* `cd ..` : go one folder up\n", "* `cd ../..` : go two folders up\n", "\n", "#### EXERCICES\n", "\n", "* 9 - Go to the _LONG_READ/A8_ directory using `../`\n", "* 10 - Go to the _SHORT_READS_. From this directory, and without any change in your working dir, list what’s in _REF directory_\n", "* 11 - List the _ifbdata/public/_ directory using `-R option`. What is there in this dir ?\n", "* 12 - Come back to your home directory using cd" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Some Commands\n", "\n", "\n", "| | |\n", "|:------|:------|\n", "| pwd | Display the full path of the current directory|\n", "| ls | List all files/directories|\n", "| cd DIR_NAME| Change the working directory|\n", "| rm nom_fichier | Remove a directory\n", "| cp file1 file2 | Make a copy of file1 and calls it file2 |\n", "| cp FILE_NAME DIR_NAME | Copy the file FILE_NAME in the directory DIR_NAME |\n", "| cp FILE DIR/NEW_FILE | Mix of 2 |\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Manipulating Files and Folders\n", "\n", "#### Exercices\n", "\n", "```{Note}\n", "Be careful with the `rm` command, in linux, there is no bin !!!\n", "```\n", "\n", "* 13 - Remove the file SV-DATA.tar.gz within your home directory - `rm`\n", "* 14 - Copy the files Day1_Get_Data.ipynb and myFirstJupyterbook.ipŷnb into the directory NOTEBOOK\n", "* 15 - Remove the files Day1_Get_Data.ipynb and myFirstJupyterbook.ipynb in your home directory - `rm command`\n", "* 16 - Create a directory called SCRIPT in your home directory - `mkdir`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n", "# Let's practice some bioinformatics tools" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### TIPS\n", "\n", "For saving the output of a command to a file, we will use the `>` character \n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1 - Manipulation of `gff` and `bed` files \n", "\n", "* Recover sequences of interest from a GFF (capture seq)\n", "* Compare two files (BED)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Download the file that contains the rice genome annotation (gff format) and the sequence of the 12 chromosomes (fasta format) - `wget`\n", "* the url of the fasta file (all.chr.con):\n", "http://rice.uga.edu/pub/data/Eukaryotic_Projects/o_sativa/annotation_dbs/pseudomolecules/version_7.0/all.dir/all.chrs.con\n", "\n", "\n", "* the url of the gff file (all.gff3) : http://rice.uga.edu/pub/data/Eukaryotic_Projects/o_sativa/annotation_dbs/pseudomolecules/version_7.0/all.dir/all.gff3\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### EXERCICE : \n", "\n", "* 1 - Go to the bank directory and type the correct wget command\n", "* 2 - Download the gff file (all.gff3) and the fasta file (all.chrs.con) in the bank directory \n", "* 3 - List the content of the bank directory\n", "* 4 - Count the line number of the gff file\n", "* 5 - Displays the firsts and lasts lines of the gff file\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### More information about gff format\n", "\n", "* the gff (General Feature Format) file is a simple tab-delimited text file for describing genomic features \n", "\n", "* see here to understand the gff format : https://www.ensembl.org/info/website/upload/gff.html" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### Counts the number of gene (line with the word \"gene\", column 3)\n", "\n", "* We will use a new linux command `grep` \n", "* for example, we want to count the number of sequences in the fasta file just downloaded " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "head -n 20 all.chrs.con" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "tail -n 20 all.chrs.con" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "wc -l all.chrs.con" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### Search and count for > motif in the rice genome (fasta)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "grep \">\" all.chrs.con" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "grep \">\" all.chrs.con -c" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### Search for the keyword `exon` in the gff file" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "grep \"exon\" all.gff3 | head" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "grep \"exon\" all.gff3 -c" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### EXERCICE" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* 1 - search/select only the lines with the gene keyword int the gff file - `grep`\n", "* 2 - save in a new file (all_gene.gff3) all the lines with the gene keyword in the gff file all.gff3\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### Compare two bed files\n", "\n", "We have found 3 genomic regions of interest from a previous analysis in our lab. The file qtl.bed contains informations about theses regions (chromosome, start, stop) / bed format " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "cat qtl.bed" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### Recover sequences of interest from a gff (capture seq)\n", "\n", "We want to extract sequences for each genomic region using `bedtools getfasta`or `fastaFromBed`\n", "\n", "`fastFromBed -fi sequences.fasta -bed bed file`\n", "\n", "Link : https://bioweb.pasteur.fr/docs/modules/bedtools/2.17.0/content/fastafromBed.html" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND with -fi and -bed options" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND with -fi and -bed and -fo options" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### Compare two files \n", "\n", "We want to extract annotation information (all_gene.gff3) for each genomic region (qtl.bed) using `bedtools intersect`\n", "\n", "\n", "Link : https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "PUT YOUR COMMAND " ] } ], "metadata": { "celltoolbar": "Format de la Cellule Texte Brut", "kernelspec": { "display_name": "Bash", "language": "bash", "name": "bash" }, "language_info": { "codemirror_mode": "shell", "file_extension": ".sh", "mimetype": "text/x-sh", "name": "bash" }, "rise": { "enable_chalkboard": true } }, "nbformat": 4, "nbformat_minor": 4 }