29905: Introduction to Python for Data Analysis and Automation in Biology

Technical University of Denmark

March 11th-14th, 2025

09:00-16:00

Instructors: Kai Blin, Marjan Mansourvar

Helpers: Niels Jakob Larsen, Christian Kronborg Cantalapiedra

General Information

Software Carpentry aims to help researchers get their work done in less time and with less pain by teaching them basic research computing skills. This hands-on workshop will cover basic concepts and tools, including program design, version control, data management, and task automation. Participants will be encouraged to help one another and to apply what they have learned to their own research problems.

For more information on what we teach and why, please see our paper "Best Practices for Scientific Computing".

Who: The course is aimed at graduate students and other researchers. You don't need to have any previous knowledge of the tools that will be presented at the workshop.

Where: DTU Lyngby Campus, Building 101, room S01. Get directions with OpenStreetMap or Google Maps.

When: March 11th-14th, 2025. Add to your Google Calendar.

Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating system (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed below).

Accessibility: We are committed to making this workshop accessible to everybody. The workshop organizers have checked that:

Materials will be provided in advance of the workshop and large-print handouts are available if needed by notifying the organizers in advance. If we can help making learning easier for you (e.g. sign-language interpreters, lactation facilities) please get in touch (using contact details below) and we will attempt to provide them.

Contact: Please email kblin@biosustain.dtu.dk for more information.

Roles: To learn more about the roles at the workshop (who will be doing what), refer to our Workshop FAQ.


Code of Conduct

Everyone who participates in Carpentries activities is required to conform to the Code of Conduct. This document also outlines how to report an incident if needed.


Schedule

Day 1

09:00 Automating Tasks with the Unix Shell
10:20 Morning break
10:35 Automating Tasks with the Unix Shell (Continued)
12:30 Lunch break
13:30 Version Control with Git
14:30 Afternoon break
14:45 Version Control with Git (Continued)
15:45 Wrap-up
16:00 END

Day 2

09:00 Introduction to Python
10:30 Morning break
10:45 Introduction to Python (Continued)
12:00 Lunch break
13:00 Introduction to Python (Continued)
14:30 Afternoon break
14:45 Introduction to Python (Continued)
15:45 Wrap-up
16:00 END

Day 3

09:00 Introduction to Python day 2
10:30 Morning break
10:45 Introduction to Python day 2 (Continued)
12:00 Lunch break
13:00 Introduction to Python day 2 (Continued)
14:30 Afternoon break
14:45 Introduction to Python day 2 (Continued)
15:45 Wrap-up
16:00 END

Day 4

09:00 Introduction to Pandas
10:30 Morning break
10:45 Visualizations with Altair
12:00 Lunch break
13:00 Introduction to Machine Learning with Scikit Learn
14:30 Afternoon break
14:45 Introduction to Machine Learning with Scikit Learn (Continued)
15:30 Wrap-up and Outlook
16:00 END

Setup

To participate in a Software Carpentry workshop, you will need access to software as described below. In addition, you will need an up-to-date web browser.

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.

The Bash Shell

Bash is a commonly-used shell that gives you the power to do tasks more quickly.

The default shell in some versions of macOS is Bash, and Bash is available in all versions, so no need to install anything. You access Bash from the Terminal (found in /Applications/Utilities). See the Git installation video tutorial for an example on how to open the Terminal. You may want to keep Terminal in your dock for this workshop.

To see if your default shell is Bash type echo $SHELL in Terminal and press the Return key. If the message printed does not end with '/bash' then your default is something else and you can run Bash by typing bash

If you want to change your default shell, see this Apple Support article and follow the instructions on "How to change your default shell".

Video Tutorial

The default shell is usually Bash and there is usually no need to install anything.

To see if your default shell is Bash type echo $SHELL in a terminal and press the Enter key. If the message printed does not end with '/bash' then your default is something else and you can run Bash by typing bash.

Text Editor

When you're writing code, it's nice to have a text editor that is optimized for writing code, with features like automatic color-coding of key words. The default text editor on macOS and Linux is usually set to Vim, which is not famous for being intuitive. If you accidentally find yourself stuck in it, hit the Esc key, followed by :+Q+! (colon, lower-case 'q', exclamation mark), then hitting Return to return to the shell.

In this course, we'll be using nano, which comes pre-installed and Visual Studio Code, so be sure to install that.

Python

Python is a popular language for research computing, and great for general-purpose programming as well. Installing all of its research packages individually can be a bit difficult, so we recommend miniforge.

Why do we use miniforge for this course?

Regardless of how you choose to install it, please make sure you install Python version 3.x (e.g., 3.11 or 3.12 is fine).

  1. In the Terminal, type
    cd ~
    and press Enter (or Return depending on your keyboard).
  2. Type
    wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
    and press Enter If you get an error message like
    bash: wget: command not found
    , try
    curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
    and press Enter.
  3. Type
    bash Miniforge3-$(uname)-$(uname -m).sh
    and then press Enter. You will follow the text-only prompts. To move through the text, press Spacebar. Type yes and press enter to approve the license. Press Enter (or Return) to approve the default location for the files. Type yes and press Enter (or Return) to prepend conda to your PATH (this makes the conda distribution the default Python).
  4. Close the terminal window and re-start the Terminal app.
  5. Type
    conda config --set auto_activate_base false
    and press Enter to stop conda from auto-activating.
  6. Close the terminal window and re-start the Terminal app one more time.
  7. Type the following commands, hit Enter after each line:
    conda config --remove channels defaults
    conda config --add channels bioconda
    conda config --add channels conda-forge
    conda config --set channel_priority strict
    conda activate base
    conda update --yes conda
    conda update --yes --all
      
    Don't worry if the first command results in an error message, we just want to make sure the "defaults" channel is not configured.
  8. Lastly, to install Jupyter Lab and all dependencies, type the following commands, hit Enter after each line:
    conda create --yes -n jupyter python=3
    conda activate jupyter
    conda install --yes numpy pandas scikit-learn matplotlib jupyter jupyterlab altair pip git altair odfpy openpyxl seaborn scikit-bio scipy umap-learn wget unzip
        
  9. To check your install, type
    wget https://dl.kblin.org/29905/check_install.py
    in your Terminal and hit Enter to download the check script.
  10. Then, run
    python3 check_install.py
    . If it prints
    All dependencies installed!
    you're good to go.
Also see our YouTube tutorials for installing miniconda and installing jupyter.
  1. In the Terminal, type
    cd ~
    and press Enter (or Return depending on your keyboard).
  2. Type
    wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
    and press Enter If you get an error message like
    bash: wget: command not found
    , try
    curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
    and press Enter.
  3. Type
    bash Miniforge3-$(uname)-$(uname -m).sh
    and then press Enter. You will follow the text-only prompts. To move through the text, press Spacebar. Type yes and press enter to approve the license. Press Enter (or Return) to approve the default location for the files. Type yes and press Enter (or Return) to prepend conda to your PATH (this makes the conda distribution the default Python).
  4. Close the terminal window and re-start the Terminal app.
  5. Type
    conda config --set auto_activate_base false
    and press Enter to stop conda from auto-activating.
  6. Close the terminal window and re-start the Terminal app one more time.
  7. Type the following commands, hit Enter after each line:
    conda config --remove channels defaults
    conda config --add channels bioconda
    conda config --add channels conda-forge
    conda config --set channel_priority strict
    conda activate base
    conda update --yes conda
    conda update --yes --all
      
    Don't worry if the first command results in an error message, we just want to make sure the "defaults" channel is not configured.
  8. Lastly, to install Jupyter Lab and all dependencies, type the following commands, hit Enter after each line:
    conda create --yes -n jupyter python=3
    conda activate jupyter
    conda install --yes numpy pandas scikit-learn matplotlib jupyter jupyterlab altair pip git altair odfpy openpyxl seaborn scikit-bio scipy umap-learn wget unzip
        
  9. To check your install, type
    wget https://dl.kblin.org/29905/check_install.py
    in your Terminal and hit Enter to download the check script.
  10. Then, run
    python3 check_install.py
    . If it prints
    All dependencies installed!
    you're good to go.
  1. In the Terminal, type
    cd ~
    and press Enter (or Return depending on your keyboard).
  2. Type
    wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
    and press Enter If you get an error message like
    bash: wget: command not found
    , try
    curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
    and press Enter.
  3. Type
    bash Miniforge3-$(uname)-$(uname -m).sh
    and then press Enter. You will follow the text-only prompts. To move through the text, press Spacebar. Type yes and press enter to approve the license. Press Enter (or Return) to approve the default location for the files. Type yes and press Enter (or Return) to prepend conda to your PATH (this makes the conda distribution the default Python).
  4. Close the terminal window and re-start the Terminal app.
  5. Type
    conda config --set auto_activate_base false
    and press Enter to stop conda from auto-activating.
  6. Close the terminal window and re-start the Terminal app one more time.
  7. Type the following commands, hit Enter after each line:
    conda config --remove channels defaults
    conda config --add channels bioconda
    conda config --add channels conda-forge
    conda config --set channel_priority strict
    conda activate base
    conda update --yes conda
    conda update --yes --all
      
    Don't worry if the first command results in an error message, we just want to make sure the "defaults" channel is not configured.
  8. Lastly, to install Jupyter Lab and all dependencies, type the following commands, hit Enter after each line:
    conda create --yes -n jupyter python=3
    conda activate jupyter
    conda install --yes numpy pandas scikit-learn matplotlib jupyter jupyterlab altair pip git altair odfpy openpyxl seaborn scikit-bio scipy umap-learn wget unzip
        
  9. To check your install, type
    wget https://dl.kblin.org/29905/check_install.py
    in your Terminal and hit Enter to download the check script.
  10. Then, run
    python3 check_install.py
    . If it prints
    All dependencies installed!
    you're good to go.