This article describes a hands-on introduction to the first steps + in deep learning, intended for researchers who are familiar with + (non-deep) machine learning.
+The use of deep learning has seen a sharp increase in popularity + and applicability over the last decade. While deep learning can be a + useful tool for researchers from a wide range of domains, taking the + first steps in the world of deep learning can be somewhat + intimidating. This introduction aims to cover the fundamentals of deep + learning in a practical and hands-on manner. By the end of the course, + students will be able to train their first neural network and + understand the subsequent steps needed to improve the model.
+The lesson starts by explaining the basic concepts of neural + networks, and then guides learners through the different steps of a + deep learning workflow. + After following this lesson, learners will be able to prepare data for + deep learning, implement a basic deep learning model in Python with + Keras, and monitor and troubleshoot the training process. In addition, + they will be able to implement and understand different layer types, + such as convolutional layers and dropout layers, and apply transfer + learning.
+We use data with permissive licenses and designed for real world + use cases:
+-
+
The Penguin dataset (Horst et al.
+ (
The Weather prediction dataset (Huber et al.
+ (
The Dollar Street Dataset (Gaviria Rojas et al.
+ (
This lesson addresses the need for an introductory lesson on deep + learning that is open-source, and can be used by instructors in a + workshop as well as for self-study. While generally usable, its target + audience are academic researchers.
+There are many free online course materials on deep learning, see
+ for example:
Nonetheless, these resources are often not available open-source + and can thus not be easily adapted to the students’ needs. Also, these + resources are intended to use for self-study. Our material can be used + for self-study, but it is primarily developed for instructors to use + in a workshop. In addition, although a diverse range of online courses + already exists, few are targeted towards academic researchers.
+There is another Carpentries lesson on deep learning: Introduction
+ to artificial neural networks in Python (Pollard et al.
+ (
Many computing centers offer (local) deep learning courses, such as
+ “CSC- Practical Deep Learning”
+ (
The pedagogical approach of this lesson is both to make learners
+ familiar with the key concepts, and let them practice with how to
+ implement them – eventually resulting in an increase in confidence and
+ the conviction that ‘I can do this myself’. The key to getting there
+ is live coding: before the course, learners have to setup a working
+ environment on their own computer. During the course, learners type in
+ the commands that are explained by the instructor on their own
+ computer. This design is based on the Software Carpentry
+ (
Researchers can often only free a limited amount of time (maximum 5 + consecutive days), since they are so involved in their daily work. To + accomplish this, we created a lesson that can be taught in 2 + consecutive days or 4 half days.
+Demand for our workshops and feedback gathered from students + demonstrated the need for a low-threshold lesson that lets researchers + take the first steps in the field of deep learning. This impression + was validated by other instructors who taught the lesson independently + to their own audiences and provided us with feedback on their + experience.
+In 2018, the Netherlands eScience Center initiated the development + of this lesson to fill the gap identified above. Over the years, the + lesson has attracted a broad community of individuals and + organizations that have used the material for teaching workshops, and + contributed to the improvement of the lesson significantly.
+The diversity of the involved parties has facilitated the + integration of various viewpoints on the lesson material. Apart from + the feedback gathered from students while teaching the workshop (see + below), the mix of contributors includes educators, data scientists, + and, most prominently, (research) software engineers. Some of them + have had years of experience in the deep learning domain, while others + have used the lesson as a first step into the field.
+Development sprints of typically two full working days have + regularly facilitated focussed collaboration sessions that have + brought together various contributors to tackle specific issues + identified in the lesson material. These sessions have also provided a + fruitful ground for discussing the various experiences with and + insights about the material. They have facilitated the iterative + improvement of the material, resulting in a mature and well-tested set + of episodes.
+This lesson material was designed using the concepts from The
+ Carpentries Curriculum Development Handbook
+ (
Live coding is central to this approach: the lesson is built up of
+ small blocks. In each block first the instructor demonstrates how to
+ do something, and students follow along on their own computer. Then,
+ the students work independently on exercises individually or in groups
+ to test their skills. This approach integrates opportunities for
+ guided practice throughout the lesson, promoting learning by helping
+ learners build up a functioning mental model of the domain and
+ transfer new knowledge from working memory to long-term memory. This
+ is in accordance with research-based successful teaching strategies
+ (
The lesson material is built in the new lesson template:
+ Carpentries Workbench
+ (
The lesson is split into a general introduction, and 4 episodes + that cover 3 distinct increasingly more complex deep learning + problems. Each of the deep learning problems is approached using the + same 10-step deep learning workflow + (https://carpentries-lab.github.io/deep-learning-intro/1-introduction.html#deep-learning-workflow).
+By going through the deep learning cycle three times with different
+ problems, learners become increasingly confident in applying this deep
+ learning workflow to their own projects. We end with an outlook
+ episode. Firstly, the outlook eposide discusses a real-world
+ application of deep learning in chemistry
+ (
This course was taught 13 times over the course of 4 years, both + online and in-person, by the Netherlands eScience Center (Netherlands, + https://www.esciencecenter.nl/) and Helmholtz-Zentrum + Dresden-Rossendorf (Germany, https://www.hzdr.de/). Apart from the + core group of contributors, the workshop was also taught at at least 3 + independent institutes, namely: University of Wisconson-Madison (US, + https://www.wisc.edu/), University of Auckland (New Zealand, + https://www.auckland.ac.nz/), and EMBL Heidelberg (Germany, + https://www.embl.org/sites/heidelberg/).
+An up-to-date list of workshops that the authors are aware of
+ having using this lesson can be found in a
+
In general, adoption of the lesson material by the instructors not + involved in the project went well. The feedback gathered from our own + and others’ teachings was used to polish the lesson further.
+The feedback we gathered from students is in general very + positive, with some responses from students to the question ‘What + was your favourite or most useful part of the workshop. Why?’ + further confirming our statement of need:
+Below are two tables summarizing results from our post-workshop + survey. We use the students’ feedback to continuously improve the + lesson.
+| + | STRONGLY DISAGREE | +DISAGREE | +UNDECIDED | +AGREE | +STRONGLY AGREE | +TOTAL | +WEIGHTED AVERAGE | +
|---|---|---|---|---|---|---|---|
| I can immediately apply what I learned at this + workshop. | +0 | +5 | +6 | +19 | +8 | +38 | +3,8 | +
| The setup and installation instructions for the lesson + were complete and easy to follow. | +0 | +0 | +4 | +13 | +21 | +38 | +4,4 | +
| Examples and tasks in the lesson were relevant and + authentic | +0 | +0 | +5 | +19 | +14 | +38 | +4,2 | +
Table 1: Agreement on statements by students from 2 workshops + taught at the Netherlands eScience Center. The results from these 2 + workshops are a good representation of the general feedback we get + when teaching this workshop.
+| + | POOR | +FAIR | +GOOD | +VERY GOOD | +EXCELLENT | +N/A | +TOTAL | +WEIGHTED AVERAGE | +
|---|---|---|---|---|---|---|---|---|
| Introduction into Deep Learning | +0 (0%) | +2 (5%) | +10 (27%) | +8 (22%) | +17 (46%) | +0 (0%) | +37 | +4,1 | +
| Classification by a Neural Network using Keras (penguins + dataset) | +0 (0%) | +1 (3%) | +5 (13%) | +16 (42%) | +16 (42%) | +0 (0%) | +38 | +4,2 | +
| Monitoring and Troubleshooting the learning process + (weather dataset) | +0 (0%) | +0 (0%) | +4 (11%) | +18 (47%) | +16 (42%) | +0 (0%) | +38 | +4,3 | +
| Advanced layer types (CIFAR-10 dataset) | +0 (0%) | +2 (5%) | +5 (13%) | +7 (18%) | +16 (42%) | +8 (21%) | +38 | +4,2 | +
Table 2: Quality of the different episodes of the workshop as + rated by students from 2 workshops taught at the Netherlands + eScience Center. The results from these 2 workshops are a good + representation of the general feedback we get when teaching this + workshop.
+Prior to submitting this paper the lesson went through the + substantial review in the process of becoming an official + Carpentries Lab (https://carpentries-lab.org/) lesson. This led to a + number of improvements to the lesson. In general the accessibility + and user-friendliness improved, for example by updating alt-texts + and using more beginner-friendly and clearer wording. Additionally, + the instructor notes were improved and many missing explanations of + important deep learning concepts were added to the lesson.
+Most importantly, the reviewers pointed out that the CIFAR-10
+ (
You can find all details of the review process on GitHub: + https://github.com/carpentries-lab/reviews/issues/25.
+This lesson can be taught as a stand-alone workshop to students
+ already familiar with machine learning and Python. It can also be
+ taught in a broader curriculum after an introduction to Python
+ programming (for example: Azalee Bostroem et al.
+ (
We would like to thank all instructors and helpers that taught the + course, and the community of people that left contributions to the + project, no matter how big or small. Also, we thank Chris Endemann + (University of Wisconson-Madison, US, https://www.wisc.edu/), Nidhi + Gowdra (University of Auckland, New Zealand, + https://www.auckland.ac.nz/), Renato Alves and Lisanna Paladin (EMBL + Heidelberg, Germany, https://www.embl.org/sites/heidelberg/), that + piloted this workshop at their institutes. We thank the Carpentries + for providing such a great framework for developing this lesson + material. We thank Sarah Brown, Johanna Bayer, and Mike Laverick for + giving us excellent feedback on the lesson during the Carpentries Lab + review process. We thank all students enrolled in the workshops that + were taught using this lesson material for providing us with + feedback.
+