fastai - Education

Fast.ai offers in-depth applied deep learning courses, ranging from utilizing vision models to recommender systems and NLP to simple regression problems. After analyzing the fastai repository, we will now look into the core mission of fast.ai, learning. The central question we want to answer in this essay is: What is new in the digital learning experience of fast.ai?

To answer this question, we will first give an introduction to the different learning resources of fastai. Next, we will give a theoretical overview of their learning strategies. Then, fastai is explained in the context of learning. After that, we will comment on the recent educational material. Lastly, we will address the ethical issues behind this project.

Multiple education materials

Deep learning is a popular topic. Software companies are interested in trying to learn how it works and how it can be integrated into their projects. However, as this is a relatively new field there is a steep learning curve for new developers and people that are interested in the topic. Normally, to grasp what deep learning is, one needs to have a solid understanding of calculus, linear algebra, and statistics. Also, coding experience is required. Many of the common Deep learning concepts are implemented in libraries such as the Deep Learning Toolbox for MATLAB1 and the Python libraries Tensorflow2 and PyTorch3. Using MATLAB requires a license and thus is not as accessible as the previously mentioned Python libraries for Deep Learning. Thus it is easier to find courses using these Python libraries.

In the past frameworks like Keras4 were introduced to make implementing Deep learning concepts with the underlying libraries easier. This library has extensive documentation available online and is used by Massive Online Open Courses (MOOCs). Platforms like Coursera5 and Udacity6 publish these courses for educational purposes. The MOOC material needs to be updated every time a big update of such a framework is released.

Figure: Learning tools available online

As stated in the article “Deep Learning MOOCs” 7, we can find many resources aimed at teaching deep learning. Next to these courses, fastai is proposed as an innovative approach. It offers a learning experience customized to a wide variety of learning approach types. These varieties of learning approaches come together in that they all use the same fastai framework.

Actionable knowledge approach

These different types of learning approaches are all related to learning the student actionable knowledge. This type of knowledge is practical and allows the student to quickly get a prototype working. For example to build a web application you need knowledge about a front-end library, backend framework, databases, and deployment to fully build an application yourself.

Directions of learning

When learning a new skill like coding, one can take roughly two approaches: a bottom-up or top-down approach8. Top-down in the case of programming means that one would start with the overarching concepts first and then build its way down to the many small details. The Bottom-up would be to start at the small details and work your way up.

Both directions have advantages and disadvantages. For the top-down approach, the main advantage is that one would immediately see the entire context of a system as all components can be immediately observed. However this is also immediately its downside, without sufficient external knowledge, the top-down approach can be hard to digest, especially for new students. In the bottom-up approach, you will first learn about various components and how they work in detail, this means however that the context can sometimes get lost.

Applying only one approach might not work, as was found in the USA9. So as with most things in life, a binary choice might not be the answer. Therefore it is interesting to consider that the top-down and bottom-up approach can be combined in various ways to fit your personal needs. This resulted in the learning by doing paradigm, where a practical approach is taken in order to learn a new skill.

Learning by example

An approach closely related to learning by doing is learning by example. This is a practical approach to learning a new skill, it is closely related to the top-down learning approach, although more regulated. A student is given some examples with the solution next to it, from those examples a student needs to infer the pattern. When using learning by doing, a student has to find the pattern by trying various solving strategies to a problem. Both strategies raise the question of whether they are efficient for students to learn a pattern.

Both strategies require students to have prerequisite knowledge. As shown in “Learning mathematics from example and by doing”10 students are given the mathematical problem of factorization. This requires them to know what factorization means, how powers work and what a second-degree polynomial function is, this prerequisite knowledge can be obtained by iteratively applying learn by example or doing. This creates a web of intertwined knowledge.

We will now shift the perspective back to fastai and in the next section, we will explain how example learning is applied in fastai through exploratory programming.

Exploratory approach

As we saw in the third essay, the whole library is coded using Jupyter notebooks. This is also applied to the lecture material. This is odd since Jupyter Notebooks are rarely used in deep learning as they tend to have a bad reputation 11.

Advantages

A notebook offers a different learning experience than just code. A Jupyter Notebook contains text, images, links to papers or external resources as well as the code. This means that a student that is learning a new concept, doesn’t have to juggle between different resources like a physical book, some notes, a video, the code, and its results. Everything is in one place, which we believe makes the learning experience smooth and potentially more enjoyable.

Figure: fastai Jupyter Notebook example

Also, Notebooks support Literate programming: “a methodology that combines a programming language with a documentation language, thereby making programs more robust, more portable, more easily maintained and arguably more fun to write… The main idea is to treat a program as a piece of literature, addressed to human beings rather than to a computer”12.

Next, the fastai notebooks were created in such a way that they support exploratory programming. The idea of exploratory programming is that “the very process of exploration is valuable in itself”13. A notebook can be seen as a scientific journal where each cell represents an experiment. If a student has any questions or doubt about something, it can test it and see the results in one place. This is a practical approach, where the comparison between different models or parameters is easily made. Jeremy Howards argues that it provides a more hands-on experience that is more fun 14. Learning by example is fully supported by fastai. A student can rerun the examples and create their own.

Lastly, another potential advantage of notebooks is that the student can always come back to its scientific journal and understand the reasoning and thinking process it had at the time of writing/coding it. Since the notebook can keep track of the experiments that performed well and poorly, it will remind the student of what works and more importantly, why it does work. Moreover, this allows spaced repetition, which has been proven to increase the learning rate15.

Disadvantages

Despite the advantages of notebooks, there are drawbacks. The first is the testability of the code. This happens because the code is such that it needs to be run sequentially and is saved as a JSON-file. This unusual way of coding does not work with the traditional testing tools. Additionally, because of the sequential manner of the notebooks, having run cells in incorrect order can cause bugs to occur since variables might be initialized improperly, while previous states are saved. These topics may jeopardize the teaching process. However, the fastai team members are aware of these issues and through the development of the nbdev library are trying to minimize them16.

Up to date material regarding the state of the art

The educational content must be up to date with state-of-the-art techniques. This will teach the student what models are capable of.

In this regard, the library fastai has the advantage of relying on PyTorch. The user can use any model present in PyTorch in the fastai pipeline. As the article17 mentions “fastai at its core is a training loop, designed to be framework agnostic. You can use any flavor of Pytorch you want and only use fastai to quickly and effectively train a model with state-of-the-art practices”. As the PyTorch library is itself often updated, the library of fastai is as well. Moreover, if a problem occurs in the bridging gap between Pytorch functionalities and fastai ones, the contributors of fastai tend to respond rapidly to these issues on the forum.

Concerning the course itself, it is regularly updated, as we have found out that there have been three different versions of the course that have been published over the last four years 18. Thus, on average almost every year there is an update which seems to be sufficient to keep up with used materials.

Accessibility

Developing a deep learning model from scratch is a challenging task and requires effort. Apart from effort, training can take a considerable amount of time. Initially, Deep learning knowledge was thought to be exclusively for a group of specialized mathematicians. However, the wide range of fields to which Deep learning can be applied was much bigger.

Through the accessibility that fastai provides, these fields can get access to this knowledge and apply it19 without extensive background knowledge. This way fastai could help develop a range of “role model” applications, in areas where deep learning is currently being poorly utilized.

Educational alternatives

While fastai offers an integrated approach of deep learning material, it is not unique. There are multiple specialized alternatives for the book, the courses and the libraries. The book “Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems” by Aurélien Géron 20 is quite popular in the AI community and has a similar rating and almost 7 times more reviews in Amazon than the fastai alternative21. On the side of the MOOC, the most popular course available of deep learning is “Deep Learning Specialization” by Andrew Ng22 with a rating of 4.9 stars given by around 111.900 users. Nevertheless, this widely popular course may be already be somewhat outdated. When referring to the libraries, the most popular alternatives are PyTorch ligthning23 and PyTorch Ignite24 which focus mainly on the ease of use but not on the learning process itself as fastai does.

Takeaways

fastai compared to other educational tools offers an integrated environment. To the best of our knowledge, it seems that it is the only project providing a library, a MOOC, a book, and a forum. The actionable knowledge approach supported by notebooks offers the student an exploratory programming experience. Their creative approach allows the student with only a bit of math and programming experience to utilize deep learning to its fullest potential, making this field more accessible.

  1. Mathworks (n.d). Deep Learning in Matlab. Retrieved March 29, 2021, from https://nl.mathworks.com/help/deeplearning/ug/deep-learning-in-matlab.html ↩︎

  2. TensorFlow (n.d.). Retrieved March 29, 2021, from https://www.tensorflow.org/ ↩︎

  3. PyTorch (n.d.). Retrieved March 29, 2021, from https://pytorch.org/ ↩︎

  4. Keras (n.d.). Retrieved March 29, 2021, from https://keras.io/ ↩︎

  5. Coursera (n.d.). Retrieved March 29, 2021, from https://www.coursera.org/ ↩︎

  6. Udacity (n.d.). Retrieved March 29, 2021, from https://www.udacity.com/ ↩︎

  7. Deep Learning MOOCs (September 12, 2018). Retrieved March 29, 2021, from https://margaretmz.medium.com/deep-learning-moocs-1be70cf9737f ↩︎

  8. Learning to code (or do anything) (December 13, 2018). Retrieved March 29, 2021, from https://medium.datadriveninvestor.com/learn-to-code-or-do-anything-top-down-bottom-up-and-other-approaches-bd2fd8eb89dd ↩︎

  9. Coordinating Top-Down and Bottom-Up Strategies for Educational Reform (n.d.). By Michael G. Fullan. Retrieved March 29, 2021, from https://www2.ed.gov/pubs/EdReformStudies/SysReforms/fullan1.html ↩︎

  10. Xinming Zhu & Herbert A. Simon. “Learning Mathematics From Examples and by Doing”. Cognition and Instruction, vol. 4, no. 3, 1987, p. 137-166. tandfonline.com, doi: 10.1207/s1532690xci0403_1. ↩︎

  11. Youtube O’Reilly (October 11, 2018). I don’t like notebooks. Retrieved March 29, 2021, from https://www.youtube.com/watch?v=7jiPeIFXb6U ↩︎

  12. Literate Programming (March 25, 1992). Retrieved March 29, 2021, from https://web.stanford.edu/group/cslipublications/cslipublications/site/0937073806.shtml ↩︎

  13. fast.ai (December 2, 2019). nbdev: use Jupyter Notebooks for everything. Retrieved March 29, 2021, from https://www.fast.ai/2019/12/02/nbdev/ ↩︎

  14. Youtube Jeremy Howard (October 29, 2020). I like Notebooks. Retrieved March 29, 2021, from https://youtu.be/9Q6sLbz37gk?t=42 ↩︎

  15. Paul Smolen, Yili Zhang, John H. Byrne. “The right time to learn: mechanisms and optimization of spaced learning”. Nature Reviews Neuroscience, vol 17, 2016, p. 77-88. Retrieved March 29, 2021, from https://arxiv.org/abs/1606.08370 ↩︎

  16. Youtube Jeremy Howard (October 29, 2020). I like Notebooks. Retrieved March 29, 2021, from https://www.youtube.com/watch?v=9Q6sLbz37gk ↩︎

  17. muellerzr.gitub.io (n.d.). Pytorch to fastai, Bridging the Gap. Retrieved March 29, 2021, from https://muellerzr.github.io/fastblog/2021/02/14/Pytorchtofastai.html ↩︎

  18. fast.ai forums (August 24, 2020). FastAI Versions to Years? Retrieved March 29, 2021, from https://forums.fast.ai/t/fastai-versions-to-years/77248 ↩︎

  19. fast.ai (October 7, 2016). Launching fast.ai. Retrieved March 29, 2021, from https://www.fast.ai/2016/10/07/fastai-launch/ ↩︎

  20. Amazon (October 15, 2019). Retrieved March 29, 2021, from https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-TensorFlow-dp-1492032646/dp/1492032646 ↩︎

  21. Amazon (August 11, 2020). Retrieved March 29, 2021, from https://www.amazon.com/Deep-Learning-Coders-fastai-PyTorch/dp/1492045527 ↩︎

  22. Coursera (n.d.). Specialization in Deep Learning. Retrieved March 29, 2021, from https://www.coursera.org/specializations/deep-learning ↩︎

  23. Pytorchlightning (n.d.). Retrieved March 29, 2021, from https://www.pytorchlightning.ai/ ↩︎

  24. Pytorch ignite (n.d.). Retrieved March 29, 2021, from https://pytorch.org/ignite/ ↩︎