Achieving the TensorFlow Developer Certification

As part of my transition from traditional software engineering projects to those with a strong AI/ML flavour, I recently spent time digging deeper into TensorFlow particularly how to customise TensorFlow, create custom training loops and distributed model training. While doing this, I remembered I’d once planned to take the TensorFlow Developer Certification – this was a few years ago, and when COVID-19 hit, I’d put it aside (and forgot about it). So, while I had some free time, I thought I’d prepare for and take the exam – which I passed scoring full marks!

My Exam Experience

It's not to say that I found the exam easy – in fact, many posts and I’d read about others taking the exam said it took them 2-3 hours to complete (out of the 5 hours). For me, I took a little over 4hrs, which I was surprised at – especially considering I had completed 3 out of the 5 questions in little over 40 minutes.

It's worth noting that this is not wholly unexpected since the questions are graduated in difficulty with question 1 being considerably easier (and its score carries less weight) than question 5. I think, thanks to how i prepared for the exam (see below), I was able to quickly provide working and high scoring models for these questions.

So what did I do with the other 3 hours and 20 minutes…well, I spent it on the last two questions.

Question 4

For Question 4, I was able to quickly build a working model and my first submission scored 4/5. Given how much time I had left for the last 2 questions I was determined to get 5/5. So spent a long time training, tunning and training the model multiple times in my quest to get 5/5.

In the end, I got frustrated especially when my last attempt dropped my score to 3/5 so I decided to accept the original 4/5 score and re-submitted my first model submission. Then something unexpected happened: the model scored 5/5 when it was re-submitted.

Reflecting on this, I guess the submitted models are assessed on random subsets of data and my re-submitted model was tested with a subset that resulted in a slightly higher accuracy and just tipped me into a 5/5 score. While pleased with the score I was frustrated that I’d spent so much time chasing that extra score when all I needed to do was re-submit.

Question 5

This question completely threw me – I knew as soon as I read the question that I wasn’t sure how to solve the problem. I understood the question and what was expected but didn't know how to build the model as i'd not come across this particular flavour of the problem.

My initial attempt at a solution scored me 2/5, and I was never going to accept that and still had over 2hrs left on the clock so I decided to try a more complex architecture to improve the model.

However, as soon as I tried a more complex architecture I started getting errors - errors that I'd not encountered before and that didn't make a lot of sense given my basic model trained without issue.

The exam allows you to use the internet, books and any other resources that you’d normally use in your work…except asking someone else to help. A search for the error didn’t help much with a solution, so I spent time looking for examples that were close to what the question was asking. Again, not much helped.

I knew what the problem was but wasn’t sure how to solve it, so I went to the Keras documentation, convinced there must be a layer that does what I needed - sure enough, there was.

It wasn’t a layer that I’d used before or seen mentioned in the learning resources I’d used.?However, as is the case with Keras layers, easy to use so I added and configured the layer. With this simple change, the model trained without error and I submitted the new model and achieved a score of 5/5.

Reflecting on this after the exam, I could see why the last question had given me such problems – I’d not considered this specific type of problem in my preparation. If I had, I would have recognised the type of problem and how to solve this type scenario; the question would have been reasonably straightforward if this had been the case.

Overall, it was an interesting venture into Vendor certification that didn’t detract too much from the other studies I was doing.

Preparing for the Certification

Given that I was already familiar with TensorFlow, the preparation for the exam wasn’t too onerous, and I already had access to suitable learning materials tailored to the exam. For me, I rewatched the TensorFlow Developer Professional Certificate course on Coursera taught by Laurence Moroney (I’d previously completed the course a few years ago). I spent most of my time practising building the models I’d expect in the exam.

Overall, it took me less than two weeks to prepare and sit the exam. If you are relatively new to Deep Learning and TensorFlow, it will likely take a few months to acquire the knowledge and practice the skills necessary to pass the exam.

You can review the Candidate Handbook (https://www.tensorflow.org/certificate) and then use the guides and tutorials for TensorFlow (https://www.tensorflow.org/overview) and Keras (https://keras.io/getting_started/) to prepare for the exam.

The challenge is the amount of documentation and tutorials available on these sites; much of which will not be covered in the TensorFlow Certification exam. So, finding the most relevant guides and tutorials can be challenging and overwhelming.

So, it’s worth considering using a learning resource tailored to the certification to help you prepare, this is especially true if you are fairly new to TensorFlow and Keras.

Specifically, I would recommend using one of the following learning resources:

• DeepLearning.AI TensorFlow Developer Professional Certificate taught by Laurence Moroney and available on Coursera
• AI And Machine Learning for Coders by Laurence Moroney
• TensorFlow Developer Certificate Guide by Oluwole Fagbohun
• Zero to Mastery TensorFlow for Deep Learning

Each of these covers the same material and is aligned with the TensorFlow Developer Certification, so pick one and go with it.

The amount of material you need for the certification exam is not huge as the exam doesn’t require anything beyond building Sequential models using existing Keras layers. So don’t feel that you need to master the Functional API, Custom Training Loops and Layers or distributed training to pass the exam…you will of course, want to master these at some point to round out your TensorFlow knowledge ??

A key part of my approach to this certification was to practice the types of problems I expected on the exam – the exam itself is highly practical - you won’t be asked about terms or concepts. Still, you will be expected to know them and apply them practically to solve a problem.

Each question comes with varying degrees of starter code (usually just enough to download the data you will use) so that you can focus on the following core skills:

• Pre-processing data from different sources
• Building and training models
• Evaluating model performance for problems such as overfitting and underfitting and adjusting the model and tuning parameters to deal with them.

From the Candidate Handbook, you can see that you will be tested on problems related to the following:

·???????? Regression

·???????? Image Classification

·???????? Natural Language Processing (Text Classification and Text Generation)

·???????? Time Series

So, after reviewing the learning material, my strategy was to practice solving these types of problems using standard public datasets and not much more than Core TensorFlow and Keras Sequential models.

Specifically, I practised solving the following types of problems

• Univariate and multivariate Regression
• Binary and Multi-Class Image Classification
• Binary and Multi-Class Text Classification
• Text Generation
• Single Step Time Series Predictions (both univariate and multivariate inputs)
• Multi-Step Time Series predictions (both univariate and multivariate inputs)
• Single and Multi-Step Time Series Predictions with multi-variate outputs

One of these was missing from my original practice list and it was the one that came up in my exam and caused me so much stress – I won’t tell you which one.

For each problem type, I formulated an idea of what an archetype model architecture would be and practised building that to solve problems using different datasets. For each of these, I would practice tuning the model to get the best performance from the model for the dataset and deal with common issues such as underfitting and overfitting. I also practised using transfer learning and pre-trained layers in my models, where I couldn’t achieve a high enough accuracy.

So, what do I mean by an Archetype? Well, imagine you are asked to solve a Binary Image Classifier problem for a dataset (such as the Cat or Dog dataset); a reasonable starting point might be to define a CNN model architecture that consists of 2-3 Convolution blocks and a set of binary classifier layers. Add to this a suitable function such as NAdam and a Categorical Cross Entropy loss function, and you have an archetype model that will get you most of the way to solving an Image Classifier problem (for this exam).? In the exam, if you start with this, train the model and submit, it will score you some points.

If you want more points, you'll need a more accurate model, so you need to look at what is causing the poor performance (is it overfitting? Is it underfitting?) and tweak the architecture and parameters to improve the performance. For an exam like this, it’s good to get some marks on the board early and then iterate with the time available to improve your score.

Of course, this approach means you may need multiple tune/train loops and you could decide to start with a more complex model (e.g. a very deep model with various regularisation and normalisation options or use transfer learning) but this approach isn't recommended for this exam for the following reasons:

• More complex models will take longer to train, so if you have to iterate on the model, it will be slower and you have more hyperparameters to consider.
• Apparently, the exam scoring takes into account the complexity of the model and you get penalised for overly complex models.

So, in general, it’s better to start with a good simple model and add complexity as needed. Based on my experience, a reasonable model will get you 80% of the marks or more.

The last advice I would give if you are thinking about achieving the certification would be to get used to saving the model per question and per training iterations. You can submit models to be scored as many times as you want (there is no limit) but only your last submitted model is used for scoring your exam performance. The scoring system doesn't keep track of your best-performing model, only your last submitted one.

So it’s worth using callbacks to save the best model for each iteration so that you can revert to an earlier better-performing model and submit that.

Good luck

?