Launch: NLP with TensorFlow (2nd Edition)

🔈The 2nd Edition of my book “NLP with TensorFlow” will be release in August, 2022 😍. I have worked for months on this new edition, employing many different amazing features found in TensorFlow to build modern ML solutions to solve NLP tasks 📊.

I wrote the first edition as a PhD student and it was an amazing experience. I was surprised to find out how well received the first edition of my book was. Currently, with close to 5 years experience in the industry, this book shows you how to process text data efficiently and build production quality ML models.

All code exercises of this book are free (and always will be).

Why NLP is important?

Natural language processing (NLP) has been in the spotlight of machine learning for many decades. Unarguably, text or language has been lifeblood of human communication.

Internet is brimming with textual data. From search queries to blogs to image captions to product reviews, there’s copious amounts of text data out there. Any software product that uses search functionality (which is a lot!) will rely heavily on NLP to understand user queries. In voice assistants like Google assistant, Alexa or Siri, once speech is converted to text, NLP is doing so much work under the hood. These tasks range from simple preprocessing techniques like lowercase or lemmatize to advanced techniques like named entity recognition and machine translation.

What does this book cover?

At a glance, the objective of this book is to make readers comfortable and confident in using TensorFlow 2 to implement cutting edge deep learning based NLP architectures complemented by efficient text processing pipelines. You will be learning to,

• Principally choose different model building APIs offered by latest TensorFlow to implement NLP models with ease
• Implement a variety of deep learning architectures including CNNs, RNNs and Transformers
• Use the tf.data API to implement simple and complex end to end text processing pipelines
• Pay attention to caveats and things to look out for when solving NLP problems (e.g. handling class imbalance)

Why you might be interested in this book?

Here I will explain some unique points that separates this book form other books.

• Pictures worth a thousand words! Though we are learning NLP, we cannot simply ignore visual diagrams that are rich in information. I try to frequently incorporate diagrams to explain complex technical topics such as data processing techniques and model architectures
• Building machine learning models is not a single step. It is a journey involving many carefully thought steps. When building models, not only you have to get the model right, but also be conscious of the nature of the data; has it been cleaned? is it a balanced dataset etc? I will naturally lead into such questions and clarify them as we solve problems.
• TensorFlow is a large and complex framework. This means there are multiple ways to achieve the same thing. For example, you will see how I use different Keras APIs depending on the complexity of the model or different methods to tokenize text (i.e. split continuous text to tokens and convert to numerical IDs) such as, Keras’s Tokenizer or the TextVectorization layer

What cool new features would I learn?

There’s lots of free resources out there. For example, the official TensorFlow documentation is a great place to start. What additional things would I learn that these resources cannot teach? I already know how to create models with TensorFlow, what new features would I learn from this book? These might be some questions lingering in some of your curious minds. Though I can’t list them all, here’s a few key points.

• TensorFlow is a complex tool and an ecosystem. It’s easy to feel overwhelmed and lost! This book will guide you through most important concepts and code to get up to speed with TensorFlow quickly.
• With TensorFlow you can create complex architectures quite easily. For example, you’ll see examples where we have token embeddings generated through an embedding table and through a CNN and character embeddings in the same model. In the last chapter, you’ll see how you can connect a Vision Transformer and a Text-based Transformer to solve problems.
• TensorFlow’s data API is quite versatile. You will see how you can leverage many different capabilities offered this API to manipulate text data to suite a range of tasks such as language generation, machine translation, etc.
• You will see how other important libraries such as transformers (from Hugging Face) and imbalaned-learn are used for our advantage.

FAQs

1. TensorFlow vs Pytorch: Is TensorFlow the right technology for me?

The short answer is that it’s not a race anymore. There’s a 50–50 split between the two technologies; TensorFlow and Pytorch. Both TensorFlow and Pytorch have matured to become well supported ecosystems for ML model development and productionization.

The best advice I can give is to master one technology. Once you do that, it’s not very difficult to translate the knowledge to the other. Under the implementation details, they share the same collection of concepts. So, kick off your journey with TensorFlow!

2. What does the future of NLP hold?

It is very difficult to anticipate what the future of NLP would look like. But there are some obvious trends you can see transpiring.

Bigger and better Transformers

Since their inception in 2017, Transformer models have been getting bigger and better. For example, GPT, GPT-2 and GPT-3 showed a clear trend in increasing parameter sizes. Most recently, Hugging Face open-sourced Bloom; a mammoth NLP model trained on data from 46 languages and 350B unique tokens. They also explained the engineering behind such models which is usually inaccessible and not disclosed by the companies behind these efforts.

Multi-modal/Multi-task/Zero-shot learning

Multi-modal and multi-task learning are hot topic in the field. Multi-modal models focus on combining different modalities of data (image, text, speech) and learning using a single model (e.g. data2vec). Multi-task learning is where you focus on learning multiple tasks (e.g. named entity recognition, machine translation, etc.) using the same model. BERT is already a step in this direction as you can fit different classification heads and adapt them for different tasks like sentiment analysis or question answering. Another related topic is zero-shot learning. The idea is to formulate different NLP tasks with a uniform format and use a single model to generate outputs.

Responsible AI

Modern ML models become more and more data-hungry due to their increasing size. This means that, for companies working with user data (e.g. PII) needs to be responsible and ethical with how they use this data for machine learning. How can we learn from user data without infiltrating user privacy and protect the data? Areas such as responsible AI, privacy preserving AI are focused on this matter.

Conclusion

I hope this book take you on an exciting journey, by the end of which you’ll be proficient in TensorFlow and can confront ML problems confidently. And of course, share your thoughts and feedback.

• LinkedIn: https://www.linkedin.com/in/thushanganegedara/
• Twitter: https://twitter.com/thush89