Free artificial intelligence and machine learning video tutorial resource: February 2021

Saturday, February 27, 2021

Finally, Instant Monsters! 🐉

❤️ Check out Weights & Biases and sign up for a free demo here: https://ift.tt/2YuG7Yf ❤️ Their mentioned post is available here: https://ift.tt/3bHQqg9 📝 The paper "Monster Mash: A Single-View Approach to Casual 3D Modeling and Animation" is available here: https://ift.tt/3hzxtwW Web demo - make sure to click "Help" and read the instructions: https://ift.tt/3rN3rfA More on Flow by Mihály Csíkszentmihályi - it is immensely important to master this! https://www.youtube.com/watch?v=8h6IMYRoCZw 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Haro, Alex Serban, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Lau, Eric Martel, Gordon Child, Haris Husic, Jace O'Brien, Javier Bustamante, Joshua Goller, Kenneth Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Robin Graham, Steef, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. If you wish to appear here or pick up other perks, click here: https://ift.tt/2icTBUb Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/twominutepapers Web: https://ift.tt/1NwkG9m #gamedev

GLOM: How to represent part-whole hierarchies in a neural network (Geoff Hinton's Paper Explained)

#glom #hinton #capsules Geoffrey Hinton describes GLOM, a Computer Vision model that combines transformers, neural fields, contrastive learning, capsule networks, denoising autoencoders and RNNs. GLOM decomposes an image into a parse tree of objects and their parts. However, unlike previous systems, the parse tree is constructed dynamically and differently for each input, without changing the underlying neural network. This is done by a multi-step consensus algorithm that runs over different levels of abstraction at each location of an image simultaneously. GLOM is just an idea for now but suggests a radically new approach to AI visual scene understanding. OUTLINE: 0:00 - Intro & Overview 3:10 - Object Recognition as Parse Trees 5:40 - Capsule Networks 8:00 - GLOM Architecture Overview 13:10 - Top-Down and Bottom-Up communication 18:30 - Emergence of Islands 22:00 - Cross-Column Attention Mechanism 27:10 - My Improvements for the Attention Mechanism 35:25 - Some Design Decisions 43:25 - Training GLOM as a Denoising Autoencoder & Contrastive Learning 52:20 - Coordinate Transformations & Representing Uncertainty 57:05 - How GLOM handles Video 1:01:10 - Conclusion & Comments Paper: https://ift.tt/3qV9tKo Abstract: This paper does not describe a working system. Instead, it presents a single idea about representation which allows advances made by several different groups to be combined into an imaginary system called GLOM. The advances include transformers, neural fields, contrastive representation learning, distillation and capsules. GLOM answers the question: How can a neural network with a fixed architecture parse an image into a part-whole hierarchy which has a different structure for each image? The idea is simply to use islands of identical vectors to represent the nodes in the parse tree. If GLOM can be made to work, it should significantly improve the interpretability of the representations produced by transformer-like systems when applied to vision or language Authors: Geoffrey Hinton Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ift.tt/3dJpBrR BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB Parler: https://ift.tt/38tQU7C LinkedIn: https://ift.tt/2Zo6XRA BiliBili: https://ift.tt/3mfyjkW If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://ift.tt/2DuKOZ3 Patreon: https://ift.tt/390ewRH Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

AI & ML Workshop DAY 2

Friday, February 26, 2021

Xây dựng bộ phân loại hình ảnh (Machine Learning: Zero to Hero, phần 4)

This video is also available in English → https://goo.gle/3aVGHni Trong phần cuối của loạt video về Machine Learning: Zero to Hero, Khánh (khanhlvg@) sẽ giải thích về cách xây dựng mô hình phân loại hình ảnh cho trò chơi oẳn tù tì. Trong phần 1, chúng ta đã đề cập đến trò chơi oẳn tù tì và việc lập trình để máy tính nhận biết hình ảnh bàn tay ra đấm, lá, kéo khó như thế nào. Tuy nhiên, sau đó chúng ta cũng đã tìm hiểu nhiều về công nghệ học máy, cách xây dựng mạng nơ ron để phát hiện các quy luật từ dữ liệu điểm ảnh, và phương pháp sử dụng mạng tích chập để phát hiện các đặc trưng trong bức ảnh. Ở phần này, chúng ta đã áp dụng những kiến thức đã học từ 3 phần trước để xây dựng mạng nơ ron để máy tính chơi oẳn tù tì. Tập dữ liệu về trò chơi oẳn tù tì → http://goo.gle/2m68kCV Xem và chạy thử code tại Colab notebook này → https://goo.gle/2m07d7W Subscribe kênh TensorFlow → https://goo.gle/TensorFlow

Giới thiệu về mạng nơ ron tích chập (Machine Learning: Zero to Hero, phần 3)

This video is also available in English → https://goo.gle/3swbpct Trong phần 3 của loạt video Machine Learning: Zero to Hero, Khánh (khanhlvg@) sẽ giới thiệu về mạng nơ ron tích chập và lý do chúng đóng vai trò quan trọng trong các ứng dụng thị giác máy tính. Tích chập là một bộ lọc xử lý hình ảnh và trích xuất các đặc điểm từ ảnh. Trong video này, bạn sẽ tìm hiểu về nguyên lý hoạt động của các mạng nơ ron tích chập qua việc xử lý và nhận diện các hình ảnh trong một tập dữ liệu về sản phẩm thời trang. Xem và chạy thử code trong codelab này → https://goo.gle/2m07asM Subscribe kênh TensorFlow → https://goo.gle/TensorFlow

Nhập môn Thị giác máy tính bằng công nghệ học máy (Machine Learning: Zero to Hero, phần 2)

This video is also available in English → https://goo.gle/3qXWK9C Trong phần 2 của loạt video Machine Learning: Zero to Hero, Khánh (khanhlvg@) sẽ giới thiệu sơ lược về Thị giác máy tính bằng học máy. Chúng ta sẽ dạy cho máy tính nhìn thấy và nhận diện các đồ vật khác nhau. Xem và chạy thử code tại Colab notebook này → https://goo.gle/34cHkDk Subscribe kênh TensorFlow → https://goo.gle/TensorFlow

Giới thiệu về công nghệ học máy (Machine Learning: Zero to Hero, phần 1)

This video is also available in English → https://goo.gle/3sJoXSj Công nghệ học máy (machine learning) là một hình thức lập trình mới. Trong đó, thay vì lập trình các chỉ thị cho máy tính bằng ngôn ngữ lập trình như Java hoặc C++, thì với học máy, bạn sẽ tạo một chương trình được huấn luyện dựa trên dữ liệu và máy tính sẽ tự suy ra các logic từ dữ liệu đó. Vậy công nghệ học máy thực sự là như thế nào? Trong phần 1 của chuỗi video Machine Learning: Zero to Hero, Lê Viết Gia Khánh (khanhlvg@), Developer Advocate tại Google, sẽ giới thiệu một ví dụ “Hello World” cơ bản về xây dựng mô hình học máy, và giới thiệu các khái niệm mà chúng ta sau đó sẽ áp dụng cho một vấn đề thú vị hơn: thị giác máy tính. Xem và chạy thử code trong codelab này → https://goo.gle/2Zp2ZF3 Subscribe kênh TensorFlow → https://goo.gle/TensorFlow

Linear Transformers Are Secretly Fast Weight Memory Systems (Machine Learning Paper Explained)

#fastweights #deeplearning #transformers Transformers are dominating Deep Learning, but their quadratic memory and compute requirements make them expensive to train and hard to use. Many papers have attempted to linearize the core module: the attention mechanism, using kernels - for example, the Performer. However, such methods are either not satisfactory or have other downsides, such as a reliance on random features. This paper establishes an intrinsic connection between linearized (kernel) attention and the much older Fast Weight Memory Systems, in part popularized by Jürgen Schmidhuber in the 90s. It shows the fundamental limitations of these algorithms and suggests new update rules and new kernels in order to fix these problems. The resulting model compares favorably to Performers on key synthetic experiments and real-world tasks. OUTLINE: 0:00 - Intro & Overview 1:40 - Fast Weight Systems 7:00 - Distributed Storage of Symbolic Values 12:30 - Autoregressive Attention Mechanisms 18:50 - Connecting Fast Weights to Attention Mechanism 22:00 - Softmax as a Kernel Method (Performer) 25:45 - Linear Attention as Fast Weights 27:50 - Capacity Limitations of Linear Attention 29:45 - Synthetic Data Experimental Setup 31:50 - Improving the Update Rule 37:30 - Deterministic Parameter-Free Projection (DPFP) Kernel 46:15 - Experimental Results 50:50 - Conclusion & Comments Paper: https://ift.tt/3kqGED3 Code: https://ift.tt/3pZp8Xw Machine Learning Street Talk on Kernels: https://youtu.be/y_RjsDHl5Y4 Abstract: We show the formal equivalence of linearised self-attention mechanisms and fast weight memories from the early '90s. From this observation we infer a memory capacity limitation of recent linearised softmax attention variants. With finite memory, a desirable behaviour of fast weight memory models is to manipulate the contents of memory and dynamically interact with it. Inspired by previous work on fast weights, we propose to replace the update rule with an alternative rule yielding such behaviour. We also propose a new kernel function to linearise attention, balancing simplicity and effectiveness. We conduct experiments on synthetic retrieval problems as well as standard machine translation and language modelling tasks which demonstrate the benefits of our methods. Authors: Imanol Schlag, Kazuki Irie, Jürgen Schmidhuber Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ift.tt/3dJpBrR BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB Parler: https://ift.tt/38tQU7C LinkedIn: https://ift.tt/2Zo6XRA BiliBili: https://ift.tt/3mfyjkW If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://ift.tt/2DuKOZ3 Patreon: https://ift.tt/390ewRH Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

Transfer Learning and Fine tuning with Tensorflow Object Detection API - Part III

#tensorflow #objectdetectionapi #objectdetection #ai #machinelearning #python #gpu #nvidia #madewithtf #transferlearning #finetuning To improve our image classification model, since we don't have too many images, we use Transfer Learning and fine-tuning. Can I teach an AI model using Tensorflow Object Detection API, how to identify the theme of a painting? The only way to find out is by trying! And I think you might be surprised by the outcome of this experiment. Article: https://spltech.co.uk/tensorflow-object-detection-api-custom-image-classification%e2%80%8a-%e2%80%8acan-i-detect-if-a-painting-is-a-nativity-painting/ FOLLOW ME Instagram: https://www.instagram.com/codemental/ Blog: https://spltech.co.uk/blog/ SUPPORT THE CHANNEL or get MENTORING http://www.patreon.com/codemental TIMESTAMPS 0:00 Explaining Transfer Learning and how it can help with improving our model 1:00 Xception architecture 1:26 Freezing neural network weights 2:43 Understanding results of training image classifier with Transfer Learning 3:50 Testing image classifier against training data set 4:34 Training the model further with fine tuning RESOURCES Tensorflow Transfer Learning Tutorial: https://www.tensorflow.org/tutorials/images/transfer_learning Article https://spltech.co.uk/tensorflow-object-detection-api-custom-image-classification%e2%80%8a-%e2%80%8acan-i-detect-if-a-painting-is-a-nativity-painting/ Github https://github.com/armindocachada/tensorflow-custom-classification-art-nativity

Thursday, February 25, 2021

DeBERTa: Decoding-enhanced BERT with Disentangled Attention (Machine Learning Paper Explained)

#deberta #bert #huggingface DeBERTa by Microsoft is the next iteration of BERT-style Self-Attention Transformer models, surpassing RoBERTa in State-of-the-art in multiple NLP tasks. DeBERTa brings two key improvements: First, they treat content and position information separately in a new form of disentangled attention mechanism. Second, they resort to relative positional encodings throughout the base of the transformer, and provide absolute positional encodings only at the very end. The resulting model is both more accurate on downstream tasks and needs less pretraining steps to reach good accuracy. Models are also available in Huggingface and on Github. OUTLINE: 0:00 - Intro & Overview 2:15 - Position Encodings in Transformer's Attention Mechanism 9:55 - Disentangling Content & Position Information in Attention 21:35 - Disentangled Query & Key construction in the Attention Formula 25:50 - Efficient Relative Position Encodings 28:40 - Enhanced Mask Decoder using Absolute Position Encodings 35:30 - My Criticism of EMD 38:05 - Experimental Results 40:30 - Scaling up to 1.5 Billion Parameters 44:20 - Conclusion & Comments Paper: https://ift.tt/30KC7ni Code: https://ift.tt/3f5JvNW Huggingface models: https://ift.tt/3stcFxe Abstract: Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks. In this paper we propose a new model architecture DeBERTa (Decoding-enhanced BERT with disentangled attention) that improves the BERT and RoBERTa models using two novel techniques. The first is the disentangled attention mechanism, where each word is represented using two vectors that encode its content and position, respectively, and the attention weights among words are computed using disentangled matrices on their contents and relative positions, respectively. Second, an enhanced mask decoder is used to incorporate absolute positions in the decoding layer to predict the masked tokens in model pre-training. In addition, a new virtual adversarial training method is used for fine-tuning to improve models' generalization. We show that these techniques significantly improve the efficiency of model pre-training and the performance of both natural language understanding (NLU) and natural langauge generation (NLG) downstream tasks. Compared to RoBERTa-Large, a DeBERTa model trained on half of the training data performs consistently better on a wide range of NLP tasks, achieving improvements on MNLI by +0.9% (90.2% vs. 91.1%), on SQuAD v2.0 by +2.3% (88.4% vs. 90.7%) and RACE by +3.6% (83.2% vs. 86.8%). Notably, we scale up DeBERTa by training a larger version that consists of 48 Transform layers with 1.5 billion parameters. The significant performance boost makes the single DeBERTa model surpass the human performance on the SuperGLUE benchmark (Wang et al., 2019a) for the first time in terms of macro-average score (89.9 versus 89.8), and the ensemble DeBERTa model sits atop the SuperGLUE leaderboard as of January 6, 2021, out performing the human baseline by a decent margin (90.3 versus 89.8). Authors: Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ift.tt/3dJpBrR BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB Parler: https://ift.tt/38tQU7C LinkedIn: https://ift.tt/2Zo6XRA BiliBili: https://ift.tt/3mfyjkW If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://ift.tt/2DuKOZ3 Patreon: https://ift.tt/390ewRH Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

AI The Tool For Tomorrow- A Dive Into Machine Learning

Come and Join As we explore Machine Learning in AI Wanna Learn C# Get Started https://www.sololearn.com https://dotnet.microsoft.com/apps/machinelearning-ai/ml-dotnet

Number Pattern Program in Java

Free Online Tutorials https://www.goeduhub.com/8464/online-tutorials-and-certification-courses Artificial Intelligence(AI) & Machine Learning(ML) training in Jaipur | Best Summer Winter Training Internship https://www.goeduhub.com/ai-machine-learning-tensorflow-best-summer-winter-industrial-internship-training

Wednesday, February 24, 2021

This is What Abraham Lincoln Really Looked Like! 🎩

❤️ Check out Weights & Biases and sign up for a free demo here: https://ift.tt/2YuG7Yf ❤️ Their mentioned post is available here: https://ift.tt/357IzXd 📝 The paper "Time-Travel Rephotography" is available here: https://ift.tt/3rmu0bk 📝 Our "Separable Subsurface Scattering" paper with Activision Blizzard is available here: https://ift.tt/2YhJnn0 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Haro, Alex Serban, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Lau, Eric Martel, Gordon Child, Haris Husic, Jace O'Brien, Javier Bustamante, Joshua Goller, Kenneth Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Mark Oates, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Robin Graham, Steef, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. If you wish to appear here or pick up other perks, click here: https://ift.tt/2icTBUb Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/twominutepapers Web: https://ift.tt/1NwkG9m

Friday, February 19, 2021

4 AI Machine Learning Data Science | Machine Learning Tutorial

Wednesday, February 17, 2021

AI / Machine Learning Rotoscoping Tutorial (?) with RunwayML

Quick and easy (Rough) rotoscoping in https://runwayml.com/ with Machine Learning.

TransGAN: Two Transformers Can Make One Strong GAN (Machine Learning Research Paper Explained)

#transformer #gan #machinelearning Generative Adversarial Networks (GANs) hold the state-of-the-art when it comes to image generation. However, while the rest of computer vision is slowly taken over by transformers or other attention-based architectures, all working GANs to date contain some form of convolutional layers. This paper changes that and builds TransGAN, the first GAN where both the generator and the discriminator are transformers. The discriminator is taken over from ViT (an image is worth 16x16 words), and the generator uses pixelshuffle to successfully up-sample the generated resolution. Three tricks make training work: Data augmentations using DiffAug, an auxiliary superresolution task, and a localized initialization of self-attention. Their largest model reaches competitive performance with the best convolutional GANs on CIFAR10, STL-10, and CelebA. OUTLINE: 0:00 - Introduction & Overview 3:05 - Discriminator Architecture 5:25 - Generator Architecture 11:20 - Upsampling with PixelShuffle 15:05 - Architecture Recap 16:00 - Vanilla TransGAN Results 16:40 - Trick 1: Data Augmentation with DiffAugment 19:10 - Trick 2: Super-Resolution Co-Training 22:20 - Trick 3: Locality-Aware Initialization for Self-Attention 27:30 - Scaling Up & Experimental Results 28:45 - Recap & Conclusion Paper: https://ift.tt/3qsHYaF Code: https://ift.tt/3tZZXr7 My Video on ViT: https://youtu.be/TrdevFK_am4 Abstract: The recent explosive interest on transformers has suggested their potential to become powerful "universal" models for computer vision tasks, such as classification, detection, and segmentation. However, how further transformers can go - are they ready to take some more notoriously difficult vision tasks, e.g., generative adversarial networks (GANs)? Driven by that curiosity, we conduct the first pilot study in building a GAN \textbf{completely free of convolutions}, using only pure transformer-based architectures. Our vanilla GAN architecture, dubbed \textbf{TransGAN}, consists of a memory-friendly transformer-based generator that progressively increases feature resolution while decreasing embedding dimension, and a patch-level discriminator that is also transformer-based. We then demonstrate TransGAN to notably benefit from data augmentations (more than standard GANs), a multi-task co-training strategy for the generator, and a locally initialized self-attention that emphasizes the neighborhood smoothness of natural images. Equipped with those findings, TransGAN can effectively scale up with bigger models and high-resolution image datasets. Specifically, our best architecture achieves highly competitive performance compared to current state-of-the-art GANs based on convolutional backbones. Specifically, TransGAN sets \textbf{new state-of-the-art} IS score of 10.10 and FID score of 25.32 on STL-10. It also reaches competitive 8.64 IS score and 11.89 FID score on Cifar-10, and 12.23 FID score on CelebA 64×64, respectively. We also conclude with a discussion of the current limitations and future potential of TransGAN. The code is available at \url{this https URL}. Authors: Yifan Jiang, Shiyu Chang, Zhangyang Wang Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ift.tt/3dJpBrR BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB Parler: https://ift.tt/38tQU7C LinkedIn: https://ift.tt/2Zo6XRA BiliBili: https://ift.tt/3mfyjkW If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://ift.tt/2DuKOZ3 Patreon: https://ift.tt/390ewRH Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

Tuesday, February 16, 2021

These Neural Networks Have Superpowers! 💪

❤️ Check out Weights & Biases and sign up for a free demo here: https://ift.tt/2YuG7Yf ❤️ Their mentioned post is available here: https://ift.tt/2OAR8Uq 📝 The paper "Taming Transformers for High-Resolution Image Synthesis" is available here: https://ift.tt/3amnCLd 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Haro, Alex Serban, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Lau, Eric Martel, Gordon Child, Haris Husic, Jace O'Brien, Javier Bustamante, Joshua Goller, Kenneth Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Robin Graham, Steef, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. If you wish to support the series, click here: https://ift.tt/2icTBUb Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/twominutepapers Web: https://ift.tt/1NwkG9m

Sunday, February 14, 2021

NFNets: High-Performance Large-Scale Image Recognition Without Normalization (ML Paper Explained)

#nfnets #deepmind #machinelearning Batch Normalization is a core component of modern deep learning. It enables training at higher batch sizes, prevents mean shift, provides implicit regularization, and allows networks to reach higher performance than without. However, BatchNorm also has disadvantages, such as its dependence on batch size and its computational overhead, especially in distributed settings. Normalizer-Free Networks, developed at Google DeepMind, are a class of CNNs that achieve state-of-the-art classification accuracy on ImageNet without batch normalization. This is achieved by using adaptive gradient clipping (AGC), combined with a number of improvements in general network architecture. The resulting networks train faster, are more accurate, and provide better transfer learning performance. Code is provided in Jax. OUTLINE: 0:00 - Intro & Overview 2:40 - What's the problem with BatchNorm? 11:00 - Paper contribution Overview 13:30 - Beneficial properties of BatchNorm 15:30 - Previous work: NF-ResNets 18:15 - Adaptive Gradient Clipping 21:40 - AGC and large batch size 23:30 - AGC induces implicit dependence between training samples 28:30 - Are BatchNorm's problems solved? 30:00 - Network architecture improvements 31:10 - Comparison to EfficientNet 33:00 - Conclusion & Comments Paper: https://ift.tt/3rK4axl Code: https://ift.tt/3jOwU5h My Video on BatchNorm: https://www.youtube.com/watch?v=OioFONrSETc My Video on ResNets: https://www.youtube.com/watch?v=GWt6Fu05voI Abstract: Batch normalization is a key component of most image classification models, but it has many undesirable properties stemming from its dependence on the batch size and interactions between examples. Although recent work has succeeded in training deep ResNets without normalization layers, these models do not match the test accuracies of the best batch-normalized networks, and are often unstable for large learning rates or strong data augmentations. In this work, we develop an adaptive gradient clipping technique which overcomes these instabilities, and design a significantly improved class of Normalizer-Free ResNets. Our smaller models match the test accuracy of an EfficientNet-B7 on ImageNet while being up to 8.7x faster to train, and our largest models attain a new state-of-the-art top-1 accuracy of 86.5%. In addition, Normalizer-Free models attain significantly better performance than their batch-normalized counterparts when finetuning on ImageNet after large-scale pre-training on a dataset of 300 million labeled images, with our best models obtaining an accuracy of 89.2%. Our code is available at this https URL deepmind-research/tree/master/nfnets Authors: Andrew Brock, Soham De, Samuel L. Smith, Karen Simonyan Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ift.tt/3dJpBrR BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB Parler: https://ift.tt/38tQU7C LinkedIn: https://ift.tt/2Zo6XRA BiliBili: https://ift.tt/3mfyjkW If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://ift.tt/2DuKOZ3 Patreon: https://ift.tt/390ewRH Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

Saturday, February 13, 2021

Mind Reading For Brain-To-Text Communication! 🧠

❤️ Check out Lambda here and sign up for their GPU Cloud: https://ift.tt/35NkCT7 📝 The paper "High-performance brain-to-text communication via imagined handwriting" is available here: https://ift.tt/3tXJt2X ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://ift.tt/2icTBUb - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Haro, Alex Serban, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Lau, Eric Martel, Gordon Child, Haris Husic, Jace O'Brien, Javier Bustamante, Joshua Goller, Kenneth Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Robin Graham, Steef, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. If you wish to support the series, click here: https://ift.tt/2icTBUb Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/twominutepapers Web: https://ift.tt/1NwkG9m #braintotext

Friday, February 12, 2021

L : 6 Relationship Between Artificial Intelligence, Machine Learning & Deep Learning.

Lecture 1 : What is Artificial Intelligence? https://youtu.be/KmSq57W-Kdw ----------------------------------------------------------------------------------- Subscribe To Our Channel: https://www.youtube.com/channel/UC1JT... -------------------------------------------------------------------------------------- Learn Artificial Intelligence: https://youtu.be/KmSq57W-Kdw ----------------------------------------------------------------------------------- Basic Structure Of C Program: https://youtu.be/hXzaKOUpRKo ------------------------------------------------------------------------------------- introduction to data structures and algorithms in Hindi In 10 Min. https://youtu.be/0B4Uv60K8QA --------------------------------------------------------------------------------------- Basics of object oriented programming language (oop's) in 10 min in Hindi: https://youtu.be/aYkGEiPKKhY --------------------------------------------------------------------------------------- Learn Data Base Management System : https://youtu.be/Jc6uq4zvCZc --------------------------------------------------------------------------------------- Learn "How To Make an Login Form" : https://youtu.be/BvOVX4iGHVA ----------------------------------------------------------------------------------------- Relationship Between Artificial Intelligence, Machine Learning & Deep Learning. #RelationshipBetweenAI,ML&DL In This Tutorial You Will Learn Relationship Between Artificial Intelligence, Machine Learning & Deep Learning., Subscribe Study Mode YT Channel To Learn More About Trending Technology -------------------------------------------------------------------------------------

Thursday, February 11, 2021

L : 6 Relationship Between Artificial Intelligence, Machine Learning & Deep Learning.

Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention (AI Paper Explained)

#transformer #nystromer #nystromformer The Nyströmformer (or Nystromformer, Nyströmer, Nystromer), is a new drop-in replacement for approximating the Self-Attention matrix in Transformers with linear memory and time requirements. Most importantly, it uses the Nystrom-Method to subselect (or segment mean) queries and keys as so-called landmarks and uses those to reconstruct the inherently low-rank attention matrix. This is relevant for many areas of Machine Learning, especially Natural Language processing, where it enables longer sequences of text to be processed at once. OUTLINE: 0:00 - Intro & Overview 2:30 - The Quadratic Memory Bottleneck in Self-Attention 7:20 - The Softmax Operation in Attention 11:15 - Nyström-Approximation 14:00 - Getting Around the Softmax Problem 18:05 - Intuition for Landmark Method 28:05 - Full Algorithm 30:20 - Theoretical Guarantees 35:55 - Avoiding the Large Attention Matrix 36:55 - Subsampling Keys vs Negative Sampling 43:15 - Experimental Results 47:00 - Conclusion & Comments Paper: https://ift.tt/2LATtOc Code: https://ift.tt/3jximXA Appendix: https://ift.tt/3aTRcGu LRA Results: https://twitter.com/tanmingxing/status/1359301186734620675 Twitter lucidrains w/ author: https://twitter.com/lucidrains/status/1359597104075661312 Twitter lucidrains w/ _clashluke: https://twitter.com/_clashluke/status/1359483460851802115 Abstract: Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the self-attention mechanism that encodes the influence or dependence of other tokens on each specific token. While beneficial, the quadratic complexity of self-attention on the input sequence length has limited its application to longer sequences -- a topic being actively studied in the community. To address this limitation, we propose Nyströmformer -- a model that exhibits favorable scalability as a function of sequence length. Our idea is based on adapting the Nyström method to approximate standard self-attention with O(n) complexity. The scalability of Nyströmformer enables application to longer sequences with thousands of tokens. We perform evaluations on multiple downstream tasks on the GLUE benchmark and IMDB reviews with standard sequence length, and find that our Nyströmformer performs comparably, or in a few cases, even slightly better, than standard Transformer. Our code is at this https URL. Authors: Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, Vikas Singh Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://ift.tt/3dJpBrR BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB Parler: https://ift.tt/38tQU7C LinkedIn: https://ift.tt/2Zo6XRA BiliBili: https://ift.tt/3mfyjkW If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://ift.tt/2DuKOZ3 Patreon: https://ift.tt/390ewRH Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

Wednesday, February 10, 2021

Join us at TensorFlow Everywhere

TensorFlow Everywhere is a series of global events led by TensorFlow and machine learning communities around the world. RSVP for an event near you at https://ift.tt/2OsWHnZ and join the conversation on social #TFEverywhere2021.

Tuesday, February 9, 2021

Perfect Virtual Hands - But At A Cost! 👐

❤️ Check out Perceptilabs and sign up for a free demo here: https://ift.tt/2WIdXXn 📝 The paper "Constraining Dense Hand Surface Tracking with Elasticity" is available here: https://ift.tt/3nlFTf2 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Haro, Alex Serban, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Eric Haddad, Eric Lau, Eric Martel, Gordon Child, Haris Husic, Jace O'Brien, Javier Bustamante, Joshua Goller, Kenneth Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Ramsey Elbasheer, Robin Graham, Steef, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. If you wish to support the series, click here: https://ift.tt/2icTBUb Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/twominutepapers Web: https://ift.tt/1NwkG9m

Machine Intelligence | Meta-Consolidation for Continual Learning | Joseph KJ

Check out the presentation where our Machine Intelligence group hosts Reading Group session with Joseph KJ(CS PhD Student, IITH) to discuss about his work 'Meta-Consolidation for Continual Learning', published in NeurIPS 2020. Paper Link: https://arxiv.org/abs/2010.00352

Monday, February 8, 2021

AI & Machine Learning Workshop: The Tutorial before your Tutorial - Part 5

#MachineLearningTutorial #AI #MachineLearning #Tutorial #ScienceandTechnology #ArtificialIntelligence #TensorFlow #Keras #SupervisedLearning #NeuralNetworks #Perceptron #Backpropagation Checkout out Part 1 of this series: https://youtu.be/poQp5N2flOw Checkout out Part 2 of this series: https://youtu.be/3R1ahtudvbM Checkout out Part 3 of this series: https://youtu.be/97CiAjqbCpU Checkout out Part 4 of this series: https://youtu.be/y7_UTqwx5Y0 Artificial Intelligence and Machine Learning with TensorFlow/Keras is a confusing and sometimes incomprehensible subject to learn on your own. The Google Machine Learning Crash Course is a good tutorial to learn AI/ML if you already have a background on the subject. The purpose of this workshop is the be the tutorial before to take the Google tutorial. I've been there and now I'm ready to pass it forward and share what I've learned. I'm not an expert but I have working code examples that I will use to teach you based on my current level of understanding of the subject. Here is the list of topics explained in this Machine Learning basics video: 1.Topics & Recap of Part 4 - (0:18) 2. Introduction to Neural Networks - (1:25) 3. Neural Networks: Learning and Training - (3:23) 4. Perceptrons - (5:41) 5. Perceptron Magic and Perceptron Anatomy - (8:16) Like/follow us on Facebook: https://www.facebook.com/Black-Magic-AI-109126344070229 Check out our Web site: https://www.blackmagicai.com/ Background Music Royalty Free background music from Bensound.com.

Sunday, February 7, 2021

Data Analytics using R | Factors in R 3.18 | AI/ML Tutorial

Imurgence, was founded to leverage training expertise and the resource building capabilities inherited from Simple and Real Analytics. The Certificate version of the same course is available on imurgence.com under the course name iDS Plus without any charges. On YouTube we cannot have the Project Submission, Certificate completion and data management , so the same is made available on website without any cost. The Course is a lucid one which help's a beginner to Learn concepts of data analytics, data science and advanced machine learning using R and Python with hands-on case study. At the end of the course the user would be able to apply the leanings on practical used cases in business. Upon successful completion of this course, the learner will be skilled in Machine Learning and AI using R & Python. Target Audience This course is ideal for anyone looking to improve their skills or start a career in data science, business analytics, artificial intelligence (AI) or machine learning. Prerequisites There are no prerequisites for this course. Available on Website same content with the following Type of Certification Certificate of Completion Format of Certification Digital Professional Association/Affiliation The certificate is issued by Imurgence an autonomous institution Method of Obtaining Certification Upon successfully completing 80% of this course, the learner will be able to download digital copy of the Certificate and Mark sheet from the Certificates section . The Mark Sheet will keep on updating as the learner progresses towards 100% completion. And also Follow and like our Facebook Page: https://www.facebook.com/Imurgence/ Follow us on Twitter: https://twitter.com/imurgence Follow us @imurgence on Instagram: https://www.instagram.com/imurgence/ Subscribe and like our YouTube Channel: https://www.youtube.com/channel/UCHfrkjoyXZZN207VSfecJ2A Use the Youtube Play list for learning: https://www.youtube.com/channel/UCHfrkjoyXZZN207VSfecJ2A/playlists Like our Company Page on LinkedIn: https://www.linkedin.com/company/imurgence For Blogs visit the link: https://www.imurgence.com/home/blogs Google MyBusiness Listing: https://g.page/Imurgence?we