ML with Recurrent Neural Networks (NLP Zero to Hero - Part 4)

Welcome to this episode in Natural Language Processing Zero to Hero with TensorFlow. In the previous videos in this series you saw how to tokenize text, and use sequences of tokens to train a neural network. In the next videos we’ll look at how neural networks can generate text and even write poetry, beginning with an introduction to Recurrent Neural Networks (RNNs). Irish songs generator Colab → https://goo.gle/3aSTLGx Watch more #CodingTensorFlow → https://goo.gle/2Y43cN4 Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies (Paper Explained)

Hail the AI Tax Collector! This very visual framework has RL Agents maximize their coins in a tiny world through collecting, building and trading. But at the same time, the government is also an AI trying to maximize social welfare via taxes. What emerges is very interesting. Paper: https://ift.tt/2YoUEEx Blog: https://ift.tt/2KJTKdv Abstract: Tackling real-world socio-economic challenges requires designing and testing economic policies. However, this is hard in practice, due to a lack of appropriate (micro-level) economic data and limited opportunity to experiment. In this work, we train social planners that discover tax policies in dynamic economies that can effectively trade-off economic equality and productivity. We propose a two-level deep reinforcement learning approach to learn dynamic tax policies, based on economic simulations in which both agents and a government learn and adapt. Our data-driven approach does not make use of economic modeling assumptions, and learns from observational data alone. We make four main contributions. First, we present an economic simulation environment that features competitive pressures and market dynamics. We validate the simulation by showing that baseline tax systems perform in a way that is consistent with economic theory, including in regard to learned agent behaviors and specializations. Second, we show that AI-driven tax policies improve the trade-off between equality and productivity by 16% over baseline policies, including the prominent Saez tax framework. Third, we showcase several emergent features: AI-driven tax policies are qualitatively different from baselines, setting a higher top tax rate and higher net subsidies for low incomes. Moreover, AI-driven tax policies perform strongly in the face of emergent tax-gaming strategies learned by AI agents. Lastly, AI-driven tax policies are also effective when used in experiments with human participants. In experiments conducted on MTurk, an AI tax policy provides an equality-productivity trade-off that is similar to that provided by the Saez framework along with higher inverse-income weighted social welfare. Authors: Stephan Zheng, Alexander Trott, Sunil Srinivasa, Nikhil Naik, Melvin Gruesbeck, David C. Parkes, Richard Socher Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

First Return Then Explore

This video explores "First Return Then Explore", the latest advancement of the Go-Explore algorithm. This paper introduces Policy-based Go-Explore where the agent is trained to return to the frontier of explored states, rather than just resetting the simulator state. This helps with stochasticity during training, removes the need for a second robustify phase, and provides a better policy for exploration from the most promising state. Thanks for watching! Please Subscribe! Paper Links: First return then explore: https://ift.tt/3aKYQAZ Go-Explore: https://ift.tt/2W9ieCm The Ingredients of Real-World RL: https://ift.tt/2VHwuTV Domain Randomization for Sim2Real Transfer: https://ift.tt/2Gkp3tA Beyond Domain Randomization: https://ift.tt/2YiVbri Jeff Clune at Rework on Go-Explore: https://www.youtube.com/watch?v=SWcuTgk2di8&t=862s World models: https://ift.tt/2IYv5zG Solving Rubik's Cube with a Robot Hand: https://ift.tt/2Mk3yMZ Exploration based language learning for text-based games: https://ift.tt/35foFYw Abandoning Objectives: https://ift.tt/2yVYXMy Specification Gaming: https://ift.tt/2RWPUle Upside-Down RL: https://ift.tt/2YjsYAW Chip Design with Deep Reinforcement Learning: https://ift.tt/3asCKTr Thanks for watching! Please Subscribe!

Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask

This paper dives into the intrinsics of the Lottery Ticket Hypothesis and attempts to shine some light on what's important and what isn't. https://ift.tt/32OR0CA Abstract: The recent "Lottery Ticket Hypothesis" paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keeping the large weights) results in models that are trainable from scratch, but only when starting from the same initial weights. The performance of these networks often exceeds the performance of the non-sparse base model, but for reasons that were not well understood. In this paper we study the three critical components of the Lottery Ticket (LT) algorithm, showing that each may be varied significantly without impacting the overall results. Ablating these factors leads to new insights for why LT networks perform as well as they do. We show why setting weights to zero is important, how signs are all you need to make the reinitialized network train, and why masking behaves like training. Finally, we discover the existence of Supermasks, masks that can be applied to an untrained, randomly initialized network to produce a model with performance far better than chance (86% on MNIST, 41% on CIFAR-10). Authors: Hattie Zhou, Janice Lan, Rosanne Liu, Jason Yosinski Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

What’s Inside a Neural Network?

❤️ Check out Weights & Biases and sign up for a free demo here: https://ift.tt/2YuG7Yf The shown blog post is available here: https://ift.tt/2okfRiO 📝 The paper "Zoom In: An Introduction to Circuits" is available here: https://ift.tt/39EOZgn Followup article: https://ift.tt/2XpdrPo 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bruno Mikuš, Bryan Learn, Christian Ahlin, Daniel Hasegan, Eric Haddad, Eric Martel, Javier Bustamante, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Michael Albrecht, Nader S., Owen Campbell-Moore, Rob Rowe, Robin Graham, Steef, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh More info if you would like to appear here: https://ift.tt/2icTBUb Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://ift.tt/2TnVBd3 Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/twominutepapers Web: https://ift.tt/1NwkG9m

[Rant] Online Conferences

Are virtual conferences good or bad? What's missing? How do we go forward? Pictures from here: https://twitter.com/srush_nlp/status/1253786329575538691 Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

AI Weekly Update - April 27th, 2020 (#19)

Thanks for watching! Please Subscribe! Please check out Machine Learning Street Talk! https://www.youtube.com/channel/UCMLtBahI5DMrt0NPvDSoIRQ Chip Design with Reinforcement Learning: https://ift.tt/3asCKTr Supervised Contrastive Learning: https://ift.tt/2VXJfst The Future of NLP: https://www.youtube.com/watch?v=G5lmya6eKtc Experience Grounds Language: https://ift.tt/2y5hWnR The Ingredients of Real World Robotic Reinforcement Learning; https://ift.tt/2VHwuTV Towards Learning Robots which can Adapt: https://ift.tt/2Yc6l1c Adversarial Latent Autoencoders: https://ift.tt/2Kx6ST4 QuantNoise: https://ift.tt/2W11mOc ResNeSt: https://ift.tt/2xe3RnL Automating Data Augmentation: https://ift.tt/2y4425A Specification Gaming: https://ift.tt/2RWPUle Analyzing RL benchmarks with random weight guessing: https://ift.tt/3eGJlNB PyBoy: https://ift.tt/2hMHkXH ICLR workshop on Slideslive: https://ift.tt/2ScuJvz

Do ImageNet Classifiers Generalize to ImageNet? (Paper Explained)

Has the world overfitted to ImageNet? What if we collect another dataset in exactly the same fashion? This paper gives a surprising answer! Paper: https://ift.tt/3cUM847 Data: https://ift.tt/3cMvDqs Abstract: We build new test sets for the CIFAR-10 and ImageNet datasets. Both benchmarks have been the focus of intense research for almost a decade, raising the danger of overfitting to excessively re-used test sets. By closely following the original dataset creation processes, we test to what extent current classification models generalize to new data. We evaluate a broad range of models and find accuracy drops of 3% - 15% on CIFAR-10 and 11% - 14% on ImageNet. However, accuracy gains on the original test sets translate to larger gains on the new test sets. Our results suggest that the accuracy drops are not caused by adaptivity, but by the models' inability to generalize to slightly "harder" images than those found in the original test sets. Authors: Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, Vaishaal Shankar Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

[Drama] Schmidhuber: Critique of Honda Prize for Dr. Hinton

Schmidhuber writes up a critique of Hinton receiving the Honda Price... AND HINTON REPLIES! Schmidhuber's Blog Entry: https://ift.tt/3boaWkE Hinton's Reply: https://ift.tt/3cEg701 Thumbnail Images: By Eviatar Bach -https://ift.tt/2Kxf5Hb By ITU/R.Farrell - https://ift.tt/3aHJxc2, CC BY 2.0, https://ift.tt/2Sb1eKC Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Is Simulating Soft and Bouncy Jelly Possible? 🦑

❤️ Check out Lambda here and sign up for their GPU Cloud: https://ift.tt/35NkCT7 📝 The paper "A Hybrid Material Point Method for Frictional Contact with Diverse Materials" is available here: https://ift.tt/2W3quUy ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://ift.tt/2icTBUb - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Christian Ahlin, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. https://ift.tt/2icTBUb Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://ift.tt/2TnVBd3 Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/karoly_zsolnai Web: https://ift.tt/1NwkG9m

How much memory does Longformer use?

A calculation of the memory requirements of the Longformer. Original video: https://youtu.be/_8KNb5iqblE Paper: https://ift.tt/2VnRHRo Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Neural Networks from Scratch - P.3 The Dot Product

Neural Networks from Scratch book: https://nnfs.io NNFSiX Github: https://ift.tt/2VybXkn Playlist for this series: https://www.youtube.com/playlist?list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3 Neural Networks IN Scratch (the programming language): https://youtu.be/eJ1HdTZAcn4 Python 3 basics: https://ift.tt/37OxERs Intermediate Python (w/ OOP): https://ift.tt/2UKxT97 Mug link for fellow mug aficionados: https://amzn.to/3cKEokU Channel membership: https://www.youtube.com/channel/UCfzlCWGWYyIQ0aLC5w48gBQ/join Discord: https://ift.tt/2AZiVqD Support the content: https://ift.tt/2qsKFOO Twitter: https://twitter.com/sentdex Instagram: https://ift.tt/2J4Oa4h Facebook: https://ift.tt/1OI3cwB Twitch: https://ift.tt/2pcWGaq #nnfs #python #neuralnetworks

Supervised Contrastive Learning

The cross-entropy loss has been the default in deep learning for the last few years for supervised learning. This paper proposes a new loss, the supervised contrastive loss, and uses it to pre-train the network in a supervised fashion. The resulting model, when fine-tuned to ImageNet, achieves new state-of-the-art. https://ift.tt/2VXJfst Abstract: Cross entropy is the most widely used loss function for supervised training of image classification models. In this paper, we propose a novel training methodology that consistently outperforms cross entropy on supervised learning tasks across different architectures and data augmentations. We modify the batch contrastive loss, which has recently been shown to be very effective at learning powerful representations in the self-supervised setting. We are thus able to leverage label information more effectively than cross entropy. Clusters of points belonging to the same class are pulled together in embedding space, while simultaneously pushing apart clusters of samples from different classes. In addition to this, we leverage key ingredients such as large batch sizes and normalized embeddings, which have been shown to benefit self-supervised learning. On both ResNet-50 and ResNet-200, we outperform cross entropy by over 1%, setting a new state of the art number of 78.8% among methods that use AutoAugment data augmentation. The loss also shows clear benefits for robustness to natural corruptions on standard benchmarks on both calibration and accuracy. Compared to cross entropy, our supervised contrastive loss is more stable to hyperparameter settings such as optimizers or data augmentations. Authors: Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, Dilip Krishnan Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

This video explores the T5 large-scale study on Transfer Learning. This paper takes apart many different factors of the Pre-Training then Fine-Tuning pipeline for NLP. This involves Auto-Regressive Language Modeling vs. BERT-Style Masked Language Modeling and XLNet-style shuffling, as well as the impact of dataset composition, size, and how to best use more computation. Thanks for watching and please check out Machine Learning Street Talk where Tim Scarfe, Yannic Kilcher and I discuss this paper! Machine Learning Street Talk: https://www.youtube.com/channel/UCMLtBahI5DMrt0NPvDSoIRQ Paper Links: T5: https://ift.tt/2pcuaXx Google AI Blog Post on T5: https://ift.tt/2SV4VF9 Train Large, Then Compress: https://ift.tt/3awfC74 Scaling Laws for Neural Language Models: https://ift.tt/2yzOOVY The Illustrated Transformer: https://ift.tt/2NLJXmf ELECTRA: https://ift.tt/2RZsM5S Transformer-XL: https://ift.tt/2LIaXXb Reformer: The Efficient Transformer: https://ift.tt/378kuhh The Evolved Transformer: https://ift.tt/2IAdYFw DistilBERT: https://ift.tt/2Y2cZa2 How to generate text (HIGHLY RECOMMEND): https://ift.tt/3d9QC7P Tokenizers: https://ift.tt/2vpu7Kx Thanks for watching! Please Subscribe!

Thinking While Moving: Deep Reinforcement Learning with Concurrent Control (Video Analysis)

Classic RL "stops" the world whenever the Agent computes a new action. This paper considers a more realistic scenario where the agent is thinking about the next action to take while still performing the last action. This results in a fascinating way of reformulating Q-learning in continuous time, then introducing concurrency and finally going back to discrete time. https://ift.tt/2xrdLTb Abstract: We study reinforcement learning in settings where sampling an action from the policy must be done concurrently with the time evolution of the controlled system, such as when a robot must decide on the next action while still performing the previous action. Much like a person or an animal, the robot must think and move at the same time, deciding on its next action before the previous one has completed. In order to develop an algorithmic framework for such concurrent control problems, we start with a continuous-time formulation of the Bellman equations, and then discretize them in a way that is aware of system delays. We instantiate this new class of approximate dynamic programming methods via a simple architectural extension to existing value-based deep reinforcement learning algorithms. We evaluate our methods on simulated benchmark tasks and a large-scale robotic grasping task where the robot must "think while moving". Authors: Ted Xiao, Eric Jang, Dmitry Kalashnikov, Sergey Levine, Julian Ibarz, Karol Hausman, Alexander Herzog Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

This AI Creates Beautiful Daytime Images ☀️

❤️ Check out Weights & Biases and sign up for a free demo here: https://ift.tt/2YuG7Yf The shown blog post is available here: https://ift.tt/2VuK7Wr 📝 The paper "High-Resolution Daytime Translation Without Domain Labels" is available here: https://ift.tt/3bvgwS7 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Christian Ahlin, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. https://ift.tt/2icTBUb Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://ift.tt/2TnVBd3 Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/karoly_zsolnai Web: https://ift.tt/1NwkG9m

[Rant] The Male Only History of Deep Learning

This casting of our field in terms of ideological narrow-sighted group-think is disgusting. Keep Science about ideas! https://twitter.com/timnitGebru/status/1252752743942328321 Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Richard Burton | Siraj Raval Podcast #5

This is my conversation with Richard Burton. He was the 1st designer at Ethereum, an early employee at Stripe, invests in technology startups, and is an avid kitesurfer. We've been friends for years so this was the funnest interview I've done yet. Enjoy! Time Markers below: 2:45 Meeting in the Hacker Hostel 9:15 Short-term economic effects of the pandemic 10:30 Richard's background at Stripe & Ethereum 17:15 Open Source Financial Systems 21:00 Richard's crowd-sourced startup (Balance) 25:00 On Remote Work 31:30 Interview rejections & getting fired 33:15 Working for a Social Media algorithm 37:00 Deep Learning has been overhyped 41:00 Opportunities in Decentralized Finance 47:00 When Cryptocurrency was overhyped 50:18 Self-Driving Tesla Problems 51:30 Bipolar Disorder 53:00 Homelessness in San Francisco 55:15 Social Media Platforms Pros & Cons 59:00 Twitter BlueSky project 01:03:00 The Future of Social Media algorithms 01:04:55 Digital Therapeutics 01:08:30 Dating & relationships 01:12:30 Social Media identities 01:17:00 Finding purpose 01:19:30 AI Education on Youtube 01:22:00 Video release frequency 01:25:00 Lessons learned 01:27:00 Silicon Valley Culture 01:28:15 Los Angeles vs San Francisco 01:30:30 The Reality of Silicon Valley 01:34:50 Judging Technical Competence 01:36:30 WeWork: An Operating System for Atoms 01:37:30 On Peter Thiel 01:40:00 Rejecting platitudes 01:42:50 Richard's book recommendation 01:44:00 "How is Money Created?" 01:47:00 Richard's next topic to learn ---------------------------------------------------------------------------------- Find Richard on Twitter here: https://twitter.com/ricburton Connect with me here: INSTAGRAM: https://bit.ly/312pLUb FACEBOOK: https://bit.ly/2OqOhx1 TWITTER: https://bit.ly/2OHYLbB WEBSITE: https://bit.ly/2OoVPQF Find my podcast on these sites: iTunes: https://ift.tt/2LSz7gI Anchor: https://ift.tt/2Y69xrw Google Podcasts: https://ift.tt/2LUSCFx Spotify: https://ift.tt/2OpE1TU Breaker: https://ift.tt/2Z350Lz Overcast: https://ift.tt/35dzBWj Learn Machine Learning in 3 Months for free: https://www.youtube.com/watch?v=Cr6Vq My Startup Tutorial Playlist: https://www.youtube.com/watch?v=oeraU Hit the Join button above to sign up to become a member of my channel for access to exclusive live streams! Signup for my newsletter for exciting updates in the field of AI: https://goo.gl/FZzJ5w Please support me on Patreon: https://ift.tt/2cMCk13

Gradient Surgery for Multi-Task Learning

Multi-Task Learning can be very challenging when gradients of different tasks are of severely different magnitudes or point into conflicting directions. PCGrad eliminates this problem by projecting conflicting gradients while still retaining optimality guarantees. https://ift.tt/2KiY7vU Abstract: While deep learning and deep reinforcement learning (RL) systems have demonstrated impressive results in domains such as image classification, game playing, and robotic control, data efficiency remains a major challenge. Multi-task learning has emerged as a promising approach for sharing structure across multiple tasks to enable more efficient learning. However, the multi-task setting presents a number of optimization challenges, making it difficult to realize large efficiency gains compared to learning tasks independently. The reasons why multi-task learning is so challenging compared to single-task learning are not fully understood. In this work, we identify a set of three conditions of the multi-task optimization landscape that cause detrimental gradient interference, and develop a simple yet general approach for avoiding such interference between task gradients. We propose a form of gradient surgery that projects a task's gradient onto the normal plane of the gradient of any other task that has a conflicting gradient. On a series of challenging multi-task supervised and multi-task RL problems, this approach leads to substantial gains in efficiency and performance. Further, it is model-agnostic and can be combined with previously-proposed multi-task architectures for enhanced performance. Authors: Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, Chelsea Finn Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Offline Reinforcement Learning

Offline Reinforcement Learning describes training an agent without interacting with the environment. The agent learns from previously collected experiences such as from another RL policy trained online or from a human demonstrator. This video explores two recent advancements in Offline RL! Thanks for watching! Please Subscribe! Paper Links: An Optimistic Perspective on Offline Reinforcement Learning: https://ift.tt/3cjwheT Datasets for Data-Driven Reinforcement Learning: https://ift.tt/2xBQ2Qt Q-Learning (Wikipedia): https://ift.tt/1Bfm6QW AVID (Robot in intro animation): https://ift.tt/38A49TH Nature-inspired robotics (Robot in intro animation): https://ift.tt/34agMD6

Longformer: The Long-Document Transformer

The Longformer extends the Transformer by introducing sliding window attention and sparse global attention. This allows for the processing of much longer documents than classic models like BERT. Paper: https://ift.tt/2VnRHRo Code: https://ift.tt/3cgxYKf Abstract: Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length. To address this limitation, we introduce the Longformer with an attention mechanism that scales linearly with sequence length, making it easy to process documents of thousands of tokens or longer. Longformer's attention mechanism is a drop-in replacement for the standard self-attention and combines a local windowed attention with a task motivated global attention. Following prior work on long-sequence transformers, we evaluate Longformer on character-level language modeling and achieve state-of-the-art results on text8 and enwik8. In contrast to most prior work, we also pretrain Longformer and finetune it on a variety of downstream tasks. Our pretrained Longformer consistently outperforms RoBERTa on long document tasks and sets new state-of-the-art results on WikiHop and TriviaQA. Authors: Iz Beltagy, Matthew E. Peters, Arman Cohan Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

[Reupload] Backpropagation and the brain

Geoffrey Hinton and his co-authors describe a biologically plausible variant of backpropagation and report evidence that such an algorithm might be responsible for learning in the brain. https://ift.tt/3bjj1qv Abstract: During learning, the brain modifies synapses to improve behaviour. In the cortex, synapses are embedded within multilayered networks, making it difficult to determine the effect of an individual synaptic modification on the behaviour of the system. The backpropagation algorithm solves this problem in deep artificial neural networks, but historically it has been viewed as biologically problematic. Nonetheless, recent developments in neuroscience and the successes of artificial neural networks have reinvigorated interest in whether backpropagation offers insights for understanding learning in the cortex. The backpropagation algorithm learns quickly by computing synaptic updates using feedback connections to deliver error signals. Although feedback connections are ubiquitous in the cortex, it is difficult to see how they could deliver the error signals required by strict formulations of backpropagation. Here we build on past and recent developments to argue that feedback connections may instead induce neural activities whose differences can be used to locally approximate these signals and hence drive effective learning in deep networks in the brain. Authors: Timothy P. Lillicrap, Adam Santoro, Luke Marris, Colin J. Akerman & Geoffrey Hinton Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Backpropagation and the brain

Geoffrey Hinton and his co-authors describe a biologically plausible variant of backpropagation and report evidence that such an algorithm might be responsible for learning in the brain. https://ift.tt/3bjj1qv Abstract: During learning, the brain modifies synapses to improve behaviour. In the cortex, synapses are embedded within multilayered networks, making it difficult to determine the effect of an individual synaptic modification on the behaviour of the system. The backpropagation algorithm solves this problem in deep artificial neural networks, but historically it has been viewed as biologically problematic. Nonetheless, recent developments in neuroscience and the successes of artificial neural networks have reinvigorated interest in whether backpropagation offers insights for understanding learning in the cortex. The backpropagation algorithm learns quickly by computing synaptic updates using feedback connections to deliver error signals. Although feedback connections are ubiquitous in the cortex, it is difficult to see how they could deliver the error signals required by strict formulations of backpropagation. Here we build on past and recent developments to argue that feedback connections may instead induce neural activities whose differences can be used to locally approximate these signals and hence drive effective learning in deep networks in the brain. Authors: Timothy P. Lillicrap, Adam Santoro, Luke Marris, Colin J. Akerman & Geoffrey Hinton Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

This AI Can Summarize Videos 🎥

❤️ Check out Linode here and get $20 free credit on your account: https://ift.tt/2LaDQJb 📝 The paper "CLEVRER: CoLlision Events for Video REpresentation and Reasoning" is available here: https://ift.tt/31Lfsox ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://ift.tt/2icTBUb - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Christian Ahlin, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. https://ift.tt/2icTBUb Thumbnail background image credit: https://ift.tt/3cwTgU7 Neural network image credit: https://ift.tt/1Lr0moZ Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://ift.tt/2TnVBd3 Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/karoly_zsolnai Web: https://ift.tt/1NwkG9m

Shortcut Learning in Deep Neural Networks

This paper establishes a framework for looking at out-of-distribution generalization failures of modern deep learning as the models learning false shortcuts that are present in the training data. The paper characterizes why and when shortcut learning can happen and gives recommendations for how to counter its effect. https://ift.tt/34KBdHp Abstract: Deep learning has triggered the current rise of artificial intelligence and is the workhorse of today's machine intelligence. Numerous success stories have rapidly spread all over science, industry and society, but its limitations have only recently come into focus. In this perspective we seek to distil how many of deep learning's problem can be seen as different symptoms of the same underlying problem: shortcut learning. Shortcuts are decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios. Related issues are known in Comparative Psychology, Education and Linguistics, suggesting that shortcut learning may be a common characteristic of learning systems, biological and artificial alike. Based on these observations, we develop a set of recommendations for model interpretation and benchmarking, highlighting recent advances in machine learning to improve robustness and transferability from the lab to real-world applications. Authors: Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, Felix A. Wichmann Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

CURL: Contrastive Unsupervised Representations for Reinforcement Learning

This video explains a new algorithm from UC Berkeley that adds the Momentum Contrastive Learning framework to Reinforcement Learning. This loss improves the mapping from raw observations into latent spaces for control. CURL achieves sample-efficiency and performance gains on the DeepMind Control Suite only from pixel inputs, without the need for physical state inputs! Thanks for watching! Please Subscribe! Paper Links: CURL: https://ift.tt/2UT4PyG Intro Video on the DeepMind Control Suite: https://www.youtube.com/watch?v=rAai4QzcYbs MoCo: https://ift.tt/2qdsi1c MoCo v2: https://ift.tt/2xtZ81r RoboNet: https://ift.tt/2N8KikK World Models: https://ift.tt/2E06KWF MuZero: https://ift.tt/37n6SiK PlaNet: https://ift.tt/2A58PRm Dreamer: https://ift.tt/2qiWpUW

Neural Networks from Scratch - P.2 Coding a Layer

Expanding from a single neuron with 3 inputs to a layer of neurons with 4 inputs. Neural Networks from Scratch book: https://nnfs.io Playlist for this series: https://www.youtube.com/playlist?list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3 Python 3 basics: https://ift.tt/37OxERs Intermediate Python (w/ OOP): https://ift.tt/2UKxT97 Mug link for fellow mug aficionados: https://amzn.to/2Vz9Hs0 Channel membership: https://www.youtube.com/channel/UCfzlCWGWYyIQ0aLC5w48gBQ/join Discord: https://ift.tt/2AZiVqD Support the content: https://ift.tt/2qsKFOO Twitter: https://twitter.com/sentdex Instagram: https://ift.tt/2J4Oa4h Facebook: https://ift.tt/1OI3cwB Twitch: https://ift.tt/2pcWGaq #nnfs #python #neuralnetworks

Feature Visualization & The OpenAI microscope

A closer look at the OpenAI microscope, a database of visualizations of the inner workings of ImageNet classifiers, along with an explanation of how to obtain these visualizations. https://ift.tt/2yFaKOF https://ift.tt/3epU6Ub https://ift.tt/2EeuSWj Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Datasets for Data-Driven Reinforcement Learning

Offline Reinforcement Learning has come more and more into focus recently in domains where classic on-policy RL algorithms are infeasible to train, such as safety-critical tasks or learning from expert demonstrations. This paper presents an extensive benchmark for evaluating offline RL algorithms in a variety of settings. Paper: https://ift.tt/2yo2QtK Code: https://ift.tt/2Vf1daL Abstract: The offline reinforcement learning (RL) problem, also referred to as batch RL, refers to the setting where a policy must be learned from a dataset of previously collected data, without additional online data collection. In supervised learning, large datasets and complex deep neural networks have fueled impressive progress, but in contrast, conventional RL algorithms must collect large amounts of on-policy data and have had little success leveraging previously collected datasets. As a result, existing RL benchmarks are not well-suited for the offline setting, making progress in this area difficult to measure. To design a benchmark tailored to offline RL, we start by outlining key properties of datasets relevant to applications of offline RL. Based on these properties, we design a set of benchmark tasks and datasets that evaluate offline RL algorithms under these conditions. Examples of such properties include: datasets generated via hand-designed controllers and human demonstrators, multi-objective datasets, where an agent can perform different tasks in the same environment, and datasets consisting of a heterogeneous mix of high-quality and low-quality trajectories. By designing the benchmark tasks and datasets to reflect properties of real-world offline RL problems, our benchmark will focus research effort on methods that drive substantial improvements not just on simulated benchmarks, but ultimately on the kinds of real-world problems where offline RL will have the largest impact. Authors: Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, Sergey Levine Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Inside TensorFlow: TF Debugging

In this episode of Inside TensorFlow, Software Engineer Shanqing Cai demonstrates to us TensorFlow Debugging for TF 2 and TF 1. Let us know what you think about this presentation in the comments below! Watch more from Inside TensorFlow playlist → https://goo.gle/31Ge5GF Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

FixMatch is a simple, yet surprisingly effective approach to semi-supervised learning. It combines two previous methods in a clever way and achieves state-of-the-art in regimes with few and very few labeled examples. Paper: https://ift.tt/2upcQ3A Code: https://ift.tt/2NMpTD2 Abstract: Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance. In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm, FixMatch, first generates pseudo-labels using the model's predictions on weakly-augmented unlabeled images. For a given image, the pseudo-label is only retained if the model produces a high-confidence prediction. The model is then trained to predict the pseudo-label when fed a strongly-augmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labels and 88.61% accuracy with 40 -- just 4 labels per class. Since FixMatch bears many similarities to existing SSL methods that achieve worse performance, we carry out an extensive ablation study to tease apart the experimental factors that are most important to FixMatch's success. We make our code available at this https URL. Authors: Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Sure, Deepfake Detectors Exist - But Can They Be Fooled?

❤️ Check out Lambda here and sign up for their GPU Cloud: https://ift.tt/35NkCT7 📝 The paper "Adversarial Deepfakes: Evaluating Vulnerability of Deepfake Detectors to Adversarial Examples" is available here: https://ift.tt/2woQFeS https://ift.tt/2wKhIlC ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://ift.tt/2icTBUb - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Christian Ahlin, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. https://ift.tt/2icTBUb Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://ift.tt/2TnVBd3 Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/karoly_zsolnai Web: https://ift.tt/1NwkG9m #Deepfake

Momentum Contrastive Learning

Contrastive Self-Supervised Learning aims to train representations to distinguish objects from one another. Momentum Contrast is one of the most successful ways of doing this avoiding the memory bottlenecks of end-to-end gradients through query and key encoders and avoiding outdated key encodings through a momentum update from the query and a queue of recently encoded keys! MoCo plays an important role in CURL, achieving DeepMind Control without physical state inputs! Thanks for watching and please subscribe! Please subscribe to Machine Learning Street Talk where Yannic Kilcher, Tim Scarfe and I will be joined by Aravind Srinivas to talk about CURL (in which MoCo plays a key role)! ML Street Talk: https://www.youtube.com/channel/UCMLtBahI5DMrt0NPvDSoIRQ Paper Links: MoCo: https://ift.tt/2pggQ4i MoCo v2: https://ift.tt/2xtZ81r SimCLR: https://ift.tt/31TZZTM CURL: https://ift.tt/2UT4PyG The Empty Brain: https://ift.tt/1sxGdLp

Control of transmon qubits using a cryogenic CMOS integrated circuit (QuantumCasts)

Control of transmon qubits using a cryogenic CMOS integrated circuit talk is presented by Research Scientist Joe Bardin for the APS March Meeting 2020. Superconducting quantum processors are controlled and measured in the analog domain and the design of the associated classical-to-quantum interface is critical in optimizing the overall performance of the quantum computer. Control of the processor is achieved using a combination of carefully shaped microwave pulses and high-precision time varying flux biases. Measurement of quantum states is typically achieved using dispersive readout, which requires a low-power pulsed microwave drive and a near quantum-limited readout chain. For control of a single qubit, a typical system employs two high-speed high-resolution (e.g., 1 Gsps/14 bit) digital-to-analog converters (DACs) and a single-sideband modulator to generate microwave control pulses. A third DAC with similar specifications is used for flux-bias control. A typical readout channel may service on the order of five qubits and contains yet another pair of DACs, with a single-sideband modulator employed to generate a stimulus signal. For measurement, the readout chain also employs a series of cryogenic amplifiers followed by further amplification, IQ demodulation, and high-speed digitization at room temperature. For today’s prototype systems with on the order of 50-100 qubits, keeping most of the electronics at room temperature makes sense. However, achieving fault tolerance—a long term goal of the community—will require implementing systems with on the order of 10^6 qubits and today’s brute force control and readout approach will not scale to these levels. Instead, a more integrated approach will be required. In this talk, we will present a review of recent work towards implementing a scalable cryogenic quantum control and readout system using silicon integrated circuit technology. After motivating the work, we will describe the design and characterization of a prototype cryogenic XY controller for transmon qubits. Detailed measurement results will be presented. The talk will conclude with a discussion of future work. Watch every episode of QuantumCasts here → https://goo.gle/QuantumCasts Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

Extracting coherence information from random circuits (QuantumCasts)

Extracting coherence information from random circuits via 'Sparkle Purity Benchmarking' talk is presented by Quantum Research Scientist Julian Kelly for the APS March Meeting 2020. Budgeting the contributions of coherent and incoherent noise sources is an important component of benchmarking quantum gates. Typically, methods such as Cross Entropy Benchmarking (XEB) or Randomized Benchmarking are used to measure an error-per-gate that includes noise and control errors. These sequences can be extended to quantify the decay of a quantum state due to noise only by measuring the state purity with tomography as described in previous publications. Here, we introduce a method that allows us to extract the same information with exponentially fewer sequences from raw XEB data. We introduce 'Speckle Purity Benchmarking' which quantifies the purity via the contrast (or “speckliness!”) of output bitstring probabilities. Pure quantum states generated by the XEB procedure will have high contrast, while incoherent mixtures will have low contrast. Compared to conventional XEB, this procedure can be done with zero information about the actual quantum process. Additionally, this can be scaled to a handful of qubits. Watch every episode of QuantumCasts here → https://goo.gle/QuantumCasts Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

Quantum supremacy: Benchmarking the Sycamore processor (QuantumCasts)

Quantum supremacy: Benchmarking the Sycamore processor talk is presented by Research Scientist Kevin Satzinger for the APS March Meeting 2020. The promise of quantum computers is that certain computational tasks might be executed exponentially faster on a quantum processor than on a classical processor. A fundamental challenge is to build a high-fidelity processor capable of running quantum algorithms in an exponentially large computational space. Here we report the use of a processor with 53 programmable superconducting qubits. In our Sycamore processor, each qubit interacts with four neighbors in a rectangular lattice using tunable couplers. A key systems engineering advance of this device is achieving high-fidelity single- and two-qubit operations, not just in isolation but also while performing a realistic computation with simultaneous gate operations across the entire processor. We benchmark the Sycamore processor using cross-entropy benchmarking, a scalable method to evaluate system performance. Our largest system benchmarks feature circuits that are intractable for classical hardware, culminating in the demonstration of quantum supremacy. Furthermore, the fidelities from full-system benchmarks agree with what we predict from individual gate and measurement fidelities, verifying the digital error model and presenting a path forward to quantum error correction. Nature 574, 505-510 (2019) Watch every episode of QuantumCasts here → https://goo.gle/QuantumCasts Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

Estimation of statistical significance of quantum supremacy (QuantumCasts)

Estimation of statistical significance in the quantum supremacy experiment with the Sycamore processor talk is presented by Software Engineer Ping Yeh for the APS March Meeting 2020. Google's quantum supremacy experiment is based on sampling of output bitstrings of random quantum circuits [1]. To demonstrate quantum supremacy, it is critical to establish that the hardware fidelity at sampling time is not degraded to zero due to errors and the sampled bitstrings correspond to the expected noisy distribution. In addition, since the run time of classical approximate simulations is proportional to fidelity, it is important to verify that the fidelity is above a threshold at which classical simulation is estimated to be hard. In this talk, I will describe the methodologies for the statistical analyses and show that the bitstring distributions are extremely unlikely to be explained by noise alone and the fidelity is significantly above the threshold. [1] Quantum supremacy using a programmable superconducting processor, Google AI Quantum and collaborators. Nature 574, 505-510 (2019). Watch every episode of QuantumCasts here → https://goo.gle/QuantumCasts Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

Imputer: Sequence Modelling via Imputation and Dynamic Programming

The imputer is a sequence-to-sequence model that strikes a balance between fully autoregressive models with long inference times and fully non-autoregressive models with fast inference. The imputer achieves constant decoding time independent of sequence length by exploiting dynamic programming. https://ift.tt/3emYHXt Abstract: This paper presents the Imputer, a neural sequence model that generates output sequences iteratively via imputations. The Imputer is an iterative generative model, requiring only a constant number of generation steps independent of the number of input or output tokens. The Imputer can be trained to approximately marginalize over all possible alignments between the input and output sequences, and all possible generation orders. We present a tractable dynamic programming training algorithm, which yields a lower bound on the log marginal likelihood. When applied to end-to-end speech recognition, the Imputer outperforms prior non-autoregressive models and achieves competitive results to autoregressive models. On LibriSpeech test-other, the Imputer achieves 11.1 WER, outperforming CTC at 13.0 WER and seq2seq at 12.5 WER. Authors: William Chan, Chitwan Saharia, Geoffrey Hinton, Mohammad Norouzi, Navdeep Jaitly Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks

Stunning evidence for the hypothesis that neural networks work so well because their random initialization almost certainly contains a nearly optimal sub-network that is responsible for most of the final performance. https://ift.tt/2HAmQIJ Abstract: Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy. However, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance. We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively. Based on these results, we articulate the "lottery ticket hypothesis:" dense, randomly-initialized, feed-forward networks contain subnetworks ("winning tickets") that - when trained in isolation - reach test accuracy comparable to the original network in a similar number of iterations. The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective. We present an algorithm to identify winning tickets and a series of experiments that support the lottery ticket hypothesis and the importance of these fortuitous initializations. We consistently find winning tickets that are less than 10-20% of the size of several fully-connected and convolutional feed-forward architectures for MNIST and CIFAR10. Above this size, the winning tickets that we find learn faster than the original network and reach higher test accuracy. Authors: Jonathan Frankle, Michael Carbin Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery

DDL is an auxiliary task for an agent to learn distances between states in episodes. This can then be used further to improve the agent's policy learning procedure. Paper: https://ift.tt/2VsWotk Blog: https://ift.tt/2K0Xxmy Abstract: Reinforcement learning requires manual specification of a reward function to learn a task. While in principle this reward function only needs to specify the task goal, in practice reinforcement learning can be very time-consuming or even infeasible unless the reward function is shaped so as to provide a smooth gradient towards a successful outcome. This shaping is difficult to specify by hand, particularly when the task is learned from raw observations, such as images. In this paper, we study how we can automatically learn dynamical distances: a measure of the expected number of time steps to reach a given goal state from any other state. These dynamical distances can be used to provide well-shaped reward functions for reaching new goals, making it possible to learn complex tasks efficiently. We show that dynamical distances can be used in a semi-supervised regime, where unsupervised interaction with the environment is used to learn the dynamical distances, while a small amount of preference supervision is used to determine the task goal, without any manually engineered reward function or goal examples. We evaluate our method both on a real-world robot and in simulation. We show that our method can learn to turn a valve with a real-world 9-DoF hand, using raw image observations and just ten preference labels, without any other supervision. Videos of the learned skills can be found on the project website: this https URL. Authors: Kristian Hartikainen, Xinyang Geng, Tuomas Haarnoja, Sergey Levine Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

I MADE BETTER AI THAN NINTENDO

The other day I was playing a Nintendo game & the AI was ABYSMAL. So I frustratingly stormed to my computer & make better AI than Nintendo using #MachineLearning. Train your own AI: https://ift.tt/3b3sP81 Github repo: https://ift.tt/3b1wu6v Watch the full Ben Hax vs Forrest battle: https://youtu.be/ougJP16f9uo Sub to my daily channel: https://www.youtube.com/channel/UCoyp8-TdLo_NQWqNI66sLYw/ SUBSCRIBE FOR MORE: http://jabrils.com/yt WISHLIST MY VIDEO GAME: https://ift.tt/33NgHFz SUPPORT ON PATREON: https://ift.tt/2pZACkg JOIN DISCORD: https://ift.tt/2QkDa9O Please follow me on social networks: twitter: https://twitter.com/jabrils_ instagram: https://ift.tt/2QNVYvI REMEMBER TO ALWAYS FEED YOUR CURIOSITY

Neural Network Learns To Look Around In Real Scenes

❤️ Check out Weights & Biases and sign up for a free demo here: https://ift.tt/2YuG7Yf Their amazing instrumentation is available here: https://ift.tt/2wthYVQ 📝 The paper "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis" is available here: https://ift.tt/2xMFwoW 📝 The paper "Gaussian Material Synthesis" is available here: https://ift.tt/2HhNzx5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Christian Ahlin, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. https://ift.tt/2icTBUb Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://ift.tt/2TnVBd3 Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/karoly_zsolnai Web: https://ift.tt/1NwkG9m

Neural Networks from Scratch - P.1 Intro and Neuron Code

Building neural networks from scratch in Python introduction. Neural Networks from Scratch book: https://nnfs.io Python 3 basics: https://ift.tt/37OxERs Intermediate Python (w/ OOP): https://ift.tt/2UKxT97 Channel membership: https://www.youtube.com/channel/UCfzlCWGWYyIQ0aLC5w48gBQ/join Discord: https://ift.tt/2AZiVqD Support the content: https://ift.tt/2qsKFOO Twitter: https://twitter.com/sentdex Instagram: https://ift.tt/2J4Oa4h Facebook: https://ift.tt/1OI3cwB Twitch: https://ift.tt/2pcWGaq #nnfs #python #neuralnetworks

CURL: Contrastive Unsupervised Representations for Reinforcement Learning

Contrastive Learning has been an established method in NLP and Image classification. The authors show that with relatively minor adjustments, CL can be used to augment and improve RL dramatically. Paper: https://ift.tt/2UT4PyG Code: https://ift.tt/2JXIYQI Abstract: We present CURL: Contrastive Unsupervised Representations for Reinforcement Learning. CURL extracts high-level features from raw pixels using contrastive learning and performs off-policy control on top of the extracted features. CURL outperforms prior pixel-based methods, both model-based and model-free, on complex tasks in the DeepMind Control Suite and Atari Games showing 2.8x and 1.6x performance gains respectively at the 100K interaction steps benchmark. On the DeepMind Control Suite, CURL is the first image-based algorithm to nearly match the sample-efficiency and performance of methods that use state-based features. Authors: Aravind Srinivas, Michael Laskin, Pieter Abbeel Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Enhanced POET: Open-Ended RL through Unbounded Invention of Learning Challenges and their Solutions

The enhanced POET makes some substantial and well-crafted improvements over the original POET algorithm and excels at open-ended learning like no system before. https://ift.tt/2xNQmLp https://youtu.be/RX0sKDRq400 Abstract: Creating open-ended algorithms, which generate their own never-ending stream of novel and appropriately challenging learning opportunities, could help to automate and accelerate progress in machine learning. A recent step in this direction is the Paired Open-Ended Trailblazer (POET), an algorithm that generates and solves its own challenges, and allows solutions to goal-switch between challenges to avoid local optima. However, the original POET was unable to demonstrate its full creative potential because of limitations of the algorithm itself and because of external issues including a limited problem space and lack of a universal progress measure. Importantly, both limitations pose impediments not only for POET, but for the pursuit of open-endedness in general. Here we introduce and empirically validate two new innovations to the original algorithm, as well as two external innovations designed to help elucidate its full potential. Together, these four advances enable the most open-ended algorithmic demonstration to date. The algorithmic innovations are (1) a domain-general measure of how meaningfully novel new challenges are, enabling the system to potentially create and solve interesting challenges endlessly, and (2) an efficient heuristic for determining when agents should goal-switch from one problem to another (helping open-ended search better scale). Outside the algorithm itself, to enable a more definitive demonstration of open-endedness, we introduce (3) a novel, more flexible way to encode environmental challenges, and (4) a generic measure of the extent to which a system continues to exhibit open-ended innovation. Enhanced POET produces a diverse range of sophisticated behaviors that solve a wide range of environmental challenges, many of which cannot be solved through other means. Authors: Rui Wang, Joel Lehman, Aditya Rawal, Jiale Zhi, Yulun Li, Jeff Clune, Kenneth O. Stanley Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Inside TensorFlow: TF-Agents

In this episode of Inside TensorFlow, Software Engineers Oscar Ramirez and Sergio Guadarrama demonstrate to us TF-Agents, the latest reinforcement library for TensorFlow. Let us know what you think about this presentation in the comments below! Links: TF-Agents GitHub → https://goo.gle/2X8DT0C TF-Agents Guides and Tutorials → https://goo.gle/2XfXpHp Watch more from Inside TensorFlow playlist → https://goo.gle/31Ge5GF Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

Evolving Normalization-Activation Layers

Normalization and activation layers have seen a long history of hand-crafted variants with various results. This paper proposes an evolutionary search to determine the ultimate, final and best combined normalization-activation layer... in a very specific setting. https://ift.tt/3aTX6Gr Abstract: Normalization layers and activation functions are critical components in deep neural networks that frequently co-locate with each other. Instead of designing them separately, we unify them into a single computation graph, and evolve its structure starting from low-level primitives. Our layer search algorithm leads to the discovery of EvoNorms, a set of new normalization-activation layers that go beyond existing design patterns. Several of these layers enjoy the property of being independent from the batch statistics. Our experiments show that EvoNorms not only excel on a variety of image classification models including ResNets, MobileNets and EfficientNets, but also transfer well to Mask R-CNN for instance segmentation and BigGAN for image synthesis, outperforming BatchNorm and GroupNorm based layers by a significant margin in many cases. Authors: Hanxiao Liu, Andrew Brock, Karen Simonyan, Quoc V. Le Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Evolving Normalization-Activation Layers

This video explains the latest large-scale AutoML study from researchers at Google and DeepMind. The product of this evolutionary AutoML search is a new normalization-activation layer that outperforms the common practice of Batch Norm followed by ReLU! A ResNet-50 with BN-ReLU achieves 76.1% ImageNet accuracy whereas EvoNorm achieves 77.8%! Thanks for watching! Please Subscribe! Paper Links: Evolving Normalization Activation Layers: https://ift.tt/34ohoFi AutoML-Zero: https://ift.tt/2Q4IQCl Searching for Activation Functions (SWISH): https://ift.tt/2gYfK5N BatchNorm: https://ift.tt/2c7r2m1 StyleGAN2: https://ift.tt/325Ino8 GauGAN (SPADE): https://ift.tt/2CsbLsZ Evaluation of ReLU: https://ift.tt/2nIWK0K

This AI Makes "Audio Deepfakes"

❤️ Check out Weights & Biases and sign up for a free demo here: https://ift.tt/2YuG7Yf Their blog post on #deepfakes is available here: https://ift.tt/3c1Oiyb 📝 The paper "Neural Voice Puppetry: Audio-driven Facial Reenactment" and its online demo are available here: Paper: https://ift.tt/2snYcc1 Demo - **Update: seems to have been disabled in the meantime, apologies!** : https://ift.tt/34nY0s1 ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://ift.tt/2icTBUb - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. https://ift.tt/2icTBUb Meet and discuss your ideas with other Fellow Scholars on the Two Minute Papers Discord: https://ift.tt/2TnVBd3 Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/karoly_zsolnai Web: https://ift.tt/1NwkG9m

[Drama] Who invented Contrast Sets?

Funny Twitter spat between researchers arguing who was the first to invent an idea that has probably been around since 1990 :D References: https://ift.tt/2UQnX0m https://twitter.com/nlpmattg/status/1247326213296672768 https://ift.tt/2xe9Hph https://twitter.com/zacharylipton/status/1247357810410762240 https://twitter.com/nlpmattg/status/1247373386839252992 https://twitter.com/zacharylipton/status/1247383141075083267 Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Answering your latest TensorFlow questions! #AskTensorFlow

Laurence Moroney and Jason Mayes team up to answer your #AskTensorFlow questions on today’s episode. Remember to use #AskTensorFlow to have your questions answered in a future episode! 0:24 - How do we extend Keras APIs, model subclassing and generally improve the ease of use? 1:42 - What about web developers who are new to AI, does the version 2.0 help them get started? 4:23 - Is there any TensorFlow.js transfer learning example for object detection? 6:06 - Does TensorFlow leverage Metal when running on iOS? 7:20 - What’s the best way high school students can better engage with TensorFlow and it’s different components/submodules for their projects and work? 8:50 - What are TF Records, why are they needed for input? Resources mentioned in this episode: Teachable Machine → https://goo.gle/3bSCzCi TensorFlow.js Website → https://goo.gle/2XLhMe0 Watch more episodes of #AskTensorFlow → https://goo.gle/AskTensorFlow Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

Evaluating NLP Models via Contrast Sets

Current NLP models are often "cheating" on supervised learning tasks by exploiting correlations that arise from the particularities of the dataset. Therefore they often fail to learn the original intent of the dataset creators. This paper argues that NLP models should be evaluated on Contrast Sets, which are hand-crafted perturbations by the dataset authors that capture their intent in a meaningful way. https://ift.tt/2UQnX0m Abstract: Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities. We propose a new annotation paradigm for NLP that helps to close systematic gaps in the test data. In particular, after a dataset is constructed, we recommend that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets. Contrast sets provide a local view of a model's decision boundary, which can be used to more accurately evaluate a model's true linguistic capabilities. We demonstrate the efficacy of contrast sets by creating them for 10 diverse NLP datasets (e.g., DROP reading comprehension, UD parsing, IMDb sentiment analysis). Although our contrast sets are not explicitly adversarial, model performance is significantly lower on them than on the original test sets---up to 25\% in some cases. We release our contrast sets as new evaluation benchmarks and encourage future dataset construction efforts to follow similar annotation processes. Authors: Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Designing Network Design Spaces

This paper explores a really interesting paper to optimize the design space of a neural architecture search! This design space is responsible for the set of all possible networks that can be found in the search. By limiting this space, we save computation and produce better networks! Thanks for watching, please subscribe! Paper Links: Designing Network Design Spaces: https://ift.tt/3dLiE9X AlexNet: https://ift.tt/SHkH4l ResNet: https://ift.tt/2dQy6SG DenseNet: https://ift.tt/2tut9qL The Evolved Transformer: https://ift.tt/2IAdYFw Hierarchical Neural Architecture Search: https://ift.tt/2AA2JIh AutoML-Zero: https://ift.tt/2Q4IQCl POET: https://ift.tt/2J4X81K

POET: Endlessly Generating Increasingly Complex and Diverse Learning Environments and Solutions

From the makers of Go-Explore, POET is a mixture of ideas from novelty search, evolutionary methods, open-ended learning and curriculum learning. https://ift.tt/34gTiMT Abstract: While the history of machine learning so far largely encompasses a series of problems posed by researchers and algorithms that learn their solutions, an important question is whether the problems themselves can be generated by the algorithm at the same time as they are being solved. Such a process would in effect build its own diverse and expanding curricula, and the solutions to problems at various stages would become stepping stones towards solving even more challenging problems later in the process. The Paired Open-Ended Trailblazer (POET) algorithm introduced in this paper does just that: it pairs the generation of environmental challenges and the optimization of agents to solve those challenges. It simultaneously explores many different paths through the space of possible problems and solutions and, critically, allows these stepping-stone solutions to transfer between problems if better, catalyzing innovation. The term open-ended signifies the intriguing potential for algorithms like POET to continue to create novel and increasingly complex capabilities without bound. Our results show that POET produces a diverse range of sophisticated behaviors that solve a wide range of environmental challenges, many of which cannot be solved by direct optimization alone, or even through a direct-path curriculum-building control algorithm introduced to highlight the critical role of open-endedness in solving ambitious challenges. The ability to transfer solutions from one environment to another proves essential to unlocking the full potential of the system as a whole, demonstrating the unpredictable nature of fortuitous stepping stones. We hope that POET will inspire a new push towards open-ended discovery across many domains, where algorithms like POET can blaze a trail through their interesting possible manifestations and solutions. Authors: Rui Wang, Joel Lehman, Jeff Clune, Kenneth O. Stanley Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Muscle Simulation...Now In Real Time! 💪

❤️ Check out Lambda here and sign up for their GPU Cloud: https://ift.tt/35NkCT7 📝 The paper "VIPER: Volume Invariant Position-based Elastic Rods" is available here: https://ift.tt/39K4hzv https://ift.tt/2RaSSlT ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://ift.tt/2icTBUb - https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join  🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Alex Haro, Alex Paden, Andrew Melnychuk, Angelos Evripiotis, Anthony Vdovitchenko, Benji Rabhan, Brian Gilman, Bryan Learn, Daniel Hasegan, Dennis Abts, Eric Haddad, Eric Martel, Evan Breznyik, Geronimo Moralez, James Watt, Javier Bustamante, Kaiesh Vohra, Kasia Hayden, Kjartan Olason, Levente Szabo, Lorin Atzberger, Lukas Biewald, Marcin Dukaczewski, Marten Rauschenberg, Maurits van Mastrigt, Michael Albrecht, Michael Jensen, Nader Shakerin, Owen Campbell-Moore, Owen Skarpness, Raul Araújo da Silva, Rob Rowe, Robin Graham, Ryan Monsurate, Shawn Azman, Steef, Steve Messina, Sunil Kim, Taras Bobrovytsky, Thomas Krcmar, Torsten Reil, Tybie Fitzhugh. https://ift.tt/2icTBUb Károly Zsolnai-Fehér's links: Instagram: https://ift.tt/2KBCNkT Twitter: https://twitter.com/karoly_zsolnai Web: https://ift.tt/1NwkG9m

Dream to Control: Learning Behaviors by Latent Imagination

Dreamer is a new RL agent by DeepMind that learns a continuous control task through forward-imagination in latent space. https://ift.tt/2qiWpUW Abstract: Learned world models summarize an agent's experience to facilitate learning complex behaviors. While learning world models from high-dimensional sensory inputs is becoming feasible through deep learning, there are many potential ways for deriving behaviors from them. We present Dreamer, a reinforcement learning agent that solves long-horizon tasks from images purely by latent imagination. We efficiently learn behaviors by propagating analytic gradients of learned state values back through trajectories imagined in the compact state space of a learned world model. On 20 challenging visual control tasks, Dreamer exceeds existing approaches in data-efficiency, computation time, and final performance. Authors: Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Inside TensorFlow: tf.data + tf.distribute

In this episode of Inside TensorFlow, Software Engineer Jiri Simsa gives us the best practices for tf.data and tf.distribute. Let us know what you think about this presentation in the comments below! Links: tf.data: Build TensorFlow input pipelines → https://goo.gle/2VTnnjk Better performance with the tf.data API → https://goo.gle/38wyKAy Distributed training with TensorFlow → https://goo.gle/39wMWdY Watch more from Inside TensorFlow Playlist → https://goo.gle/31Ge5GF Subscribe to the TensorFlow channel → https://goo.gle/TensorFlow

Can we Contain Covid-19 without Locking-down the Economy?

My thoughts on the let-the-young-get-infected argument. https://ift.tt/2wBjdmc Abstract: In this article, we present an analysis of a risk-based selective quarantine model where the population is divided into low and high-risk groups. The high-risk group is quarantined until the low-risk group achieves herd-immunity. We tackle the question of whether this model is safe, in the sense that the health system can contain the number of low-risk people that require severe ICU care (such as life support systems). Authors: Shai Shalev-Shwartz, Amnon Shashua Links: YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher BitChute: https://ift.tt/38iX6OV Minds: https://ift.tt/37igBpB

Accelerate models with TFLite Delegates (TF Dev Summit '20)

Delegates enable TensorFlow Lite to run relevant parts of Neural Networks on other executors. In this talk, we give a brief overview of delegation and the available delegates in TFLite. We will also go over relevant tooling to assist users in choosing the right delegate and pointers for writing their own delegate. Speaker: Sachin Joglekar - Software Engineer Resource: TensorFlow Lite delegates → https://goo.gle/2PSS0S2 Watch all TensorFlow Dev Summit 2020 sessions → https://goo.gle/TFDS20 Subscribe to the TensorFlow YouTube channel → https://goo.gle/TensorFlow