David Rawlinson on One of our key projects is a memory system that can learn to associate distant cause & effect while only using local, immediate & unsupervised credit assignment. Our approach is called RSM - Recurrent Sparse Memory. We recently uploaded a preprint describing RSM. This is the first of several ...
Read More
Naoya Arakawa on Originally published in March 2019 in an electronic journal in Japanese Introduction The purpose of this essay is to survey the relationship between decision making and large-scale cerebral networks with regard to conscious access, a purported neural correlate of consciousness, and to provide c ...
Read More David Rawlinson on We've uploaded a new paper to arXiv presenting our algorithm for biologically-plausible learning of distant cause & effect using only local and immediate credit assignment. This is a big step for us - it ticks almost all our requirements for a general purpose representation. The training regi ...
Read More David Rawlinson on As AI/ML researchers, we have obviously pondered the risks of AI. We even wrote about it. But what might surprise you is the risks that keep AI/ML folk awake at night aren’t the ones you hear about in the media. We’re not worried about runaway “paperclip maximizers” or “skynet”-style mac ...
Read More
Abdelrahman Ahmed on Adaptive optimization methods, such as Adam and Adagrad, maintain some statistics over time about the variables and gradients (e.g. moments) which affect the learning rate. These statistics won’t be very accurate when working with sparse tensors, where most of its elements are zero or near zero. W ...
Read More David Rawlinson on We recently talked about Capsules networks and equivariances. NB: If you’re not familiar with Capsules networks, read this first. Our primary objective with Capsules networks is to exploit their enhanced generalization abilities. However, what we’ve found instead raises new questions about h ...
Read More
Gideon Kowadlo on We recently published 2 new ML/neuroscience research projects as part of the Request for Research (RFRs) projects, with WBAI. They’re fascinating topics that have arisen through the relationship with our advisor Elkhonon Goldberg from the Luria Neuroscience Institute.
Read More Abdelrahman Ahmed on It’s such a joy to be able to test an idea, go straight to the idea without wrestling with the tools. We recently developed an experimental setup which, so far, looks like it will do just that. I’m excited about it and hope it can help you too, so here it is. We’ll go through the why we created another framework, and how each module in the experiment setup works.
Read More David Rawlinson on We are exploring the nature of equivariance, a concept that is now closely associated with the capsules network architecture (see key papers Sabour et al, and Hinton et al). Machine learning representations that capture equivariance must learn the way that patterns in the input vary together, in a ...
Read More Gideon Kowadlo on Over the last few years, there have been several breakthroughs and exciting new research directions in Reinforcement Learning, Hippocampus Inspired Architectures, Attention and Few-Shot Learning. There has been a move towards multi-component, heterogeneous, stateful architectures, many guided by ide ...
Read More
