One of our key projects is a memory system that can learn to associate distant cause & effect while only using local, immediate & unsupervised… Read More »Learning partially-observable higher-order sequences using local and immediate credit assignment
We’ve uploaded a new paper to arXiv presenting our algorithm for biologically-plausible learning of distant cause & effect using only local and immediate credit assignment.… Read More »Learning distant cause and effect using only local and immediate credit assignment
Adaptive optimization methods, such as Adam and Adagrad, maintain some statistics over time about the variables and gradients (e.g. moments) which affect the learning rate.… Read More »Optimization using Adam on Sparse Tensors
We recently talked about Capsules networks and equivariances. NB: If you’re not familiar with Capsules networks, read this first. Our primary objective with Capsules networks is… Read More »Predictive Capsules Networks – Research update
It’s such a joy to be able to test an idea, go straight to the idea without wrestling with the tools. We recently developed an experimental setup which, so far, looks like it will do just that. I’m excited about it and hope it can help you too, so here it is. We’ll go through the why we created another framework, and how each module in the experiment setup works.
This is the second part of our comparison between convolutional competitive learning and convolutional or fully-connected sparse autoencoders. To understand our motivation for this comparison,… Read More »Convolutional Competitive Learning vs. Sparse Autoencoders (2/2)
Competitive learning is a branch of unsupervised learning that was popular a long, long time ago in the 1990s. Older readers may remember – the days… Read More »Convolutional Competitive Learning vs. Sparse Autoencoders (1/2)
We’ve just uploaded a spin-off research paper to arXiv titled “Sparse Unsupervised Capsules Generalize Better”. So what’s it all about? Capsules Networks You may have… Read More »Sparse Unsupervised Capsules Generalize Better
The dataset is an integral part of an ML engineer’s toolkit. We recently compiled useful information about a range of these well known datasets. It’s all in one place, and hopefully useful to others as well.