RE•WORK
Yoshua Bengio, considered one of the ‘Godfathers of Artificial Intelligence’ discusses Recurrent independent mechanisms, sample complexity, end-to-end adaptation, multivariate categorical MLP conditionals and more.
When summarising his talk, Professor Bengio gave three key points to keep in mind when ‘looking forward’
– We must build a world model which meta-learns causal effects in abstract space of causal variables. This requires a necessity to quickly adapt to change and generalize out-of-distribution by sparsely recombining modules
– The necessity to acquire knowledge and encourage exploratory behaviour
– The need to bridge the gap between the aforementioned system 1 and system 2 ways of thinking, with old neural networks and consciousness reasoning taken into account
Source
I am amazed to hear how he defined the definition of attention. That alone has potential to open up a new stage in AI.
Very informative talk. Nice job.
Thank you Yoshua for an interesting and very informative presentation; I was amazed to find how much your comprehensive ideas for making progress in Deep Learning resonates with an intelligent network simulation model for transportation where the network is the world model representing the actual (abstract) space; much like the neo-cortex where thoughts and reasoning is considered to happen (Kurzweil); William James talks about streams of thoughts (flow of electrons/moving agents) strength of flow is the capacity of synapses in the embedded network of neurons. A neuron can be as complex as a transportation network (David Eagleman). Learning from the integral of flow (path) heading to an End equals probability to the End point given a choice (Stephen Hawking); relates to synapse plasticity; composed by combination of prior probabilities and causal parameters as factors (Bayesian) minimizing errors. Note, a neuron directed towards an end is a construction which creates all indifferent paths simultaneously given the learned parameters by use of the implicate order; pure bagprop which in a way illus trait what intelligens is. These ideas lead to a global transport mind which may create the future in a rational way for the benefit of all travellers. Best wishes Niels Hoffmann PaxAssign Limited, Skotland.
I need a causation-based one!
I agree with a lot of what Yoshua is explaining. Even moving away from vectors and into using sets that implies attention. Great talk!