EMERGENCE.

Рет қаралды 18,828

Күн бұрын

Patreon: / mlst
Discord: / discord
Pod version: anchor.fm/machinelearningstre...
An emergent behaviour or emergent property can appear when a number of simple entities operate in an environment, forming more complex behaviours as a collective. If emergence happens over disparate size scales, then the reason is usually a causal relation across different scales. Weak emergence describes new properties arising in systems as a result of the low-level interactions, these might be interactions between components of the system or the components and their environment.
In our epic introduction we focus a lot on the concept of self-organisation, complex systems, cellular automata and strong vs weak emergence. In the main show we discuss this in detail with Dr. Daniele Grattarola and cover his recent NeurIPS paper on learning graph cellular automata.
Featuring;
Dr. Daniele Grattarola
Dr. Tim Scarfe
Dr. Keith Duggar
Prof. David Chalmers
Prof. Ken Stanley
Prof. Julian Togelius
Dr. Joscha Bach
David Ha
Dr. Pei Wang
[00:00:00] Special Edition Intro: Emergence and Cellula Automata
[00:49:02] Intro to Daniele and CAs
[00:57:23] Numerical analysis link with CA (PDEs)
[00:59:50] The representational dichotomy of discrete and continous at different scales
[01:05:21] Universal computation in CAs
[01:10:27] Computational irreduciblity
[01:16:33] Is the universe discrete?
[01:20:49] Emergence but with the same computational principle
[01:23:10] How do you formalise the emergent phenomenon
[01:25:44] Growing cellula automata
[01:33:53] Openeded and unbounded computation is required for this kind of behaviour
[01:37:31] Graph cellula automata
[01:43:40] Connection to protein folding
[01:46:24] Are CAs the best tool for the job?
[01:49:37] Where to go to find more information
Thanks to our patrons!
VIP patrons;
Alex McNamara
Ebonia Elliott-Lewis
All access patrons;
John Mitchell
Konstantinos Barmpas
And also;
Federico Rios
Francesco Puddu
Ian Finley
John
Lukas Segelmark
MarkMc
Paul Oreto
Philip Leggier
Timothy O’Hear
Łukasz Stafiniak
References;
(intro)
Strong and Weak Emergence [Chalmers]
consc.net/papers/emergence.pdf
Weak Emergence [Bedau]
people.reed.edu/~mab/papers/we...
Complexity - a guided tour [Melanie Mitchell]
www.amazon.co.uk/Complexity-G...
en.wikipedia.org/wiki/Emergen...
en.wikipedia.org/wiki/Reducti...
The Case for Strong Emergence [Sabine Hossenfelder]
philpapers.org/rec/HOSTCF-3
The Future of Artificial Intelligence is Self-Organizing and Self-Assembling [Sebastian Risi]
sebastianrisi.com/self_assemb...
Thanks to marknzed from Discord
(main show)
Learning Graph Cellular Automata [Grattarola]
arxiv.org/pdf/2110.14237.pdf
Grattarola article on GCA
danielegrattarola.github.io/p...
Dwarf Fortress
procedural-generation.tumblr....
Growing Neural Cellular Automata [Alexander Mordvintsev]
distill.pub/2020/growing-ca/
AlphaFold [Deepmind]
www.deepmind.com/research/hig...
Mandlebrot set
en.wikipedia.org/wiki/Mandelb...
FDM/PDE
en.wikipedia.org/wiki/Finite_...
en.wikipedia.org/wiki/Partial...
A new kind of science [Wolfram]
www.wolframscience.com/nks/
Wofram physics project
www.wolframphysics.org/
writings.stephenwolfram.com/2...
Game of life clock
codegolf.stackexchange.com/qu...
AlphaFold
www.nature.com/articles/s4158...
Michael Levin
as.tufts.edu/biology/people/f...
ase.tufts.edu/biology/labs/le...
www.growbyginkgo.com/2022/03/...
www.growbyginkgo.com/2020/06/...
Emergent garden
/ emergentgarden
/ max_romana
The edge of chaos
en.wikipedia.org/wiki/Edge_of...
Computation at the edge of chaos: Phase transitions and emergent computation [Langton]
www.sciencedirect.com/science...
Lenia
chakazul.github.io/lenia.html
Slackermanz
slackermanz.com/understanding...
Daniele doing a talk on his work
• Learning Graph Cellula...
Smarter podcast (Daniele’s pod!)
/ @smarterpodcast6806

Пікірлер: 79

@danielegrattarola2680 2 жыл бұрын

Thanks for having me on, I had a really great time chatting with you guys. The work you do with the channel is absolutely insane, such a high production value!!!

@granthawkins88 2 жыл бұрын

Your introductions are so spectacular. Really sets MLST apart. Thank you!

@mattbabik8417 2 жыл бұрын

One of my favorite topics as a junior neuroscientist. Finding simplicity in the unexplainable

@neuronwave 2 жыл бұрын

Heading off to the Santa Fe Institute in June to study and can't think of a better talk to listen to prior to going. Thank you.

@GBlunted 2 жыл бұрын

Wolfram has been on Lex Friedman's podcast 3 times in which they go into depth on CA and Wolfram's theory of everything, all of which are good to listen to to try and understand Wolfram's grasp of languages and various stuff he's into! 👍🏽

@ConnoisseurOfExistence Жыл бұрын

Very interesting! Need to come back to rewatch properly. Emergence can be boiled down to the very basics of even the mathematics set theory: if a set has no elements, will also have no properties at all; if you add 1 element, now the set has more properties; if you add 1 more element, new properties emerge; 3,4,5 elements... properties increase exponentially. 'Merely quantitative differences, beyond a certain point, pass into qualitative changes'...

@CristianGarcia Жыл бұрын

Love this! I was into Complex Systems before ML started booming and took over. Talks from the Santa Fe Institute are amazing.

@ianfinley89 2 жыл бұрын

Man what an incredible episode! I'm going to have to watch it a few more times because it is packed to the brim with information.

@MachineLearningStreetTalk 2 жыл бұрын

Thanks Ian! 😍

@LuigiSimoncini 8 ай бұрын

Awesome, thanks!

@federicodidio4891 2 жыл бұрын

Loved the attitude of Dr. Grattarola: modesty, curiosity and striving for semplicity over this-is-how-it-works and my-method-is-key.

@MachineLearningStreetTalk 2 жыл бұрын

Exactly!

@dr.mikeybee Жыл бұрын

Hearing about Sebastian Risi, I'm immediately reminded of spooky action at a distance. Contrary to the idea of localization where self-assembly is based solely on local interactions. How might a system with an object that has a property read by two separate functions operate? Now you have cellular automata but only when not in opposition to a map like DNA. Moreover, the map is evolved over generations.

@Bobby-bz8bk 2 жыл бұрын

So dope as always. Thank you!

@stevengill1736 Жыл бұрын

Absolutely fascinating! I'd wondered what happened to CAs since Life in the 1980s. So excited to see the new developments. David Bohm's implicate and explicate orders keep popping into mind, I wonder why? Thank you....

@sai4007 2 жыл бұрын

Top notch content!

@slawekddd Жыл бұрын

Thanks for a great interview

@patrickcooperai 2 жыл бұрын

Great video guys! Initially I found the idea of Chalmers viewing consciousness as strongly emergent to be counterintuitive at first, but when you parse in the role of supervenience it becomes clear his proposed dualism and emergence from the physical are not contradictory. I thought this distinction was well handled here.

@oncedidactic 2 жыл бұрын

Tons and tons of stuff in this amazing episode, thanks!! One thing I kept thinking of as it applies to all the topics in some sense, but did not quite come up- concept of Boltzmann brains and ergodicity. General observation and the random brain idea seem to point that real world systems are not truly ergodic and this is another mark against infinity in reality, more in the time dimension I suppose? Anyway the Boltzmann brain question really makes you wonder though if a lesser form of proto life would (must?) arise on the basis of large scale state space exploration via correct seeding somewhere in the system. This gets to guest’s comments on CA rules vs state encoding the behavior. Great stuff!!

@anonymoushandle196 2 жыл бұрын

Great video! This feels a bit uncanny, especially the work on neural cellular automata, since, through sheer coincidence, I was talking with someone yesterday about how the nodes in Hopfield networks can be thought about _like_ cellular automata.

@dragolov Жыл бұрын

Deep respect!

@hannesstark5024 2 жыл бұрын

Fantastic! Another episode that is really inspiring for digging deeper. Thank you! There is a typo at 28:42 I think.

@MachineLearningStreetTalk 2 жыл бұрын

Yes sorry about the editing mistake there

@michaelwangCH 2 жыл бұрын

Watch out, Dalle2 becomes imaginative and more human like. We are close to reach general intelligent. It is great work to understand human intelligence.

@spiritcrusha 2 жыл бұрын

Is Sabine implying that strong emergence exists? From I gathered, she is saying that strong emergence cannot be ruled out due to our current fundamental theories being more effective than fundamental.

@MachineLearningStreetTalk 2 жыл бұрын

Yes that's right

@danielaronoff6921 2 жыл бұрын

Hayek describes the program of the Scottish Enlightenment - which founded the modern disciplines of economics, legal and linguistic studies and evolutionary biology - as the study of emergent phenomena grounded in the basic units of analysis. In the humanities the point of view was summarized by Adam Ferguson’s phrase “ the result of human action but not of human design.”

@EricFontenelle Жыл бұрын

God I love this channel

@mobiusinversion 2 жыл бұрын

Are there any analogues to the continuum hypothesis when it comes to the informal notion of emergence “levels”?

@keithallpress9885 Жыл бұрын

At 12:25 the discussion cut short, but there is a connection between organisations and brains none-the-less.. The rules at the lower level in a company are a combination of fixed and adaptive, the employees have autonomy and will evolve a response, and it will involve middle management regulation plus employee initiative. By analogy the brain injury also has a response, not so much to top-down objectives but the analogy would be cognitive demand, where neighboring brain areas will pick up the demand, something we'll known as "plasticity". I can well imagine that this feature would evolve under selection pressure. If neural networks are to become biologically inspired then the language of description will need to evolve to embrace more organisational concepts.

@NullHand 10 ай бұрын

I think this is being done. But deep in the "Big Labs", not the pre-trained LLMs released to/on the public. We hear rumours of AI systems that contain specialized "subsystem" AIs in some form of network coms architecture. This would be a fair simulacrum of vertebrate brains.

@dr.mikeybee 2 жыл бұрын

If I build a radio receiver, but there are no radio waves, there is no emergent behavior. If there are radio waves, then sound emerges. It seems to me that if consciousness requires a metaphysical signal, we have strong emergence. If consciousness requires that we have some connection to an unknowable source, that's in every practical sense strong emergence. My intuition tells me that the unknowable is a vastly larger set than the knowable; so strong emergence seems entirely possible, but this is based on the limitations of our sensors and ability to understand, not on any shortcomings of the practices like logic and reasoning. Moreover, the phenomenon of being unknowable is functional not intrinsic; so what is unknowable to one species may be knowable to another. The question that stands like a 500 pound gorilla in the room is what good does this understanding provide? What questions do we need to ask when attempting the analysis of some phenomenon or another? How do we keep from going down unproductive rabbit holes? Otherwise, what is the prize at the end of this speculative quest?

@StormOrMelody 2 жыл бұрын

I think you all would really enjoy having Michael Levin on the show (one of the co-authors of the "Growing Neural Cellular Automata" paper). He has plenty of talks on youtube, and also recently gave a talk at ICLR in the workshop "From Cells to Societies: Collective Learning Across Scales" if you want to check out some of his work.

@MachineLearningStreetTalk 2 жыл бұрын

We will, thanks!

@asdf8asdf8asdf8asdf 4 ай бұрын

Need Stuart Kauffmann on show to talk on emergence and non-algorithmic evolution and the “adjacent possible”

@CharlesVanNoland 2 жыл бұрын

#ManyToManyMapping is the key.

@aldousd666 Жыл бұрын

What would you say stellar nuclear fusion is, vis a vis its reliance on quantum tunneling? I want to say weak, but our description of it is inexact. I don't think we can avoid having to use our ability to describe it as a factor somehow

@michaelwangCH 2 жыл бұрын

The topic of complex system theory is still unresearched area. Because complex systems are depending on initial condition, highly non-linear and stochastic. Therefore those system can not be simulated - the each outcome of the simulations will be different, if the search spaces are not restricted.

@dr.mikeybee 2 жыл бұрын

If the universe is essentially computational, what does it mean to be infinite? Does space extend to point b somewhere out in the tremendum? Perhaps not until it is computed. Go there, and it exists.

@parker9163 Жыл бұрын

I'm surprised Stephen Wolfram wasn't mentioned at all in this video... He coined the term: computational irreducibility

@MachineLearningStreetTalk Жыл бұрын

He was mentioned my friend! Check 33:10 Hopefully we will get him on the show soon to discuss his new book / work

@therainman7777 11 ай бұрын

He was mentioned. Try again.

@Iophiel 2 жыл бұрын

What's interesting is not that there is emergence in each 'layer', it is that we percieve 'layers' in the emergence at all...

@therainman7777 11 ай бұрын

No, it’s definitely interesting that there is emergence in each layer. Try again.

@dr.mikeybee Жыл бұрын

It's interesting to see that Joscha is a logical positivist.

@xaxabogbart Жыл бұрын

Are you able to share when the discussion with Joscha Bach is scheduled to be released? Thank you.

@MachineLearningStreetTalk Жыл бұрын

realistically not for another couple of months

@xaxabogbart Жыл бұрын

@@MachineLearningStreetTalk Thank you.

@_ARCATEC_ 2 жыл бұрын

💓

@michaelwangCH 2 жыл бұрын

It is also not true that we are not capable to change structure of atoms - we as human need to build tools. The tools we built reflect our brain and thoughts - we are the tools what we built.

@dr.mikeybee 2 жыл бұрын

If you look at self-organizing systems, they do use a plan, but it has been coded in the most compressed format, and therefore, it is hidden. Keep in mind that an "understanding" of the physics of the entire system are a part of the encoding. That is to say, the encoding would be meaningless without the laws of physics and the attributes of the various substrates.

@therainman7777 11 ай бұрын

We know.

@spandexleotard 2 жыл бұрын

If the ants programming only makes sense in the context of environment and colony, is the ants programming strongly emergent from the colony?

@Dante3085 Жыл бұрын

I am still starved for that Joscha Bach episode XD

@MachineLearningStreetTalk Жыл бұрын

Sorry for the delay, me and Keith are building a startup company and it's sapping all our time. I hope we can focus more on MLST soon

@swayson5208 2 жыл бұрын

Coherence is a fractal.

@therainman7777 11 ай бұрын

Nonsense.

@michaelwangCH 2 жыл бұрын

Please explain why does the hyper-specialization lead to generalization?

@oncedidactic 2 жыл бұрын

Counterintuitive at first, but idea is that generalization (if you unpack what it really means) requires you handle novelty, and you cannot train for novelty, you need a toolbox that may let you handle aspects of the novel state. Implication of modular specialized functions, not optimized for specific synergy or task necessarily, so that you have open ended correspondence with novel states and hence they are tractable, hence you are “generalized”.

@michaelwangCH 2 жыл бұрын

An expert system can only do one thing. I. e. a chess computer can only play chess - those systems are hyper-specialized in specific tasks. How can those systems generalize, if they are trained for deterministic tasks. How can a hyper-specialized system generalize where the search space is infinite? How can those systems identify the stepping stone lead to generalization.

@gerdmoellmann Жыл бұрын

The background music is unnerving

@CandidDate 2 жыл бұрын

Thoughts are CA. Conversations are CA. CA is cellular automata, whereby one level of complexity is defined by a discretely related similar level of complexity. DNA is a CA. The turning of food into energy is a CA. The big bang was the instantiation of the biggest CA. Need I say more?

@therainman7777 11 ай бұрын

You shouldn’t say more, because everything you just said is total nonsense. Congrats.

@Iophiel 2 жыл бұрын

The OBSERVABLE universe is OBVIOUSLY not continuous, as nobody has ever seen the back of their own head.

@therainman7777 11 ай бұрын

Wow. You have no idea what these terms mean or what you’re talking about. What a hilarious comment 😂

@Iophiel 11 ай бұрын

@@therainman7777 Yeh, right, next thing you'll try and tell me is that the OBSERVABLE universe is ISOTROPIC.

@connorkapooh2002 2 ай бұрын

were you having a giggle at the accent? lol

@sedenions 2 жыл бұрын

Pei next :)

@MachineLearningStreetTalk 2 жыл бұрын

Pei is the one after next :) It will be worth it I promise!

@Earthgazer Жыл бұрын

Can't help but feel deeply flattered that, not only did I arrive at the stance that infinity is cognitively meaningless at the age of 18 or so, but there are professors in their 50's that still don't even understand the claim

@therainman7777 11 ай бұрын

One of the most masturbatory comments I’ve ever seen, congrats 😂