what do you think about the view about The sudden increase in power of large model is more like a phase transition than an emergence? I mean , of course there are some kinds of emergencies in concept, but because of the vague definitions of emergence, it seems phase transition is a better description and We might have to figure out what's breaking symmetry in the model, which seems like a more definite well question.
btw, I noticed you mentioned the work of Rosas and Erik Hole in your review, I wanna know what's your personal opinion with these existing theories about emergence.
I know they are not easy to answer here. In fact, the true purpose of my comment is to thank you for your updates! I'am always following closely because it is all interesting topic for me!
A comment about existing theories of emergence would require a book, ton say the least. In the next future I will post something about that.
Concerning phase transitions, it’s somehow easier to answer. The theory of phase transitions is well developed in physics, and it is rigorous enough allowing for testing new models.
I think that the word “emergence” is being misused nowadays, especially in the case of LLMs.
The transitions being observed for increasing size of such systems might lead to emergent abilities, but before that we should be able to esclude simpler explanations such as the purely statistical ones. The hype should be accompanied by scientifically sound explanations and validation for each claim.
Interesting topic and so rich references!
what do you think about the view about The sudden increase in power of large model is more like a phase transition than an emergence? I mean , of course there are some kinds of emergencies in concept, but because of the vague definitions of emergence, it seems phase transition is a better description and We might have to figure out what's breaking symmetry in the model, which seems like a more definite well question.
btw, I noticed you mentioned the work of Rosas and Erik Hole in your review, I wanna know what's your personal opinion with these existing theories about emergence.
I know they are not easy to answer here. In fact, the true purpose of my comment is to thank you for your updates! I'am always following closely because it is all interesting topic for me!
Thanks for the kind words and the interest!
A comment about existing theories of emergence would require a book, ton say the least. In the next future I will post something about that.
Concerning phase transitions, it’s somehow easier to answer. The theory of phase transitions is well developed in physics, and it is rigorous enough allowing for testing new models.
I think that the word “emergence” is being misused nowadays, especially in the case of LLMs.
The transitions being observed for increasing size of such systems might lead to emergent abilities, but before that we should be able to esclude simpler explanations such as the purely statistical ones. The hype should be accompanied by scientifically sound explanations and validation for each claim.