There is much debate around where in the hype cycle we are with Artificial Intelligence, especially with the current focus on large language models (LLMs). LLMs are only a sub-set of AI, and not the whole story, but they are a critical turning point in AI, make no mistake.
The semantic and heurstical logic capability of LLMs cannot be denied, in fact this logic ability has allowed pioneer models such as GPT-4 to outperform humans on significant congintive benchmarks, such as the MMLU (Massive Multitask Language Understanding).
This benchmark is designed to measure knowledge across 57 subject areas and is similar to how we evaluate humans cognitive ability in specific domains. Evaluated human non-specialists on the MMLU benchmark is 34.5%.
Expert-level measured human performance averaged at 89.8% in one of these subject areas. This makes GPT-4 at 86.4% at close to professional expert level across 57 professional & academic subjects.
Above are a raft of congitive tests that OpenAI use to measure their models cognitive ability, and when GPT-4 is given an IQ test, we find that it performs at genius level.
Considering that the GPT 4 model used was operating with an 8,000 token (~6,000 word) context window, this is remarkable. We should also remain cognisant of the limitations that current LLMs still have, in that they are challenged by confabulations (hallucinations), scrutability and bias. These are more likely than not transitory challenges, given the level of investment and attention this technology is undergoing.
Regardless in my opinion, the hype is warranted.
Now the fear.
Are we as a society able to re-organise quickly enough to a world where the price of cognitive labour drops to almost zero? The kinds of unemployment that this technology can, will and is, creating could, and probably should, warrant a serious rethink and restructure of what it means to be a capitalist competition uber all alles society, where society is unable to compete with (for lack of a better word) it's tools.
There seems to be a spectrum between a utopic fully-automated, luxury space communism future and a dystopic 3 people own everything, while everyone else fights for scraps future. Reality will fall somewhere between these we should expect.
This is an amazing space to be in.
Right now we have the ability to partner with an emerging cognitive and transformative (is species maybe the right word?) technology. As the agency of AI is debated and increases we have the possibility to harness and augment our own capabilties in ways that we still don't really understand. Current AI agents can do a lot of the cognitive heavy lifting across almost all professional functions, and the nature of competition will dictate adoption.
What we are finding is that Polanyis Paradox is the limiting factor in AI agent capabilties. We know what we want them to do, but we are struggling to apply their intellectual traction to the highway of the real world in a meaningful way.
Polanyi’s Paradox is the term first coined by the Hungarian-British polymath Michael Polanyi when he discovered that: “We know more than we can tell”, i.e., the knowledge of how the world functions is, to a large extent, beyond our explicit understanding.
The next generation of multi-modal models could cross this chasm, which we will not know until it does. The nature of emergent properties is that they are unpredicatable as we don't really understand them yet.
The nuance of our societal professional roles, the 3 limitations of cognitive AI, and the availability of compute, will continue to exist for a long time still which should give a good degree of job security for the next decade at least.
Better job security is to partner with an AI agent, and maybe free more of your time?
About the Author
Rudy Nausch is a tech leader and consultant specializing in AI technologies.
I welcome discussions on how AI can help shape your organization's future and how we can develop AI Agents together.
Visit daimonic.ai for more.
Comments