GAI before AGI?
I am drafting a post to introduce something I have built. I wrote the title
Asynchronous. Inference. Primitives.
and I knew, before a landing page there has to be a "Why and How did I even land here?" page.
Let me take you back to the past where I encountered each of those words. Each of these time periods has a role in forming my thoughts around what to build and more precisely what not to build.
Back to the Future
It is mid 2024. AI is democratized, thanks to both Chat & GPT in the ChatGPT app. GPT is the brain behind serving useful AI to the consumers in real time. And Chat is of course chat. The default surface, easy, convenient and familiar. So AI chatbots are proliferating exponentially. There is almost no obvious reason to look beyond chat interfaces.
Except. I have two.
The first one is 2. The other has just turned a month old.
But I am not looking for alternatives to Chat interfaces simply because I am a new dad or I have less time or I do not like chatting. The real issue is more layered.
I am making the most out of my paternity leave. Some days a 1 hour window becomes available at 4am in the morning. Then there are days when a supposedly 3 hour downtime is done in 45 minutes. Other days I barely touch a device except for a few minutes in between.
So the shape and the size of my time is changing. It is unstable, uncertain and uneven.
All this stop and go got me thinking beyond the default way of using AI.
There surely seems to be room for a real life AI alongside the real time AI.
How can I always come back to a useful artifact? How can I hit flow state even with frequent stops? Without digging through tons of siloed chat histories to fast track myself back into a flow state. Can we have something not session based. One that adapts to the asymmetries of life. Some alternatives, more options so we are not always defaulting to synchronous conversational interface for everything AI.
Maybe something asynchronous?
Back to the Future Part II
It is somewhere near the end of 2024. Context window sizes are steadily increasing. I am relying on persistent chats for multiple Projects spread across my devices running a variety of models.
And with the multimodal models becoming mainstream, images mostly screenshots started dominating my prompts. I use them to prepare for interview calls, upskill and create connections with what I understand already, among many other things.
But now the models are turning into a commodity. Between the aggressive release cycles, all the options across open, free, local, thinking and reasoning variants, and my ever increasing number of screenshots, I am back to looking beyond the status quo again.
So with more screenshots and more capable models two things happened.
- I was running smaller sprints with a set of screenshots to achieve a specific outcome. And sometimes these had to be repeated. So this workflow based inference was unnecessarily hard inside chats.
- And then imagine the retrieval, repurposing and remixing. Maybe I have an interview call tomorrow and I want a bunch of screenshots turned into flashcards to help me prepare. Maybe there are two sets of screenshots which can be consolidated into one solid learning block for me to upskill faster.
So I started to believe such tasks are easier if I could somehow decouple prompts and inference. Separate submission from execution? A solution where I had finer control over submission of prompts in a batch and probably retrieve them as one solid artifact? Push some jobs and pull results whenever time becomes available? Without scrolling or getting in and out of chat windows? Where artifacts, results and inference are easy to find, repurpose and retain?
Unlike last time when I chose changing diapers over interfaces, this time I went and did something about all of the above opinions.
I am building a small utility with the tech stack I know and understand. I want it to be the simplest MVP decoupling inference from the chat window.
Back to the Future Part III
This is right around Summer 2025. Let me first acknowledge the elephant, I mean the Agent in the room. Chances are you are one or you might be using one to summarize this post and all while running a dozen others on your phones and terminals and browsers. Agents cracked writing code first and have not stopped.
And if you are wondering what happened to that little utility I was building around Christmas 2024, it did not fly very far. But building it got me deeper into small models and non conversational interfaces for local AI.
The models were still a problem though. There were not many reliable, performant small models that ran well on my 2017 MacBook Pro with Intel silicon.
But now that seems to be changing. There has been tremendous improvement in the performance and availability of open, free, small models doing a lot of exciting things.
So you would think with ease of building agents and all these new small models one could just sit in a coffee shop and literally prompt oneself out of this corner.
As I am onboarding myself with all these building blocks, primarily the libraries, tools, frameworks and SDKs, I am realizing that they are too big to be building blocks.
Because I have a specific problem I want solved involving screenshots and my laptop and small models, I am hoping to find something smaller, modular with the ability to grow with my solution.
Maybe a set of primitives?
Small enough to understand. Small enough to swap out. Small enough to keep only the parts I need.
The Future
Ok so we are in the Now(ish!), somewhere from the beginning of Summer 2025 till about whenever this post gets published.
Here is a hypothesis given I started local AI fairly early, on what is now ancient hardware. This was when local AI was not even close to being considered a default daily driver. Small models were not as performant.
So I did not roll out of my cloud accounts into my laptop just to give small models a shot. Rather I was waiting there first with a well defined problem statement. Waiting for all the help I could get with the growing number of tools and libraries and frameworks within the local AI infra ecosystem.
This order has proven to be important. Because it made it easy for me to see the bloat & bundling early on than in retrospect.
All the dependencies and packages and solutions were mostly trying to accomodate the infra around chatting features and coding tasks, more context so you can chat more, more memory and harness engineering to reduce hallucination. Much of these building blocks assume a series of calls to an always-on server running a session underneath. All these make these building blocks too big to begin with and too complex an abstraction to build on top of.
And because there are not many alternatives to these server shaped AI interfaces and primitives every hardware, every app, every problem and the fixes for it all of them inherit this baggage. With little freedom to opt out of defaults like GUI or context, memory related dependencies and packages.
Most Local AI infrastructure seems like a hand me down from the Cloud.
I understand the leverage of context and persistent conversation providing a sense of infinite intelligence but I have problems with that being the default and the lack of options.
If I have a model locally downloaded and once I am sure of the prompts and the inference results there has to be another option than running it behind a localhost just so that I can with it.
The Pivot
That is where I arrived at from Life and now I am trying to address it with asynchronous inference primitives, so a work order can reach a model without a chat window. And once you can do it reliably, repeatedly and durably, the intelligence becomes GA. Generally Available.
Or in short GAI.
And I am not sure GAI comes before or after AGI. I am just building for
GAI alongside AGI.