Recently, I decided to build something end to end.
Not a prototype. Not a POC. An actual working system.
For most of my career, this kind of work was distributed. Different people owned different parts. Data moved through layers. Assumptions were discussed, sometimes challenged, sometimes left undocumented, but at least visible across the system.
That structure was not perfect.
Pipelines had hidden logic.
Interpretations carried bias.
Decisions were not always fully documented.
But knowledge was spread out.
This time, I built it alone.
With AI and cloud-based environments, it is now possible to move across the entire stack. Data ingestion, transformation, modelling, validation, and deployment can sit within one workflow, driven by a single person.
At first, it feels efficient.
Fewer dependencies.
Faster iteration.
Less coordination.
But the experience is different.
Not because the system does not work. It does.
But because the way it works becomes harder to separate from the person building it.
The logic is still there.
The assumptions are still there.
The decisions are still being made.
But they are compressed.
What used to be spread across roles now sits closer together. Some of it is captured in code. Some of it in prompts. Some of it in choices that are made along the way and not always revisited.
This is not entirely new.
There have always been systems that only one person fully understood. The difference now is how much can be built that way, and how quickly that complexity can accumulate.
A dataset is no longer just a dataset. It reflects a sequence of decisions about joins, filters, assumptions, and edge cases. A model is shaped not only by data, but by how it was iterated, tested, and adjusted.
The system works, but understanding how it works requires retracing those steps.
I suppose..
What changes here is not just who builds the system, but where the complexity sits. In distributed environments, complexity is spread across people and processes. In end-to-end workflows, it becomes more concentrated.
This does not automatically make the system worse. In some cases, it makes it faster and more coherent. For smaller or self-contained use cases, that concentration may not matter at all.
But as the system becomes more connected to other processes, the visibility of that complexity starts to matter more. Not because others cannot work on it, but because the path from input to output is less obvious without reconstructing the decisions behind it.
This is where the question begins to shift.
Not whether one person can build everything.
But what needs to be visible for something to be continued by someone else.
Because building end to end is now straightforward.
What is less clear is how much of that thinking needs to be externalized for the system to exist beyond the person who built it.
No comments:
Post a Comment