A few weeks ago, we asked our Substack community which AI questions deserve more attention. We wanted to hear from people who see AI differently, whether because of an unpopular opinion, an experience that changed their view or a question they felt nobody was asking. The replies went in directions we didn’t expect. Someone brought up Dune. Someone else proposed AI systems run by local communities, possibly through public libraries. Writers joined in, along with professionals thinking about what AI means for their industries and people interested in machine consciousness.
At first these looked like separate conversations. Reading them together, we kept coming back to control: who holds it, and how much of our own independence we give up as we come to depend on AI for decisions, work and relationships. The contributors didn’t always agree, and we’ve tried to keep their disagreements intact.
The Arrakis problem: depending on AI someone else controls
Amer Altaf approached the question through Denis Villeneuve’s Dune. He compared House Atreides, which receives Arrakis from the Emperor and then loses it in an attack the Emperor secretly supports, to countries that depend on AI controlled elsewhere. Many countries outside the United States and China, he argued, have access to AI on terms set by others. When we asked how much dependence is too much, he offered a practical test: could a country or business keep operating if the technology were switched off the next day?
That test works well for organizations too. A company might build external AI models into its core workflows, train its staff on them and reorganize around what they can do. If the provider then changed its terms or withdrew access, the company’s ability to adapt would depend on how much of that capability it actually controls. Countries face the same problem when they invest heavily in AI adoption while relying on models and infrastructure built abroad. Sovereignty probably doesn’t mean every country building everything itself. Amer’s example does show that access to a technology and control over your dependence on it are two separate things.
Access to AI versus control over it
Couture & Code shared an essay, The Label Sewn Inside, that raised a related point for creators. In our exchange about it, we talked about the difference between owning your work and controlling what happens to it on a platform. Keeping original copies and understanding privacy settings help, but creators still work under rules set by the platform’s owners, and individual precautions only go so far when the important decisions are made elsewhere.
The skills we might not notice we’re losing
Mahjabeen brought the question down to daily work. She described using AI to finish in ten minutes a task that used to take her an hour. The speed was exciting at first. Over time, she felt more and more dependent on the tool, found herself struggling with the task and decided to go back to doing it herself.
One person’s experience doesn’t mean everyone who uses AI will lose the ability to work without it. It does show why the trade-off is easy to miss: the time saved is obvious right away, while the effect on learning shows up slowly, if at all.
Take a hypothetical student learning to write. AI can help them structure an argument or explain an unfamiliar concept, and that can support learning. If they start relying on it for every difficult paragraph, though, they get fewer chances to develop their own sense of what makes an argument convincing. Developers face a version of the same problem: AI can produce working code in seconds, but understanding why it works and spotting what could go wrong still takes expertise.
Some tasks don’t need to stay difficult just because people once did them by hand. The useful distinction is between AI that removes unnecessary effort and AI that removes the practice through which people build a skill.
AI doesn’t need to be conscious to be dangerous
Other contributors were concerned with a different kind of control: how AI systems themselves behave.
In his essay To Understand AI Is to Anthropomorphize It, Karlo Vrancic argues that comparisons with human behavior can help us understand AI, and he considers whether future AI agents might develop more persistent goals. Anthropomorphism is usually treated as something to avoid, and we think Karlo is right that human comparisons can be a good starting point for understanding an unfamiliar system. Trouble starts when the comparison is pushed further than the evidence allows. A system can act as though it’s pursuing a goal without wanting anything, and it can produce responses that resemble emotion without that proving it feels anything.
Mike Aguilar’s article Sentient AI? examines the difference between possessing information and having subjective experience. In a later reply, he stressed keeping humans involved in decisions and in governing AI systems. We agree, especially where AI decisions carry serious consequences. We’d add that a system doesn’t need anything like consciousness to be dangerous. An AI agent working toward a poorly defined objective, or acting on bad information, can cause harm without any intention behind it.
These perspectives don’t settle the debate about machine consciousness, but they raise an important question about control. As AI systems become capable of making more decisions and taking more actions independently, how do we ensure that the people relying on them still understand their limitations and remain responsible for the outcomes?
Will AI bring us closer together?
The sharpest disagreement in the discussion was between nahar emet and Lonica Smith . Nahar argued that better human coordination could be one of AI’s most overlooked effects, since AI can work across language barriers and take on the organizing that once made large collaborations expensive. Lonica saw the opposite happening. In her view, AI is already reducing coordination and increasing isolation, and its risks for intellectual property and trade secrets give people even less reason to share. In a later reply, nahar acknowledged that current uses of AI can isolate people, while maintaining that its potential for coordination is real.
We think both are right, about different moments. Nahar’s case rests on what AI makes possible, and Lonica’s on how people use it now. Much of today’s use is individual: someone asks an assistant a question they might once have asked a colleague. That’s a reasonable trade on any given day, but those small exchanges are how colleagues learn from each other, and an assistant that answers everything removes many of the occasions for them. The coordination Nahar describes is unlikely to arrive as a side effect of individual use. It would take tools designed for groups, and people choosing to use AI to reach others instead of to avoid asking them.
Peter Eidos’ two-word contribution, “Cognitive Symbiosis,” names the outcome that would answer both sides. He didn’t elaborate, but the phrase suggests a relationship in which AI strengthens what people can do, including, we’d add, their ability to work with each other. Getting there will take deliberate choices, because the path of least resistance leads toward the isolation Lonica describes.
What are we willing to give away?
The perspectives we received don’t add up to a single conclusion. Some contributors want to build alternatives to today’s centralized AI infrastructure, while others focus on learning, governance, ethics or the consequences of how AI is used now. What connects many of them, in our reading, is how much influence people keep over the systems they increasingly rely on.
AI progress is usually measured by capability: how intelligent the latest model is and how much it can automate. Those measures leave out whether the people using the technology become more independent or less, and a more capable tool can push in either direction.
Amer’s test applies at a personal level too. Instead of asking whether a business could function if a technology were switched off, we can ask whether we could still make the decision or form the opinion without AI doing it for us. The answer doesn’t need to be yes every time, but it’s worth noticing when it starts to be no.
Over to you
This article grew out of a conversation with our Substack community, and we’d like to thank everyone who shared their experiences and ideas. We’ve connected these contributions through our own interpretation, so not every contributor will agree with the conclusions we’ve drawn, and we’d like to hear where they don’t.
What’s one thing you would never want to delegate entirely to AI? Tell us in the comments. We’d like to keep this conversation going and draw on your perspectives in future articles.
Amer’s switch-off test is the question I think every company should be asking. You pay for the model, build your workflows on it, maybe let people go, and then what? What if the price triples, or access is restricted? It has happened: in June, Anthropic suspended two of its models for almost three weeks under US export controls. I resonate because I asked a number of similar questions a coupled of days ago in my post.