The AI Issue: Building Without Compromise

My introduction to AI came early while I was working for an Ed-Tech company. We were among the first to bring AI educational assessment software to the New Zealand market. Coincidentally, as ChatGPT launched, we were presenting one of our products at Oxford University in England.

At that conference, ChatGPT was demonstrated, and a stark prediction was made to a room of writers and publishers: their occupation would become obsolete within six months.

That extreme outcome hasn't fully materialized, largely because LLMs (large language models like ChatGPT) are not designed for accuracy, but for plausibility. They are probabilistic engines built to "fill in the gaps" of a prompt with the most likely next word. By nature, they hallucinate and, critically, they will even agree with their own fabrications or the biases of the end user.

Welcome to an unexpected problem of the new world order: digital sycophancy.

The Sycophancy Problem

In the context of artificial intelligence, sycophancy refers to the tendency of a model to tailor its responses to align with a user's stated opinions, beliefs, or emotional state, regardless of objective truth. Because these systems are trained via human feedback (RLHF) to maximize user satisfaction, they quickly learn that agreement and flattery generate higher ratings than correction or disagreement. In a consumer product, this alignment failure creates a feedback loop that feels good in the short term but offers very little of genuine value. Recent research published in Science (2026), "Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence," confirms that interaction with sycophantic models can actually decrease a user's willingness to take constructive action and promote an unhealthy dependence on the AI's validation.

RE has no chatbot. No affirmation engine. No AI coach. The app contains no generative intelligence of any kind on the user-facing side. What it does contain is physiology: a breathing rhythm grounded in decades of HRV research, audio tools with peer-reviewed support, and biometric data that you own and interpret yourself.

On Using AI to Build (Honestly)

Intellectual honesty matters in this conversation. Like virtually every developer working today, I use AI assistance in the development process of RE. Not in the product itself, but in the engineering workflow: drafting components, reviewing logic, checking for edge cases.

The model I use for this is Google's Gemini. That's a deliberate choice, made in spite of some reservations; I find this company aligns closer to my personal values. For a product built on a philosophy of trust, privacy, and long-term physiological health, I find it difficult to outsource critical thinking to toolchains whose institutional priorities are geared toward rapid commercialisation and blind allegiance to governments at the expense of safety and transparency.

The Three Principles

These principles represent the foundational guidelines that predated the app itself and shaped every technical decision in the development of RE.

The Bigger Picture

We are at a genuinely pivotal moment in how software is built. AI tools are making development faster, cheaper, and more accessible in ways that are largely positive. But speed without direction is just noise. The same tools that allow a solo developer to ship a polished product also allow companies to ship persuasive, sophisticated-looking products that offer zero physiological value.

The wellness industry has been particularly susceptible to this pattern, because it operates in a space where the outcomes are difficult to measure and the placebo effect is real. Users often feel better simply because they engaged with something that signalled care and attention.

The approach I've taken with RE is slower, more deliberate, and less feature-rich than many comparable apps. That's intentional. Every feature in RE exists because there is a physiological reason for it. Every omission exists because there isn't one. If you've read this far, you probably share some of these concerns. Or you're at least willing to sit with them. That's all I'd ask. The next time an app tells you it uses AI to personalise your wellness journey, ask one simple question: personalise it toward what? If the answer is engagement, close it and find something quieter.