Case Study
We Renamed One Bot. Engagement Doubled. The Rest Never Shipped.
What on-ground research in tier-3 and tier-4 government schools revealed about an app students only opened when required to — and why the fix that worked never reached the other bots.
- Brand
- SwiftChat.AI
- Role
- Growth Hacker (part-time, remote)
- Period
- September 2023 – February 2024
- Market
- India — tier-3 and tier-4 government school students, mostly below class 10
Jump to:The Short AnswerThe SituationWhat the Data Already SaidWhat the Ground Research FoundThe Fix, and the ProofWhy It Never ShippedHow to Apply ThisEvidence & Limits
The short answer
I was asked to lift bot engagement, which was sitting at 25–30%. Everyone assumed the answer was gamification.
Going on the ground told us otherwise. These students share one smartphone with their parents, get limited access to it, and spend that access on YouTube — because YouTube is the one platform parents have already classified as learning. The app itself was pushed to schools by central and state government, so students opened it for the mandatory weekly quiz and attendance, and closed it. We weren't competing with other learning apps. We were competing with the only app that had already cleared the gatekeeper.
The fix we proposed was not gamification. It was names. Stories bot became Kisse, Puzzles bot became Bhujo Toh Jaane, English bot became Seekho English — the language the students actually think in, describing what the thing does rather than what category it belongs to.
We got one renamed. Engagement on the story bot went from roughly 20% to around 40%. The remaining renames never shipped, and that part is on me.
Going on the ground told us otherwise. These students share one smartphone with their parents, get limited access to it, and spend that access on YouTube — because YouTube is the one platform parents have already classified as learning. The app itself was pushed to schools by central and state government, so students opened it for the mandatory weekly quiz and attendance, and closed it. We weren't competing with other learning apps. We were competing with the only app that had already cleared the gatekeeper.
The fix we proposed was not gamification. It was names. Stories bot became Kisse, Puzzles bot became Bhujo Toh Jaane, English bot became Seekho English — the language the students actually think in, describing what the thing does rather than what category it belongs to.
We got one renamed. Engagement on the story bot went from roughly 20% to around 40%. The remaining renames never shipped, and that part is on me.
The transferable principle
When adoption is mandated, engagement metrics measure compliance, not interest. Gamification layered on compulsory usage amplifies a motivation that was never there.
The situation
SwiftChat is a conversational learning app built around subject bots — science, maths, English, stories, puzzles — where a student can ask a question and get an answer back in chat. This was September 2023, before ChatGPT was mainstream in this market, so a bot that answered your maths question was genuinely novel.
The distribution model is the part that matters. The app was pushed to students and teachers in government schools by central and state government as part of digitising education. It did not win its users; it was issued to them. The audience was tier-3 and tier-4 government school students, most below class 10.
I joined as a part-time Growth Hacker in September 2023, working remotely, on GTM strategy for retention and engagement. The brief I was handed was narrow and specific: bot engagement was hovering at 25–30%, and my job was to raise it.
The distribution model is the part that matters. The app was pushed to students and teachers in government schools by central and state government as part of digitising education. It did not win its users; it was issued to them. The audience was tier-3 and tier-4 government school students, most below class 10.
I joined as a part-time Growth Hacker in September 2023, working remotely, on GTM strategy for retention and engagement. The brief I was handed was narrow and specific: bot engagement was hovering at 25–30%, and my job was to raise it.
What the data already said
The team had CleverTap and GA4 in place, but neither was really being read. There was tracking without interpretation — a common state, and a more dangerous one than having no data, because everyone assumes the question has been answered.
I rebuilt the reporting in GA4 with custom reports, working from two metrics: engagement rate — engaged sessions over total sessions, where an engaged session means ten seconds or more, or a conversion, or two or more screen views — and average engagement time per screen, which for us meant time spent inside a given bot.
Here is what nobody had said out loud. A typical GA4 engagement rate runs somewhere around 55–70%. Ours was 25–30%. That means roughly seven in ten sessions were under ten seconds, on a single screen, with nothing completed.
That is not a weak engagement number. That is the statistical signature of compliance. It is what it looks like when people open something because they have been told to, do the one thing required of them, and leave. The data had been saying so for months. We read it as an engagement problem to be solved with features, because that was the answer we were already looking for.
I rebuilt the reporting in GA4 with custom reports, working from two metrics: engagement rate — engaged sessions over total sessions, where an engaged session means ten seconds or more, or a conversion, or two or more screen views — and average engagement time per screen, which for us meant time spent inside a given bot.
Here is what nobody had said out loud. A typical GA4 engagement rate runs somewhere around 55–70%. Ours was 25–30%. That means roughly seven in ten sessions were under ten seconds, on a single screen, with nothing completed.
That is not a weak engagement number. That is the statistical signature of compliance. It is what it looks like when people open something because they have been told to, do the one thing required of them, and leave. The data had been saying so for months. We read it as an engagement problem to be solved with features, because that was the answer we were already looking for.
What the ground research found
So we went and watched students use it.
Three things came back, none of which were visible in any dashboard.
The phone is not theirs. It belongs to a parent, and access is rationed. Whatever happens on that phone happens inside a short, contested window.
That window goes to YouTube. Not because students prefer it in the abstract, but because it is the platform parents are comfortable with — in a parent's mental model, YouTube is where you can learn things. It has already been approved. Every other app is competing for time that has to be justified.
Nobody was there voluntarily. Students opened SwiftChat for the weekly quiz used for assessment, or to mark attendance. That was the job the app did for them. Everything else in it was, from their point of view, optional software attached to a compulsory task.
Which reframed my brief entirely. I had been asked to raise engagement inside an app that its users had no reason to be in, on a device they did not control, in a time window already spoken for by the one platform their parents trusted. The gamification ideas we had been confidently discussing were, as I put it at the time, working only in our heads.
Three things came back, none of which were visible in any dashboard.
The phone is not theirs. It belongs to a parent, and access is rationed. Whatever happens on that phone happens inside a short, contested window.
That window goes to YouTube. Not because students prefer it in the abstract, but because it is the platform parents are comfortable with — in a parent's mental model, YouTube is where you can learn things. It has already been approved. Every other app is competing for time that has to be justified.
Nobody was there voluntarily. Students opened SwiftChat for the weekly quiz used for assessment, or to mark attendance. That was the job the app did for them. Everything else in it was, from their point of view, optional software attached to a compulsory task.
Which reframed my brief entirely. I had been asked to raise engagement inside an app that its users had no reason to be in, on a device they did not control, in a time window already spoken for by the one platform their parents trusted. The gamification ideas we had been confidently discussing were, as I put it at the time, working only in our heads.
The fix, and the proof
If you cannot out-compete YouTube for a rationed slot, and you cannot manufacture voluntary motivation, the remaining lever is friction. Make it instantly obvious what each bot is and why you would open it.
The bots were named by subject category, in English: Stories bot, Puzzles bot, English bot, Maths bot. Perfectly clear to the people who built them. To a class-7 student in a tier-4 government school, they read as a menu of school subjects — which is to say, as more school.
So we proposed renaming them into the register the students actually speak: Kisse or Kahaniyan for stories, Bhujo Toh Jaane for puzzles, Seekho English for English. Not translation for its own sake — a name that tells you what you are about to get, in the language you think in.
This is the same principle I had run into at Entri three years earlier, in an entirely different product: meet users in the register of their current competence, not the register of the category.
We renamed one bot. The stories bot became Kisse, and its engagement rate went from roughly 20% to around 40% — a gain of about twenty percentage points, close to a doubling, from changing a label and nothing else.
One bot is one data point, and I would not call a single rename a controlled experiment. But it was a strong enough signal to justify doing the rest.
The bots were named by subject category, in English: Stories bot, Puzzles bot, English bot, Maths bot. Perfectly clear to the people who built them. To a class-7 student in a tier-4 government school, they read as a menu of school subjects — which is to say, as more school.
So we proposed renaming them into the register the students actually speak: Kisse or Kahaniyan for stories, Bhujo Toh Jaane for puzzles, Seekho English for English. Not translation for its own sake — a name that tells you what you are about to get, in the language you think in.
This is the same principle I had run into at Entri three years earlier, in an entirely different product: meet users in the register of their current competence, not the register of the category.
We renamed one bot. The stories bot became Kisse, and its engagement rate went from roughly 20% to around 40% — a gain of about twenty percentage points, close to a doubling, from changing a label and nothing else.
One bot is one data point, and I would not call a single rename a controlled experiment. But it was a strong enough signal to justify doing the rest.
Why it never shipped
The rest never happened. A change that should have taken a day or two stretched across a month, stalled, and was eventually dropped. I left the engagement shortly after.
That failure is mine, and it is the most useful thing in this case study.
I had spent weeks diagnosing how students behaved and none at all diagnosing how the company made decisions. I walked in with a rename that touched work other people had built, named and defended, and I brought it as a conclusion rather than as something they had helped reach. I had a validated result on one bot and assumed the number would carry the argument. Numbers do not carry arguments through organisations. People do, and I had not brought any with me.
What I would do differently is not analytical. I would have found the people whose work the rename touched and involved them in the ground research itself — let them watch a student scroll past Stories bot — rather than presenting the finding afterwards as a verdict on their naming. I would have mapped who had to agree before I built the plan, not after. A stakeholder map would have been worth more than another GA4 report.
The lesson generalises past this account, and it is the one I most often see consultants and growth people learn late: a correct recommendation that cannot survive the organisation is not a correct recommendation. Feasibility is part of the answer, not a constraint applied to it afterwards.
That failure is mine, and it is the most useful thing in this case study.
I had spent weeks diagnosing how students behaved and none at all diagnosing how the company made decisions. I walked in with a rename that touched work other people had built, named and defended, and I brought it as a conclusion rather than as something they had helped reach. I had a validated result on one bot and assumed the number would carry the argument. Numbers do not carry arguments through organisations. People do, and I had not brought any with me.
What I would do differently is not analytical. I would have found the people whose work the rename touched and involved them in the ground research itself — let them watch a student scroll past Stories bot — rather than presenting the finding afterwards as a verdict on their naming. I would have mapped who had to agree before I built the plan, not after. A stakeholder map would have been worth more than another GA4 report.
The lesson generalises past this account, and it is the one I most often see consultants and growth people learn late: a correct recommendation that cannot survive the organisation is not a correct recommendation. Feasibility is part of the answer, not a constraint applied to it afterwards.
The second principle
A recommendation that can't survive the organisation isn't a correct recommendation. Diagnose how decisions get made with the same rigour you diagnose users.
How to apply this
What transfers out of this account into any similar one:
- Ask who chose to install it. If adoption was mandated — government, employer, school — your engagement metrics are measuring compliance. Interpret them accordingly and stop expecting features to fix motivation.
- Compare your engagement rate to the benchmark before you plan anything. Sitting far below typical is diagnostic information, not just a bad score. Ours was describing the problem months before anyone read it that way.
- Find out who actually controls the device. For younger or lower-income audiences the phone is shared and access is rationed, which means you are competing for a slot rather than for attention.
- Identify what has already cleared the gatekeeper. Our real competitor was the one app parents had pre-approved. That is a positioning problem, not a product one.
- Name things in the user's register, not your category's. Internal naming is written by people who already understand the product. Users are deciding whether to spend a scarce minute on it.
- Map the decision before you build the plan. Work out who must agree, and bring them into the research rather than presenting them with its conclusions.
- Treat a validated result as the start of the argument, not the end of it. A doubled engagement rate did not carry my proposal. It would not have carried yours either.
Evidence and limits
I worked on this part-time and remotely as a Growth Hacker between September 2023 and February 2024. The engagement ended with the core proposal unshipped.
Both figures here are from recall and are approximations: a 25–30% baseline bot engagement rate across the app, and roughly 20% to around 40% on the stories bot after renaming. I no longer have access to the analytics, so I can't pull them from source. The stories-bot result is a single bot changed at a single point in time, without a holdout — a strong signal rather than a clean experiment, and I would not present it as more than that.
The on-ground research findings are qualitative observation of students using the app, not survey data.
I have written this without characterising the internal disagreement or the people involved, because my account of it would be one-sided and it isn't the useful part. The useful part is that I failed to build the agreement the plan needed, which is mine to report.
Both figures here are from recall and are approximations: a 25–30% baseline bot engagement rate across the app, and roughly 20% to around 40% on the stories bot after renaming. I no longer have access to the analytics, so I can't pull them from source. The stories-bot result is a single bot changed at a single point in time, without a holdout — a strong signal rather than a clean experiment, and I would not present it as more than that.
The on-ground research findings are qualitative observation of students using the app, not survey data.
I have written this without characterising the internal disagreement or the people involved, because my account of it would be one-sided and it isn't the useful part. The useful part is that I failed to build the agreement the plan needed, which is mine to report.
What can be independently checked:
