Lo Resolví
Tech

Updating an app in closed testing and with real users. What changes

I found a subscription bug before launching my app. Here is what changes when touching the database in closed testing versus with real paying users.

In this article
  1. The stages before reaching everyone
  2. The bug I found
  3. Fail-open and fail-closed
  4. How I fixed it
  5. In closed testing
  6. With real users
  7. Changing or expanding the database without hurting anyone
  8. The order for deploying the fix
  9. What I am taking away

I have an app approved on Google Play that has not launched publicly yet. Three of us use it by invitation, and I use it every day. This week I audited the subscription activation flow from end to end and found a bug that could hand out the paid plan forever. Fixing it today is easy. With real users it would have been a very different story, and that is what this post is about.

CifraLab mobile app from cifralabapp.com on the guitar chords screen alongside the summary of safe updates for closed testing and production

The stages before reaching everyone

Google Play does not let you jump straight to the public. There are testing tracks first, and in each stage the risk of changing something is different.

  1. Internal testing. Just you and a small group. Useful to check that the app opens and does not crash.
  2. Closed testing. Only people you invite can install it. If your developer account is personal and was created after November 13, 2023, Google requires a closed test with at least 12 testers opted in for 14 days in a row before letting you apply for production.
  3. Production. Anyone can download it, pay and store their data.

As long as invitations stay closed, the records on your server belong to people you know. You can message any of them if something breaks. In production you cannot.

The bug I found

My app sells a monthly subscription. The server decides whether someone has the paid plan. To do that, it stores the expiration date of each purchase after asking Google.

Google notifies my server every time a subscription changes. A renewal, a cancellation or a refund. On every notification, my server asked Google again and saved the new date.

The bug lived in an edge case. When Google no longer recognized a purchase, such as a subscription that had expired long ago, it returned no date at all. My server then saved an empty date.

And when checking whether someone had the plan, the server asked if that date was already in the past. With an empty date the answer was no. That account stayed on the paid plan forever.

A missing value turned into access. Nobody was ever going to complain about free access, so with real users I might never have noticed.

Fail-open and fail-closed

Think of a door with an electric lock. If the power goes out, the door can stay unlocked or it can stay locked. Both are design choices.

In payments and access control, when in doubt the door stays locked. If I do not know when a purchase expires, I do not grant the plan and I ask Google again. My server did the opposite. When in doubt, it opened the door.

How I fixed it

Before touching the code I wrote two automated tests that reproduced the bug. One simulated a Google notification with no date. The other stored one purchase with an empty date and another with a malformed date. Both failed. That confirmed the bug was real.

The fix had two parts. When Google does not return a valid date, the server now saves today’s date and marks the purchase expired. And when a stored Google purchase has no valid date, it counts as expired and the server asks Google again. If the subscription is actually active, Google confirms it and the user loses nothing.

Manual promotions I grant by hand stay unchanged, because those are allowed to have no end date. With the fix in place, both new tests and every existing test passed.

In closed testing

Today the only real records are mine and two other people’s. Nobody has paid real money yet. At this stage a fix like this carries almost zero risk.

I can deploy it, watch the logs and adjust it without affecting anyone. If anything goes wrong, I message the other two people and we sort it out. That is why it pays to hunt for these bugs before opening invitations.

Even so, I follow the same discipline I would use with thousands of users. If I get used to skipping steps now, I will skip them later too.

With real users

With people paying, the exact same fix hurts in both directions.

  1. If you do not fix it, you lose money silently. Every refund or old purchase that arrives without a date leaves an account on the paid plan for free forever.
  2. If you fix it carelessly, you revoke access from someone who paid. If a paying user happens to have an empty date from an old bug, the fix treats it as expired. If Google happens to fail to respond right at that moment, they lose access briefly. To someone who paid, that feels like theft.

That is why, with real users, you first check how many accounts the change touches.

Changing or expanding the database without hurting anyone

The date bug is one case of a broader rule. Every time you change what your server stores, someone can lose data. These are the rules I follow in testing and production.

  1. Additive changes only. A new column is optional and has a default value. Old rows keep working without touching them.
  2. Nothing is deleted or renamed while any live version of the app still uses it. Users do not all update on the day you release.
  3. The server accepts older app versions. If an older version sends a field you no longer expect, the server ignores it instead of crashing.
  4. Backup before every schema change. It is the only reliable way back if something goes wrong.
  5. Test against old data. Before releasing, I open a copy of the previous version’s database with the new build.
  6. A network error is never an empty cloud. If the server fails to respond, the app must never assume the user has no data or wipe what is on the phone.

In closed testing you can bend one of these rules if needed, because you know every person using it. With real users there are no exceptions.

The order for deploying the fix

This is how I do it, even with no paying users yet.

  1. Count who is affected. A read-only query counts active purchases with an empty date. If it returns zero, the change touches nobody.
  2. If any exist, fix them first. Ask Google for each one’s real expiration date and save it. Nobody loses access for even a minute.
  3. Back up the database.
  4. Deploy the fix with all tests passing.
  5. Watch server logs on day one. If anyone with an active subscription lost access, fix it immediately by querying Google.
  6. Only then, write about it. That is why this post goes live after the fix is already deployed.

What I am taking away

  1. An empty value must never become permission. When in doubt, deny access.
  2. The most valuable tests cover messy edge cases. Empty, broken, no response.
  3. Writing the failing test first proves the bug is real.
  4. Anything that costs server resources or touches shared data is decided by the server on every request. Things that only run on the phone, like hiding ads, can be bypassed with a modified app, and I accept that because it touches nobody’s data.
  5. The best time to catch these bugs is while invitations are still closed.

If you are still preparing your launch on Google Play, I also wrote about Google Play closed testing and what can get your app rejected.

Checked on November 21, 2026. The closed testing requirement is in the Play Console help page on app testing requirements for new personal developer accounts. Subscription states and Google notifications are covered in the subscription lifecycle guide.

Comments

Write a comment

Comments are reviewed before they are published. Read the Privacy policy.

← Newer post