Registering Service Worker causes NO INTERNET chrome page on default install #11987

Describe the bug

Run a brand new create react app project and register a service worker. Build your app using yarn build and serve it using serve -s build. Open a new chrome tab in incognito mode. While the page will serve normally for the first 10 seconds, when the service worker activates, the page suddenly breaks with an appear offline message.

Did you try recovering your dependencies?

This can be reproduced using a brand new project.

Which terms did you search for in User Guide?

None of the troubleshooting tips in https://create-react-app.dev/docs/troubleshooting/ are helpful.

Environment

node -v
v16.11.1
npx -v
6.14.8

Steps to reproduce

First install a fresh project

npx create-react-app@v5.0.0 my-app --template cra-template-pwa-typescript
cd my-app
yarn install

then apply this patch

diff --git a/src/index.tsx b/src/index.tsx
index e459b13..4a7b0ba 100644
--- a/src/index.tsx
+++ b/src/index.tsx
@@ -15,7 +15,7 @@ ReactDOM.render(
 // If you want your app to work offline and load faster, you can change
 // unregister() to register() below. Note this comes with some pitfalls.
 // Learn more about service workers: https://cra.link/PWA
-serviceWorkerRegistration.unregister();
+serviceWorkerRegistration.register();

 // If you want to start measuring performance in your app, pass a function
 // to log results (for example: reportWebVitals(console.log))

And finally build and serve

yarn build
serve -s build

Actual behavior

Try to open an incognito page on Google Chrome (v97 or above). The page will load for a few seconds. If you refresh the page and show the inspector and go to the Application tab, it displays the dinosaur.

Expected behavior

It should display the create react app homepage

Reproducible demo

See steps to reproduce.