SIGSEGV: segmentation violation #676

Activity

1. m-hm

   added

   bugSomething isn't workingSomething isn't working

   on Mar 19, 2024
2. dunglas commented on Mar 19, 2024

   

   dunglas

   on Mar 19, 2024

   Member

   More actions

   Can you show your Dockerfile, please?
3. m-hm commented on Mar 19, 2024

   

   m-hm

   on Mar 19, 2024

   Author

   More actions

   No Dockerfile, Download from https://github.com/dunglas/frankenphp/releases/download/v1.1.1/frankenphp-linux-x86_64
4. dunglas commented on Mar 19, 2024

   

   dunglas

   on Mar 19, 2024

   Member

   More actions

   I tried v1.1.1 on an Ubuntu (x86_64) machine and a Mac and could not reproduce the crash.
   Could you provide a minimal reproducer, please? Also, could you try downgrading to v1.1.0 to test if it fixes the issue?

   Thanks.
5. dunglas

   added

   questionFurther information is requestedFurther information is requested

   on Mar 19, 2024
6. emaia commented on Mar 19, 2024

   

   emaia

   on Mar 19, 2024

   Last edited by emaia

   More actions

   Hi! I'm also getting this same error sporadically but with some frequency.
   I'm running on WSL2 with local binary, downloaded from https://github.com/dunglas/frankenphp/releases/tag/v1.1.1 Same error with v1.1.0

   Caddyfile:

   {
       frankenphp
       order php_server before file_server
   }
   
   example.localhost {
       root * public/
       encode zstd br gzip
       php_server
   }
   

   I'm running a Laravel app with bun run dev (to compile Tailwind with Vite). This error happens when I change some file (JS, CSS or Blade) and hot reload occurs.
7. dunglas commented on Mar 19, 2024

   

   dunglas

   on Mar 19, 2024

   Member

   More actions

   I suspect that this is this bug: php/php-src#13648

   Could you try to gather a stack trace using a debug build to confirm that this is it? In this case, a workaround is to use the Docker image or to compile FrankenPHP with a dynamic link to libphp locally.
8. emaia commented on Mar 22, 2024

   

   emaia

   on Mar 22, 2024

   More actions

   @dunglas It works without errors using Docker version.

   My output with debug binary:

   SIGSEGV: segmentation violation
   PC=0x7f4d9ac01a1b m=24 sigcode=1 addr=0x7f4d4462be70
   signal arrived during cgo execution
   
   goroutine 182 gp=0xc000bb21c0 m=24 mp=0xc000601008 [syscall, locked to thread]:
   runtime.cgocall(0x7f4d99240ce0, 0xc000bbdce0)
           /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc000bbdcb8 sp=0xc000bbdc80 pc=0x7f4d97e70d6b
   github.com/dunglas/frankenphp._Cfunc_frankenphp_update_server_context(0x1, 0xa, 0x0, 0x7f4d43bc9120, 0x7f4d43bc9140, 0x0, 0x0, 0x7f4d43bc9150, 0x0, 0x0, ...)
           _cgo_gotypes.go:1060 +0x52 fp=0xc000bbdce0 sp=0xc000bbdcb8 pc=0x7f4d990b84b2
   github.com/dunglas/frankenphp.updateServerContext(0xc000d06b40, 0x1, 0x0)
           /go/src/app/frankenphp.go:419 +0x4e5 fp=0xc000bbddd8 sp=0xc000bbdce0 pc=0x7f4d990baa05
   github.com/dunglas/frankenphp.go_execute_script(0x7f4d97ee0d01?)
           /go/src/app/frankenphp.go:506 +0x10f fp=0xc000bbde40 sp=0xc000bbddd8 pc=0x7f4d990bb10f
   _cgoexp_feab5cfeb5bc_go_execute_script(0x7f4d9e3f16b0?)
           _cgo_gotypes.go:1111 +0x16 fp=0xc000bbde58 sp=0xc000bbde40 pc=0x7f4d990c0816
   runtime.cgocallbackg1(0x7f4d990c0800, 0x7f4d455ca850, 0x0)
           /usr/local/go/src/runtime/cgocall.go:403 +0x2a5 fp=0xc000bbdf18 sp=0xc000bbde58 pc=0x7f4d97e714a5
   runtime.cgocallbackg(0x7f4d990c0800, 0x7f4d455ca850, 0x0)
           /usr/local/go/src/runtime/cgocall.go:322 +0x138 fp=0xc000bbdf90 sp=0xc000bbdf18 pc=0x7f4d97e71158
   runtime.cgocallbackg(0x7f4d990c0800, 0x7f4d455ca850, 0x0)
           <autogenerated>:1 +0x2b fp=0xc000bbdfb8 sp=0xc000bbdf90 pc=0x7f4d97ee54eb
   runtime.cgocallback(0x0, 0x0, 0x0)
           /usr/local/go/src/runtime/asm_amd64.s:1079 +0xcd fp=0xc000bbdfe0 sp=0xc000bbdfb8 pc=0x7f4d97ee2b2d
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000bbdfe8 sp=0xc000bbdfe0 pc=0x7f4d97ee2d81
   
   goroutine 1 gp=0xc0000061c0 m=nil [select (no cases)]:
   runtime.gopark(0xc00091f840?, 0x7f4d97fa1b4c?, 0x77?, 0x2?, 0x4?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000af9820 sp=0xc000af9800 pc=0x7f4d97eaa84e
   runtime.block()
           /usr/local/go/src/runtime/select.go:103 +0x26 fp=0xc000af9850 sp=0xc000af9820 pc=0x7f4d97ebbfe6
   github.com/caddyserver/caddy/v2/cmd.cmdRun({0x0?})
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/cmd/commandfuncs.go:283 +0xbfc fp=0xc000af9b60 sp=0xc000af9850 pc=0x7f4d98598a9c
   github.com/caddyserver/caddy/v2/cmd.init.1.func2.WrapCommandFuncForCobra.1(0xc000156c08, {0x7f4d9ac1048f?, 0x4?, 0x7f4d9ac10297?})
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/cmd/cobra.go:137 +0x2f fp=0xc000af9ba0 sp=0xc000af9b60 pc=0x7f4d985a52af
   github.com/spf13/cobra.(*Command).execute(0xc000156c08, {0x7f4d9f2e8000, 0x0, 0x0})
           /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983 +0xaca fp=0xc000af9d28 sp=0xc000af9ba0 pc=0x7f4d98519b4a
   github.com/spf13/cobra.(*Command).ExecuteC(0x7f4d9f189040)
           /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x3ff fp=0xc000af9e00 sp=0xc000af9d28 pc=0x7f4d9851a41f
   github.com/spf13/cobra.(*Command).Execute(...)
           /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
   github.com/caddyserver/caddy/v2/cmd.Main()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/cmd/main.go:66 +0x5b fp=0xc000af9e60 sp=0xc000af9e00 pc=0x7f4d9859d31b
   main.main()
           /go/src/app/caddy/frankenphp/main.go:24 +0x18d fp=0xc000af9f50 sp=0xc000af9e60 pc=0x7f4d9923fd8d
   runtime.main()
           /usr/local/go/src/runtime/proc.go:271 +0x29d fp=0xc000af9fe0 sp=0xc000af9f50 pc=0x7f4d97eaa3fd
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000af9fe8 sp=0xc000af9fe0 pc=0x7f4d97ee2d81
   
   goroutine 17 gp=0xc000006700 m=nil [select, locked to thread]:
   runtime.gopark(0xc0000d6e10?, 0x2?, 0x8?, 0x0?, 0xc0000d6df4?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d6c90 sp=0xc0000d6c70 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc0000d6e10, 0xc0000d6df0, 0x7f4d9f2e8000?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0000d6db0 sp=0xc0000d6c90 pc=0x7f4d97ebc725
   github.com/dunglas/frankenphp.go_fetch_request()
           /go/src/app/frankenphp.go:473 +0x75 fp=0xc0000d6e48 sp=0xc0000d6db0 pc=0x7f4d990bae55
   _cgoexp_feab5cfeb5bc_go_fetch_request(0x7f4d5001a890)
           _cgo_gotypes.go:1103 +0x14 fp=0xc0000d6e58 sp=0xc0000d6e48 pc=0x7f4d990c07d4
   runtime.cgocallbackg1(0x7f4d990c07c0, 0x7f4d5001a890, 0x0)
           /usr/local/go/src/runtime/cgocall.go:403 +0x2a5 fp=0xc0000d6f18 sp=0xc0000d6e58 pc=0x7f4d97e714a5
   runtime.cgocallbackg(0x7f4d990c07c0, 0x7f4d5001a890, 0x0)
           /usr/local/go/src/runtime/cgocall.go:322 +0x138 fp=0xc0000d6f90 sp=0xc0000d6f18 pc=0x7f4d97e71158
   runtime.cgocallbackg(0x7f4d990c07c0, 0x7f4d5001a890, 0x0)
           <autogenerated>:1 +0x2b fp=0xc0000d6fb8 sp=0xc0000d6f90 pc=0x7f4d97ee54eb
   runtime.cgocallback(0x0, 0x0, 0x0)
           /usr/local/go/src/runtime/asm_amd64.s:1079 +0xcd fp=0xc0000d6fe0 sp=0xc0000d6fb8 pc=0x7f4d97ee2b2d
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d6fe8 sp=0xc0000d6fe0 pc=0x7f4d97ee2d81
   
   goroutine 2 gp=0xc000006c40 m=nil [force gc (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c2fa8 sp=0xc0000c2f88 pc=0x7f4d97eaa84e
   runtime.goparkunlock(...)
           /usr/local/go/src/runtime/proc.go:408
   runtime.forcegchelper()
           /usr/local/go/src/runtime/proc.go:326 +0xb8 fp=0xc0000c2fe0 sp=0xc0000c2fa8 pc=0x7f4d97eaa6b8
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c2fe8 sp=0xc0000c2fe0 pc=0x7f4d97ee2d81
   created by runtime.init.6 in goroutine 1
           /usr/local/go/src/runtime/proc.go:314 +0x1a
   
   goroutine 3 gp=0xc000007180 m=nil [GC sweep wait]:
   runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c3780 sp=0xc0000c3760 pc=0x7f4d97eaa84e
   runtime.goparkunlock(...)
           /usr/local/go/src/runtime/proc.go:408
   runtime.bgsweep(0xc0000ea000)
           /usr/local/go/src/runtime/mgcsweep.go:318 +0xdf fp=0xc0000c37c8 sp=0xc0000c3780 pc=0x7f4d97e9341f
   runtime.gcenable.gowrap1()
           /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc0000c37e0 sp=0xc0000c37c8 pc=0x7f4d97e87ce5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c37e8 sp=0xc0000c37e0 pc=0x7f4d97ee2d81
   created by runtime.gcenable in goroutine 1
           /usr/local/go/src/runtime/mgc.go:203 +0x66
   
   goroutine 4 gp=0xc000007340 m=nil [GC scavenge wait]:
   runtime.gopark(0x7f4d9f2569e0?, 0x7f4d9b122860?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c3f78 sp=0xc0000c3f58 pc=0x7f4d97eaa84e
   runtime.goparkunlock(...)
           /usr/local/go/src/runtime/proc.go:408
   runtime.(*scavengerState).park(0x7f4d9f2569e0)
           /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc0000c3fa8 sp=0xc0000c3f78 pc=0x7f4d97e90da9
   runtime.bgscavenge(0xc0000ea000)
           /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc0000c3fc8 sp=0xc0000c3fa8 pc=0x7f4d97e91359
   runtime.gcenable.gowrap2()
           /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc0000c3fe0 sp=0xc0000c3fc8 pc=0x7f4d97e87c85
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c3fe8 sp=0xc0000c3fe0 pc=0x7f4d97ee2d81
   created by runtime.gcenable in goroutine 1
           /usr/local/go/src/runtime/mgc.go:204 +0xa5
   
   goroutine 5 gp=0xc000007c00 m=nil [finalizer wait]:
   runtime.gopark(0xc0000c2648?, 0x7f4d97e7ae65?, 0xa8?, 0x1?, 0xc0000061c0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c2620 sp=0xc0000c2600 pc=0x7f4d97eaa84e
   runtime.runfinq()
           /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc0000c27e0 sp=0xc0000c2620 pc=0x7f4d97e86d27
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c27e8 sp=0xc0000c27e0 pc=0x7f4d97ee2d81
   created by runtime.createfing in goroutine 1
           /usr/local/go/src/runtime/mfinal.go:164 +0x3d
   
   goroutine 6 gp=0xc0002b21c0 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c4750 sp=0xc0000c4730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c47e0 sp=0xc0000c4750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c47e8 sp=0xc0000c47e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 18 gp=0xc000102700 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000be750 sp=0xc0000be730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000be7e0 sp=0xc0000be750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000be7e8 sp=0xc0000be7e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 34 gp=0xc000502000 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000508750 sp=0xc000508730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005087e0 sp=0xc000508750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005087e8 sp=0xc0005087e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 7 gp=0xc0002b2380 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c4f50 sp=0xc0000c4f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c4fe0 sp=0xc0000c4f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c4fe8 sp=0xc0000c4fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 19 gp=0xc0001028c0 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000bef50 sp=0xc0000bef30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000befe0 sp=0xc0000bef50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000befe8 sp=0xc0000befe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 35 gp=0xc0005021c0 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000508f50 sp=0xc000508f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000508fe0 sp=0xc000508f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000508fe8 sp=0xc000508fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 8 gp=0xc0002b2540 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c5750 sp=0xc0000c5730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c57e0 sp=0xc0000c5750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c57e8 sp=0xc0000c57e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 20 gp=0xc000102a80 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000bf750 sp=0xc0000bf730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000bf7e0 sp=0xc0000bf750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000bf7e8 sp=0xc0000bf7e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 36 gp=0xc000502380 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000509750 sp=0xc000509730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005097e0 sp=0xc000509750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005097e8 sp=0xc0005097e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 9 gp=0xc0002b2700 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c5f50 sp=0xc0000c5f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c5fe0 sp=0xc0000c5f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c5fe8 sp=0xc0000c5fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 21 gp=0xc000102c40 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000bff50 sp=0xc0000bff30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000bffe0 sp=0xc0000bff50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000bffe8 sp=0xc0000bffe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 37 gp=0xc000502540 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000509f50 sp=0xc000509f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000509fe0 sp=0xc000509f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000509fe8 sp=0xc000509fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 10 gp=0xc0002b28c0 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000504750 sp=0xc000504730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005047e0 sp=0xc000504750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005047e8 sp=0xc0005047e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 22 gp=0xc000102e00 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c0750 sp=0xc0000c0730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c07e0 sp=0xc0000c0750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c07e8 sp=0xc0000c07e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 38 gp=0xc000502700 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00050a750 sp=0xc00050a730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00050a7e0 sp=0xc00050a750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00050a7e8 sp=0xc00050a7e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 11 gp=0xc0002b2a80 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000504f50 sp=0xc000504f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000504fe0 sp=0xc000504f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000504fe8 sp=0xc000504fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 23 gp=0xc000102fc0 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c0f50 sp=0xc0000c0f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c0fe0 sp=0xc0000c0f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c0fe8 sp=0xc0000c0fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 39 gp=0xc0005028c0 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00050af50 sp=0xc00050af30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00050afe0 sp=0xc00050af50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00050afe8 sp=0xc00050afe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 24 gp=0xc000103180 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c1750 sp=0xc0000c1730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c17e0 sp=0xc0000c1750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c17e8 sp=0xc0000c17e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 12 gp=0xc0002b2c40 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000505750 sp=0xc000505730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005057e0 sp=0xc000505750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005057e8 sp=0xc0005057e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 40 gp=0xc000502a80 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00050b750 sp=0xc00050b730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00050b7e0 sp=0xc00050b750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00050b7e8 sp=0xc00050b7e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 13 gp=0xc0002b2e00 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000505f50 sp=0xc000505f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000505fe0 sp=0xc000505f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000505fe8 sp=0xc000505fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 25 gp=0xc000103340 m=nil [GC worker (idle)]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c1f50 sp=0xc0000c1f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c1fe0 sp=0xc0000c1f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c1fe8 sp=0xc0000c1fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 41 gp=0xc000502c40 m=nil [GC worker (idle)]:
   runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0x94?, 0x57?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00050bf50 sp=0xc00050bf30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00050bfe0 sp=0xc00050bf50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 14 gp=0xc0002b2fc0 m=nil [GC worker (idle)]:
   runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0xab?, 0x71?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000506750 sp=0xc000506730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005067e0 sp=0xc000506750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005067e8 sp=0xc0005067e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 26 gp=0xc000103500 m=nil [GC worker (idle)]:
   runtime.gopark(0x75b188bf8f6b?, 0x1?, 0xbc?, 0x5d?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011e750 sp=0xc00011e730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011e7e0 sp=0xc00011e750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011e7e8 sp=0xc00011e7e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 42 gp=0xc000502e00 m=nil [GC worker (idle)]:
   runtime.gopark(0x75af833a6444?, 0x1?, 0xbe?, 0x9b?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011a750 sp=0xc00011a730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011a7e0 sp=0xc00011a750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011a7e8 sp=0xc00011a7e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 15 gp=0xc0002b3180 m=nil [GC worker (idle)]:
   runtime.gopark(0x75b188bb7a40?, 0x1?, 0x64?, 0x4b?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000506f50 sp=0xc000506f30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000506fe0 sp=0xc000506f50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000506fe8 sp=0xc000506fe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 27 gp=0xc0001036c0 m=nil [GC worker (idle)]:
   runtime.gopark(0x75b188b472a3?, 0x1?, 0xff?, 0x4c?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011ef50 sp=0xc00011ef30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011efe0 sp=0xc00011ef50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011efe8 sp=0xc00011efe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 43 gp=0xc000502fc0 m=nil [GC worker (idle)]:
   runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0xa0?, 0xd6?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011af50 sp=0xc00011af30 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011afe0 sp=0xc00011af50 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011afe8 sp=0xc00011afe0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 16 gp=0xc0002b3340 m=nil [GC worker (idle)]:
   runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0x63?, 0x3e?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000507750 sp=0xc000507730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005077e0 sp=0xc000507750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005077e8 sp=0xc0005077e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 28 gp=0xc000103880 m=nil [GC worker (idle)]:
   runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0xea?, 0x69?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011f750 sp=0xc00011f730 pc=0x7f4d97eaa84e
   runtime.gcBgMarkWorker()
           /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011f7e0 sp=0xc00011f750 pc=0x7f4d97e89de5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011f7e8 sp=0xc00011f7e0 pc=0x7f4d97ee2d81
   created by runtime.gcBgMarkStartWorkers in goroutine 1
           /usr/local/go/src/runtime/mgc.go:1234 +0x1c
   
   goroutine 50 gp=0xc0005836c0 m=nil [select]:
   runtime.gopark(0xc000121f80?, 0x2?, 0xf1?, 0xb5?, 0xc000121f74?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000121e18 sp=0xc000121df8 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc000121f80, 0xc000121f70, 0x0?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000121f38 sp=0xc000121e18 pc=0x7f4d97ebc725
   github.com/golang/glog.(*fileSink).flushDaemon(0x7f4d9f256438)
           /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:351 +0xb9 fp=0xc000121fc8 sp=0xc000121f38 pc=0x7f4d98853bf9
   github.com/golang/glog.init.1.gowrap1()
           /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:166 +0x25 fp=0xc000121fe0 sp=0xc000121fc8 pc=0x7f4d98852c85
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000121fe8 sp=0xc000121fe0 pc=0x7f4d97ee2d81
   created by github.com/golang/glog.init.1 in goroutine 1
           /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:166 +0x126
   
   goroutine 30 gp=0xc000583c00 m=nil [sync.Cond.Wait]:
   runtime.gopark(0x7f4d9e42e220?, 0xc000314080?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d5c70 sp=0xc0000d5c50 pc=0x7f4d97eaa84e
   runtime.goparkunlock(...)
           /usr/local/go/src/runtime/proc.go:408
   sync.runtime_notifyListWait(0xc0005100d8, 0x3)
           /usr/local/go/src/runtime/sema.go:569 +0x15a fp=0xc0000d5cc0 sp=0xc0000d5c70 pc=0x7f4d97edebfa
   sync.(*Cond).Wait(0xc00011b538?)
           /usr/local/go/src/sync/cond.go:70 +0x85 fp=0xc0000d5d00 sp=0xc0000d5cc0 pc=0x7f4d97eecaa5
   github.com/maypok86/otter/internal/queue.(*Growable[...]).Pop(0x7f4d9e427400)
           /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/queue/growable.go:71 +0x74 fp=0xc0000d5d98 sp=0xc0000d5d00 pc=0x7f4d990caad4
   github.com/maypok86/otter/internal/core.(*Cache[...]).process(0x7f4d9e444860)
           /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:386 +0xe8 fp=0xc0000d5fc0 sp=0xc0000d5d98 pc=0x7f4d990c8968
   github.com/maypok86/otter/internal/core.NewCache[...].gowrap2()
           /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:167 +0x25 fp=0xc0000d5fe0 sp=0xc0000d5fc0 pc=0x7f4d990bfb85
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d5fe8 sp=0xc0000d5fe0 pc=0x7f4d97ee2d81
   created by github.com/maypok86/otter/internal/core.NewCache[...] in goroutine 1
           /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:167 +0x6b3
   
   goroutine 66 gp=0xc000583500 m=nil [select]:
   runtime.gopark(0xc0001216e8?, 0x3?, 0x0?, 0x0?, 0xc0001216da?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000121580 sp=0xc000121560 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc0001216e8, 0xc0001216d4, 0xc0001216d0?, 0x2, 0xc000428140?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0001216a0 sp=0xc000121580 pc=0x7f4d97ebc725
   github.com/caddyserver/certmagic.(*RingBufferRateLimiter).permit(0xc0000f8eb0)
           /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:217 +0x86 fp=0xc000121728 sp=0xc0001216a0 pc=0x7f4d9832ce46
   github.com/caddyserver/certmagic.(*RingBufferRateLimiter).loop(0xc0000f8eb0)
           /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:89 +0x8b fp=0xc0001217c8 sp=0xc000121728 pc=0x7f4d9832c80b
   github.com/caddyserver/certmagic.NewRateLimiter.gowrap1()
           /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:45 +0x25 fp=0xc0001217e0 sp=0xc0001217c8 pc=0x7f4d9832c745
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001217e8 sp=0xc0001217e0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/certmagic.NewRateLimiter in goroutine 1
           /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:45 +0x13c
   
   goroutine 46 gp=0xc0008228c0 m=nil [chan receive]:
   runtime.gopark(0x7f4d984a97c5?, 0x7f4d9e1c26e0?, 0x1?, 0x9e?, 0xc000695ed8?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000695e48 sp=0xc000695e28 pc=0x7f4d97eaa84e
   runtime.chanrecv(0xc0000eec00, 0x0, 0x1)
           /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc000695ec0 sp=0xc000695e48 pc=0x7f4d97e733ff
   runtime.chanrecv1(0xc0000eec00?, 0xc000695fa8?)
           /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000695ee8 sp=0xc000695ec0 pc=0x7f4d97e73012
   github.com/caddyserver/caddy/v2.trapSignalsCrossPlatform.func1()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap.go:43 +0xe5 fp=0xc000695fe0 sp=0xc000695ee8 pc=0x7f4d984cace5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000695fe8 sp=0xc000695fe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2.trapSignalsCrossPlatform in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap.go:38 +0x1a
   
   goroutine 47 gp=0xc000822a80 m=nil [chan receive]:
   runtime.gopark(0x7f4d984a97a5?, 0x7f4d9e1c26e0?, 0x1?, 0x0?, 0xc0000d4da8?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d4d18 sp=0xc0000d4cf8 pc=0x7f4d97eaa84e
   runtime.chanrecv(0xc00051f2c0, 0xc0000d4f70, 0x1)
           /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc0000d4d90 sp=0xc0000d4d18 pc=0x7f4d97e733ff
   runtime.chanrecv2(0xc00051f2c0?, 0xc000696780?)
           /usr/local/go/src/runtime/chan.go:447 +0x12 fp=0xc0000d4db8 sp=0xc0000d4d90 pc=0x7f4d97e73032
   github.com/caddyserver/caddy/v2.trapSignalsPosix.func1()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap_posix.go:35 +0x105 fp=0xc0000d4fe0 sp=0xc0000d4db8 pc=0x7f4d984ca585
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d4fe8 sp=0xc0000d4fe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2.trapSignalsPosix in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap_posix.go:31 +0x1a
   
   goroutine 68 gp=0xc000682700 m=nil [select, locked to thread]:
   runtime.gopark(0xc000691fa8?, 0x2?, 0x60?, 0x1e?, 0xc000691f94?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000691e38 sp=0xc000691e18 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc000691fa8, 0xc000691f90, 0x0?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000691f58 sp=0xc000691e38 pc=0x7f4d97ebc725
   runtime.ensureSigM.func1()
           /usr/local/go/src/runtime/signal_unix.go:1034 +0x1a5 fp=0xc000691fe0 sp=0xc000691f58 pc=0x7f4d97ed96c5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000691fe8 sp=0xc000691fe0 pc=0x7f4d97ee2d81
   created by runtime.ensureSigM in goroutine 46
           /usr/local/go/src/runtime/signal_unix.go:1017 +0xc8
   
   goroutine 69 gp=0xc0006828c0 m=12 mp=0xc000580808 [syscall]:
   runtime.notetsleepg(0x7f4d9f2e97c0, 0xffffffffffffffff)
           /usr/local/go/src/runtime/lock_futex.go:246 +0x29 fp=0xc0006927a0 sp=0xc000692778 pc=0x7f4d97e79489
   os/signal.signal_recv()
           /usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc0006927c0 sp=0xc0006927a0 pc=0x7f4d97edefe9
   os/signal.loop()
           /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0006927e0 sp=0xc0006927c0 pc=0x7f4d984a9f53
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006927e8 sp=0xc0006927e0 pc=0x7f4d97ee2d81
   created by os/signal.Notify.func1.1 in goroutine 46
           /usr/local/go/src/os/signal/signal.go:151 +0x1f
   
   goroutine 84 gp=0xc000822fc0 m=nil [IO wait]:
   runtime.gopark(0x7f4d97e88096?, 0x7f4d9f2ea7e0?, 0xe8?, 0x7b?, 0x7f4d97e7ac25?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d7b50 sp=0xc0000d7b30 pc=0x7f4d97eaa84e
   runtime.netpollblock(0x7f4d97e7aabb?, 0x97e704c6?, 0x4d?)
           /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0000d7b88 sp=0xc0000d7b50 pc=0x7f4d97ea2937
   internal/poll.runtime_pollWait(0x7f4d508e26d0, 0x72)
           /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0000d7ba8 sp=0xc0000d7b88 pc=0x7f4d97edcda5
   internal/poll.(*pollDesc).wait(0x3?, 0xc000429f80?, 0x0)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0000d7bd0 sp=0xc0000d7ba8 pc=0x7f4d97f964e7
   internal/poll.(*pollDesc).waitRead(...)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).Accept(0xc000455e80)
           /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc0000d7c78 sp=0xc0000d7bd0 pc=0x7f4d97f9b88c
   net.(*netFD).accept(0xc000455e80)
           /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc0000d7d30 sp=0xc0000d7c78 pc=0x7f4d98046f89
   net.(*TCPListener).accept(0xc000494da0)
           /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc0000d7d58 sp=0xc0000d7d30 pc=0x7f4d980612de
   net.(*TCPListener).Accept(0xc000494da0)
           /usr/local/go/src/net/tcpsock.go:327 +0x30 fp=0xc0000d7d88 sp=0xc0000d7d58 pc=0x7f4d980604d0
   github.com/caddyserver/caddy/v2.(*deleteListener).Accept(0x7f4d9f1510e0?)
           <autogenerated>:1 +0x24 fp=0xc0000d7da0 sp=0xc0000d7d88 pc=0x7f4d984cf644
   net/http.(*onceCloseListener).Accept(0x7f4d9e410628?)
           <autogenerated>:1 +0x24 fp=0xc0000d7db8 sp=0xc0000d7da0 pc=0x7f4d981c91a4
   net/http.(*Server).Serve(0xc000409680, {0x7f4d501b5b98, 0xc000494dc0})
           /usr/local/go/src/net/http/server.go:3255 +0x33e fp=0xc0000d7ee8 sp=0xc0000d7db8 pc=0x7f4d981a55be
   github.com/caddyserver/caddy/v2.replaceLocalAdminServer.func2()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/admin.go:449 +0x133 fp=0xc0000d7fe0 sp=0xc0000d7ee8 pc=0x7f4d984ac8d3
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d7fe8 sp=0xc0000d7fe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2.replaceLocalAdminServer in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/admin.go:445 +0x4b2
   
   goroutine 85 gp=0xc000823180 m=nil [select]:
   runtime.gopark(0xc0006c6f40?, 0x3?, 0x0?, 0x0?, 0xc0006c6e3a?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006c6ce0 sp=0xc0006c6cc0 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc0006c6f40, 0xc0006c6e34, 0x1?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0006c6e00 sp=0xc0006c6ce0 pc=0x7f4d97ebc725
   github.com/caddyserver/certmagic.(*Cache).maintainAssets(0xc000224280, 0x0)
           /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/maintain.go:69 +0x31f fp=0xc0006c6fc0 sp=0xc0006c6e00 pc=0x7f4d9832233f
   github.com/caddyserver/certmagic.NewCache.gowrap1()
           /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/cache.go:127 +0x25 fp=0xc0006c6fe0 sp=0xc0006c6fc0 pc=0x7f4d982ffda5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c6fe8 sp=0xc0006c6fe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/certmagic.NewCache in goroutine 1
           /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/cache.go:127 +0x1f6
   
   goroutine 87 gp=0xc000823a40 m=nil [select]:
   runtime.gopark(0xc0006c7fa0?, 0x2?, 0x8?, 0x10?, 0xc0006c7f8c?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006c7e30 sp=0xc0006c7e10 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc0006c7fa0, 0xc0006c7f88, 0x7f4d97e722f7?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0006c7f50 sp=0xc0006c7e30 pc=0x7f4d97ebc725
   github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy.(*metricsUpstreamsHealthyUpdater).Init.func1()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/reverseproxy/metrics.go:61 +0xca fp=0xc0006c7fe0 sp=0xc0006c7f50 pc=0x7f4d98f5972a
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c7fe8 sp=0xc0006c7fe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy.(*metricsUpstreamsHealthyUpdater).Init in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/reverseproxy/metrics.go:48 +0x4f
   
   goroutine 88 gp=0xc000823c00 m=nil [select]:
   runtime.gopark(0xc0006ddfa8?, 0x2?, 0x0?, 0x0?, 0xc0006ddf9c?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000a3ee40 sp=0xc000a3ee20 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc000a3efa8, 0xc0006ddf98, 0x0?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000a3ef60 sp=0xc000a3ee40 pc=0x7f4d97ebc725
   github.com/caddyserver/caddy/v2/modules/caddytls.(*TLS).keepStorageClean.func1()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddytls/tls.go:540 +0x93 fp=0xc000a3efe0 sp=0xc000a3ef60 pc=0x7f4d98b02c93
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000a3efe8 sp=0xc000a3efe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2/modules/caddytls.(*TLS).keepStorageClean in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddytls/tls.go:532 +0xe5
   
   goroutine 32 gp=0xc000a00700 m=nil [select]:
   runtime.gopark(0xc00011ff70?, 0x2?, 0xa0?, 0xfe?, 0xc00011ff64?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011fe08 sp=0xc00011fde8 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc00011ff70, 0xc00011ff60, 0x0?, 0x0, 0xc00011ff10?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc00011ff28 sp=0xc00011fe08 pc=0x7f4d97ebc725
   github.com/caddyserver/caddy/v2/modules/caddypki.(*PKI).maintenance(0xc00068f740)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddypki/maintain.go:38 +0xef fp=0xc00011ffc8 sp=0xc00011ff28 pc=0x7f4d98aee08f
   github.com/caddyserver/caddy/v2/modules/caddypki.(*PKI).Start.gowrap1()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddypki/pki.go:113 +0x25 fp=0xc00011ffe0 sp=0xc00011ffc8 pc=0x7f4d98aef745
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011ffe8 sp=0xc00011ffe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2/modules/caddypki.(*PKI).Start in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddypki/pki.go:113 +0x47f
   
   goroutine 33 gp=0xc000a00a80 m=nil [IO wait]:
   runtime.gopark(0xc00050e200?, 0xc0000d29c8?, 0xd8?, 0x3d?, 0x7f4d98293805?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d2978 sp=0xc0000d2958 pc=0x7f4d97eaa84e
   runtime.netpollblock(0xc0009800c4?, 0x9e3fad28?, 0x4d?)
           /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0000d29b0 sp=0xc0000d2978 pc=0x7f4d97ea2937
   internal/poll.runtime_pollWait(0x7f4d508e24e0, 0x72)
           /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0000d29d0 sp=0xc0000d29b0 pc=0x7f4d97edcda5
   internal/poll.(*pollDesc).wait(0xc0003a8680?, 0x0?, 0x0)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0000d29f8 sp=0xc0000d29d0 pc=0x7f4d97f964e7
   internal/poll.(*pollDesc).waitRead(...)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).RawRead(0xc0003a8680, 0xc0008fe000)
           /usr/local/go/src/internal/poll/fd_unix.go:708 +0x12a fp=0xc0000d2a58 sp=0xc0000d29f8 pc=0x7f4d97f9c50a
   net.(*rawConn).Read(0xc00040eec8, 0xc00050e200?)
           /usr/local/go/src/net/rawconn.go:44 +0x36 fp=0xc0000d2a90 sp=0xc0000d2a58 pc=0x7f4d9805c356
   golang.org/x/net/internal/socket.(*syscaller).recvmmsg(0xc0004420d8, {0x7f4d9e40b500?, 0xc00040eec8?}, {0xc00050e200?, 0x7f4d9e3f39f8?, 0x0?}, 0x7f4d97e88096?)
           /go/pkg/mod/golang.org/x/net@v0.22.0/internal/socket/mmsghdr_unix.go:120 +0x70 fp=0xc0000d2ac0 sp=0xc0000d2a90 pc=0x7f4d98293c30
   golang.org/x/net/internal/socket.(*Conn).recvMsgs(0xc00052aae0, {0xc0003c5808, 0x8, 0x8}, 0x0)
           /go/pkg/mod/golang.org/x/net@v0.22.0/internal/socket/rawconn_mmsg.go:24 +0x176 fp=0xc0000d2b80 sp=0xc0000d2ac0 pc=0x7f4d98294e16
   golang.org/x/net/internal/socket.(*Conn).RecvMsgs(...)
           /go/pkg/mod/golang.org/x/net@v0.22.0/internal/socket/socket.go:267
   golang.org/x/net/ipv4.(*payloadHandler).ReadBatch(0xc0009744c0, {0xc0003c5808?, 0x0?, 0x2?}, 0x0?)
           /go/pkg/mod/golang.org/x/net@v0.22.0/ipv4/batch.go:80 +0x57 fp=0xc0000d2bf0 sp=0xc0000d2b80 pc=0x7f4d9829c557
   golang.org/x/net/ipv4.(*PacketConn).ReadBatch(0x7f4d9f255040?, {0xc0003c5808?, 0x0?, 0x0?}, 0x0?)
           <autogenerated>:1 +0x27 fp=0xc0000d2c28 sp=0xc0000d2bf0 pc=0x7f4d982a2087
   github.com/quic-go/quic-go.(*oobConn).ReadPacket(0xc000826120)
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/sys_conn_oob.go:174 +0x6e2 fp=0xc0000d2dd0 sp=0xc0000d2c28 pc=0x7f4d98471742
   github.com/quic-go/quic-go.(*Transport).listen(0xc0004093b0, {0x7f4d9e416d90, 0xc000826120})
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:337 +0x149 fp=0xc0000d2fb8 sp=0xc0000d2dd0 pc=0x7f4d98473449
   github.com/quic-go/quic-go.(*Transport).init.func1.gowrap1()
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:241 +0x28 fp=0xc0000d2fe0 sp=0xc0000d2fb8 pc=0x7f4d98472cc8
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d2fe8 sp=0xc0000d2fe0 pc=0x7f4d97ee2d81
   created by github.com/quic-go/quic-go.(*Transport).init.func1 in goroutine 1
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:241 +0x43b
   
   goroutine 98 gp=0xc000a00c40 m=nil [select]:
   runtime.gopark(0xc00011c6d0?, 0x3?, 0x0?, 0x0?, 0xc00011c672?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011c4d0 sp=0xc00011c4b0 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc00011c6d0, 0xc00011c66c, 0x0?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc00011c5f0 sp=0xc00011c4d0 pc=0x7f4d97ebc725
   github.com/quic-go/quic-go.(*Transport).runSendQueue(0xc0004093b0)
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:266 +0x107 fp=0xc00011c7c8 sp=0xc00011c5f0 pc=0x7f4d98472e07
   github.com/quic-go/quic-go.(*Transport).init.func1.gowrap2()
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:242 +0x25 fp=0xc00011c7e0 sp=0xc00011c7c8 pc=0x7f4d98472c65
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011c7e8 sp=0xc00011c7e0 pc=0x7f4d97ee2d81
   created by github.com/quic-go/quic-go.(*Transport).init.func1 in goroutine 1
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:242 +0x478
   
   goroutine 99 gp=0xc000a00fc0 m=nil [select]:
   runtime.gopark(0xc00011cf90?, 0x2?, 0x0?, 0x0?, 0xc00011ce9c?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d3cf8 sp=0xc0000d3cd8 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc0000d3f90, 0xc00011ce98, 0x0?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0000d3e18 sp=0xc0000d3cf8 pc=0x7f4d97ebc725
   github.com/quic-go/quic-go.(*baseServer).run(0xc0001c3e60)
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:286 +0x129 fp=0xc0000d3fc8 sp=0xc0000d3e18 pc=0x7f4d98466249
   github.com/quic-go/quic-go.newServer.gowrap1()
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:272 +0x25 fp=0xc0000d3fe0 sp=0xc0000d3fc8 pc=0x7f4d984660e5
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d3fe8 sp=0xc0000d3fe0 pc=0x7f4d97ee2d81
   created by github.com/quic-go/quic-go.newServer in goroutine 1
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:272 +0x496
   
   goroutine 100 gp=0xc000a01180 m=nil [select]:
   runtime.gopark(0xc0006cae78?, 0x5?, 0x0?, 0x0?, 0xc0006cad86?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006cabd8 sp=0xc0006cabb8 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc0006cae78, 0xc0006cad7c, 0x0?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0006cacf8 sp=0xc0006cabd8 pc=0x7f4d97ebc725
   github.com/quic-go/quic-go.(*baseServer).runSendQueue(0xc0001c3e60)
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:299 +0x1bf fp=0xc0006cafc8 sp=0xc0006cacf8 pc=0x7f4d9846653f
   github.com/quic-go/quic-go.newServer.gowrap2()
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:273 +0x25 fp=0xc0006cafe0 sp=0xc0006cafc8 pc=0x7f4d98466085
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006cafe8 sp=0xc0006cafe0 pc=0x7f4d97ee2d81
   created by github.com/quic-go/quic-go.newServer in goroutine 1
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:273 +0x4d6
   
   goroutine 101 gp=0xc000a01340 m=nil [select]:
   runtime.gopark(0xc0000d8ea8?, 0x3?, 0x60?, 0x0?, 0xc0000d8e92?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d8d38 sp=0xc0000d8d18 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc0000d8ea8, 0xc0000d8e8c, 0xc0000d8e80?, 0x0, 0xc00083a1c8?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0000d8e58 sp=0xc0000d8d38 pc=0x7f4d97ebc725
   github.com/quic-go/quic-go.(*baseServer).accept(0xc0001c3e60, {0x7f4d9e410778, 0xc000974460})
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:321 +0x9d fp=0xc0000d8ee8 sp=0xc0000d8e58 pc=0x7f4d984667fd
   github.com/quic-go/quic-go.(*EarlyListener).Accept(...)
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:160
   github.com/caddyserver/caddy/v2.(*fakeCloseQuicListener).Accept(0xc000976360, {0xc0008fe020?, 0x0?})
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/listeners.go:677 +0x31 fp=0xc0000d8f30 sp=0xc0000d8ee8 pc=0x7f4d984c29b1
   github.com/quic-go/quic-go/http3.(*Server).ServeListener(0xc00083a160, {0x7f4d9e40b350, 0xc000976360})
           /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/http3/server.go:283 +0x11b fp=0xc0000d8fb8 sp=0xc0000d8f30 pc=0x7f4d9848d0db
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).serveHTTP3.gowrap1()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:576 +0x28 fp=0xc0000d8fe0 sp=0xc0000d8fb8 pc=0x7f4d98ceb668
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d8fe8 sp=0xc0000d8fe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).serveHTTP3 in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:576 +0x339
   
   goroutine 102 gp=0xc000a01500 m=nil [IO wait]:
   runtime.gopark(0xc00006ef01?, 0xc000a42b28?, 0x55?, 0x12?, 0xc000a42a88?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000a42a50 sp=0xc000a42a30 pc=0x7f4d97eaa84e
   runtime.netpollblock(0x0?, 0x97e704c6?, 0x4d?)
           /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000a42a88 sp=0xc000a42a50 pc=0x7f4d97ea2937
   internal/poll.runtime_pollWait(0x7f4d508e25d8, 0x72)
           /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000a42aa8 sp=0xc000a42a88 pc=0x7f4d97edcda5
   internal/poll.(*pollDesc).wait(0x7?, 0x0?, 0x0)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000a42ad0 sp=0xc000a42aa8 pc=0x7f4d97f964e7
   internal/poll.(*pollDesc).waitRead(...)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).Accept(0xc0003a8600)
           /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc000a42b78 sp=0xc000a42ad0 pc=0x7f4d97f9b88c
   net.(*netFD).accept(0xc0003a8600)
           /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc000a42c30 sp=0xc000a42b78 pc=0x7f4d98046f89
   net.(*TCPListener).accept(0xc00052aa40)
           /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc000a42c58 sp=0xc000a42c30 pc=0x7f4d980612de
   net.(*TCPListener).Accept(0xc00052aa40)
           /usr/local/go/src/net/tcpsock.go:327 +0x30 fp=0xc000a42c88 sp=0xc000a42c58 pc=0x7f4d980604d0
   github.com/caddyserver/caddy/v2.(*deleteListener).Accept(0xc0003624dd?)
           <autogenerated>:1 +0x24 fp=0xc000a42ca0 sp=0xc000a42c88 pc=0x7f4d984cf644
   crypto/tls.(*listener).Accept(0xc0008d2180)
           /usr/local/go/src/crypto/tls/tls.go:66 +0x27 fp=0xc000a42ce0 sp=0xc000a42ca0 pc=0x7f4d980f7327
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*http2Listener).Accept(0xc000976390)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/http2listener.go:35 +0x5e fp=0xc000a42e70 sp=0xc000a42ce0 pc=0x7f4d98cd433e
   net/http.(*onceCloseListener).Accept(0xc000442900?)
           <autogenerated>:1 +0x24 fp=0xc000a42e88 sp=0xc000a42e70 pc=0x7f4d981c91a4
   net/http.(*Server).Serve(0xc0004091d0, {0x7f4d9e40c3a0, 0xc000976390})
           /usr/local/go/src/net/http/server.go:3255 +0x33e fp=0xc000a42fb8 sp=0xc000a42e88 pc=0x7f4d981a55be
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start.gowrap1()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x28 fp=0xc000a42fe0 sp=0xc000a42fb8 pc=0x7f4d98cc6a48
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000a42fe8 sp=0xc000a42fe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x1b95
   
   goroutine 103 gp=0xc000a016c0 m=nil [IO wait]:
   runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000a43c20 sp=0xc000a43c00 pc=0x7f4d97eaa84e
   runtime.netpollblock(0x0?, 0x97e704c6?, 0x4d?)
           /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000a43c58 sp=0xc000a43c20 pc=0x7f4d97ea2937
   internal/poll.runtime_pollWait(0x7f4d508e23e8, 0x72)
           /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000a43c78 sp=0xc000a43c58 pc=0x7f4d97edcda5
   internal/poll.(*pollDesc).wait(0x9?, 0xc000a39f80?, 0x0)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000a43ca0 sp=0xc000a43c78 pc=0x7f4d97f964e7
   internal/poll.(*pollDesc).waitRead(...)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).Accept(0xc0003a8780)
           /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc000a43d48 sp=0xc000a43ca0 pc=0x7f4d97f9b88c
   net.(*netFD).accept(0xc0003a8780)
           /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc000a43e00 sp=0xc000a43d48 pc=0x7f4d98046f89
   net.(*TCPListener).accept(0xc00052abe0)
           /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc000a43e28 sp=0xc000a43e00 pc=0x7f4d980612de
   net.(*TCPListener).Accept(0xc00052abe0)
           /usr/local/go/src/net/tcpsock.go:327 +0x30 fp=0xc000a43e58 sp=0xc000a43e28 pc=0x7f4d980604d0
   github.com/caddyserver/caddy/v2.(*deleteListener).Accept(0x7f4d9f1510e0?)
           <autogenerated>:1 +0x24 fp=0xc000a43e70 sp=0xc000a43e58 pc=0x7f4d984cf644
   net/http.(*onceCloseListener).Accept(0x7f4d9e410628?)
           <autogenerated>:1 +0x24 fp=0xc000a43e88 sp=0xc000a43e70 pc=0x7f4d981c91a4
   net/http.(*Server).Serve(0xc0004094a0, {0x7f4d501b5b98, 0xc00052ac20})
           /usr/local/go/src/net/http/server.go:3255 +0x33e fp=0xc000a43fb8 sp=0xc000a43e88 pc=0x7f4d981a55be
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start.gowrap1()
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x28 fp=0xc000a43fe0 sp=0xc000a43fb8 pc=0x7f4d98cc6a48
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000a43fe8 sp=0xc000a43fe0 pc=0x7f4d97ee2d81
   created by github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start in goroutine 1
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x1b95
   
   goroutine 54 gp=0xc000a01a40 m=nil [IO wait]:
   runtime.gopark(0xcdb4da524ae8b237?, 0x50263a591be2d575?, 0x37?, 0xb2?, 0xb?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006c4818 sp=0xc0006c47f8 pc=0x7f4d97eaa84e
   runtime.netpollblock(0x7f4d97f07378?, 0x97e704c6?, 0x4d?)
           /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0006c4850 sp=0xc0006c4818 pc=0x7f4d97ea2937
   internal/poll.runtime_pollWait(0x7f4d508e22f0, 0x72)
           /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0006c4870 sp=0xc0006c4850 pc=0x7f4d97edcda5
   internal/poll.(*pollDesc).wait(0xc00036a400?, 0xc0001ce900?, 0x0)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006c4898 sp=0xc0006c4870 pc=0x7f4d97f964e7
   internal/poll.(*pollDesc).waitRead(...)
           /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
   internal/poll.(*FD).Read(0xc00036a400, {0xc0001ce900, 0x900, 0x900})
           /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc0006c4930 sp=0xc0006c4898 pc=0x7f4d97f977da
   net.(*netFD).Read(0xc00036a400, {0xc0001ce900?, 0x7f4d508f2958?, 0xc000153938?})
           /usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc0006c4978 sp=0xc0006c4930 pc=0x7f4d98044fa5
   net.(*conn).Read(0xc0000c6960, {0xc0001ce900?, 0xc0006c4a58?, 0x7f4d97e7aabb?})
           /usr/local/go/src/net/net.go:179 +0x45 fp=0xc0006c49c0 sp=0xc0006c4978 pc=0x7f4d98058465
   net.(*TCPConn).Read(0xc000362540?, {0xc0001ce900?, 0x7f4d97e7c7db?, 0x6c4b30?})
           <autogenerated>:1 +0x25 fp=0xc0006c49f0 sp=0xc0006c49c0 pc=0x7f4d980697e5
   crypto/tls.(*atLeastReader).Read(0xc000153938, {0xc0001ce900?, 0x0?, 0xc000153938?})
           /usr/local/go/src/crypto/tls/conn.go:806 +0x3b fp=0xc0006c4a38 sp=0xc0006c49f0 pc=0x7f4d980bbefb
   bytes.(*Buffer).ReadFrom(0xc000362630, {0x7f4d9e3fbca0, 0xc000153938})
           /usr/local/go/src/bytes/buffer.go:211 +0x98 fp=0xc0006c4a90 sp=0xc0006c4a38 pc=0x7f4d97ef94f8
   crypto/tls.(*Conn).readFromUntil(0xc000362388, {0x7f4d9e3fc0e0, 0xc0000c6960}, 0xc0006c4aa0?)
           /usr/local/go/src/crypto/tls/conn.go:828 +0xde fp=0xc0006c4ac8 sp=0xc0006c4a90 pc=0x7f4d980bc0de
   crypto/tls.(*Conn).readRecordOrCCS(0xc000362388, 0x0)
           /usr/local/go/src/crypto/tls/conn.go:626 +0x3cf fp=0xc0006c4d48 sp=0xc0006c4ac8 pc=0x7f4d980b91ef
   crypto/tls.(*Conn).readRecord(...)
           /usr/local/go/src/crypto/tls/conn.go:588
   crypto/tls.(*Conn).Read(0xc000362388, {0xc0005fa9e0, 0x9, 0x7f4d97ebcc26?})
           /usr/local/go/src/crypto/tls/conn.go:1370 +0x156 fp=0xc0006c4db8 sp=0xc0006c4d48 pc=0x7f4d980bfa96
   io.ReadAtLeast({0x7f4d504aed00, 0xc000362388}, {0xc0005fa9e0, 0x9, 0x9}, 0x9)
           /usr/local/go/src/io/io.go:335 +0x90 fp=0xc0006c4e00 sp=0xc0006c4db8 pc=0x7f4d97ef0930
   io.ReadFull(...)
           /usr/local/go/src/io/io.go:354
   golang.org/x/net/http2.readFrameHeader({0xc0005fa9e0, 0x9, 0x0?}, {0x7f4d504aed00?, 0xc000362388?})
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/frame.go:237 +0x65 fp=0xc0006c4e50 sp=0xc0006c4e00 pc=0x7f4d985b06c5
   golang.org/x/net/http2.(*Framer).ReadFrame(0xc0005fa9a0)
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/frame.go:498 +0x85 fp=0xc0006c4ef8 sp=0xc0006c4e50 pc=0x7f4d985b0e05
   golang.org/x/net/http2.(*serverConn).readFrames(0xc00019fba0)
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:820 +0x87 fp=0xc0006c4fc8 sp=0xc0006c4ef8 pc=0x7f4d985bbac7
   golang.org/x/net/http2.(*serverConn).serve.gowrap7()
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:932 +0x25 fp=0xc0006c4fe0 sp=0xc0006c4fc8 pc=0x7f4d985bd125
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c4fe8 sp=0xc0006c4fe0 pc=0x7f4d97ee2d81
   created by golang.org/x/net/http2.(*serverConn).serve in goroutine 80
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:932 +0x56a
   
   goroutine 80 gp=0xc000822540 m=nil [select]:
   runtime.gopark(0xc0006257c8?, 0x5?, 0xf5?, 0x29?, 0xc00062543e?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006252d0 sp=0xc0006252b0 pc=0x7f4d97eaa84e
   runtime.selectgo(0xc0006257c8, 0xc000625434, 0xc000d009c0?, 0x0, 0x0?, 0x1)
           /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0006253f0 sp=0xc0006252d0 pc=0x7f4d97ebc725
   golang.org/x/net/http2.(*serverConn).serve(0xc00019fba0)
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:940 +0x897 fp=0xc000625868 sp=0xc0006253f0 pc=0x7f4d985bc937
   golang.org/x/net/http2.(*Server).ServeConn(0xc000974370, {0x7f4d9e41aea0, 0xc000362388}, 0xc000625b30)
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:531 +0xbad fp=0xc000625b08 sp=0xc000625868 pc=0x7f4d985bab8d
   golang.org/x/net/http2.ConfigureServer.func1(0xc0004091d0, 0xc000362388, {0x7f4d9e3fc740, 0xc00052a840})
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:321 +0x125 fp=0xc000625b90 sp=0xc000625b08 pc=0x7f4d985b9f25
   net/http.(*conn).serve(0xc000442900, {0x7f4d9e410740, 0xc000b85aa0})
           /usr/local/go/src/net/http/server.go:1947 +0x12f3 fp=0xc000625fb8 sp=0xc000625b90 pc=0x7f4d981a1173
   net/http.(*Server).Serve.gowrap3()
           /usr/local/go/src/net/http/server.go:3285 +0x28 fp=0xc000625fe0 sp=0xc000625fb8 pc=0x7f4d981a59a8
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000625fe8 sp=0xc000625fe0 pc=0x7f4d97ee2d81
   created by net/http.(*Server).Serve in goroutine 102
           /usr/local/go/src/net/http/server.go:3285 +0x4b4
   
   goroutine 114 gp=0xc0009896c0 m=18 mp=0xc00052d808 [syscall, locked to thread]:
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c9fe8 sp=0xc0006c9fe0 pc=0x7f4d97ee2d81
   
   goroutine 131 gp=0xc000989c00 m=19 mp=0xc000600808 [syscall, locked to thread]:
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c5fe8 sp=0xc0006c5fe0 pc=0x7f4d97ee2d81
   
   goroutine 132 gp=0xc000bb28c0 m=20 mp=0xc000680808 [syscall, locked to thread]:
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000ab3fe8 sp=0xc000ab3fe0 pc=0x7f4d97ee2d81
   
   goroutine 181 gp=0xc000683340 m=23 mp=0xc0009f0008 [syscall, locked to thread]:
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000bb9fe8 sp=0xc000bb9fe0 pc=0x7f4d97ee2d81
   
   goroutine 384 gp=0xc000a92540 m=nil [chan receive]:
   runtime.gopark(0x2?, 0xc0008344e0?, 0x51?, 0x0?, 0xc0006202e0?)
           /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000620290 sp=0xc000620270 pc=0x7f4d97eaa84e
   runtime.chanrecv(0xc000834480, 0x0, 0x1)
           /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc000620308 sp=0xc000620290 pc=0x7f4d97e733ff
   runtime.chanrecv1(0xc000620390?, 0xc000620368?)
           /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000620330 sp=0xc000620308 pc=0x7f4d97e73012
   github.com/dunglas/frankenphp.ServeHTTP({0x7f4d9e40c310, 0xc0004f8870}, 0xc000d06b40)
           /go/src/app/frankenphp.go:465 +0x20e fp=0xc0006203f0 sp=0xc000620330 pc=0x7f4d990bacae
   github.com/dunglas/frankenphp/caddy.FrankenPHPModule.ServeHTTP({{0x7f4d9ac2dfcd, 0x10}, {0xc0003733f0, 0x1, 0x1}, 0xc000125534, 0x0, 0xc00050d780}, {0x7f4d9e40c310, 0xc0004f8870}, ...)
           /go/src/app/caddy/caddy.go:275 +0x4ac fp=0xc000620790 sp=0xc0006203f0 pc=0x7f4d991c110c
   github.com/dunglas/frankenphp/caddy.(*FrankenPHPModule).ServeHTTP(0xc000620870?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0x7f4d9e23dce0?, {0x7f4d9e3fe340?, 0xc000053060?})
           <autogenerated>:1 +0xe5 fp=0xc000620848 sp=0xc000620790 pc=0x7f4d991c7265
   github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40c310?, 0xc0004f8870?}, 0xc000314c40?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000620888 sp=0xc000620848 pc=0x7f4d98ce887a
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3fe340?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0xc000d067e0?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc0006208b0 sp=0xc000620888 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40c310, 0xc0004f8870}, 0xc000d067e0)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc000620928 sp=0xc0006208b0 pc=0x7f4d98ce85e5
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x0?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0xc000768e80?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620950 sp=0xc000620928 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite.Rewrite.ServeHTTP({{0x0, 0x0}, {0xc0001176e0, 0x1d}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/rewrite/rewrite.go:137 +0x3f3 fp=0xc000620ad8 sp=0xc000620950 pc=0x7f4d98f1bc53
   github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite.(*Rewrite).ServeHTTP(0xc000620c28?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0x7f4d9e23dce0?, {0x7f4d9e3fe340?, 0xc000053080?})
           <autogenerated>:1 +0xd6 fp=0xc000620c00 sp=0xc000620ad8 pc=0x7f4d98f1dc76
   github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40c310?, 0xc0004f8870?}, 0xc00050d000?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000620c40 sp=0xc000620c00 pc=0x7f4d98ce887a
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3fe340?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0xc000d067e0?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620c68 sp=0xc000620c40 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40c310, 0xc0004f8870}, 0xc000d067e0)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc000620ce0 sp=0xc000620c68 pc=0x7f4d98ce85e5
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e410740?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0xe?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620d08 sp=0xc000620ce0 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40c310, 0xc0004f8870}, 0xc000d067e0)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:268 +0x244 fp=0xc000620d80 sp=0xc000620d08 pc=0x7f4d98ce8504
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0009b9750?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0x2?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620da8 sp=0xc000620d80 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp/encode.(*Encode).ServeHTTP(0xc000314b00, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0, {0x7f4d9e3fe340, 0xc0000530c0})
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/encode/encode.go:138 +0x252 fp=0xc000620e10 sp=0xc000620da8 pc=0x7f4d98d313d2
   github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc00078c120?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000620e50 sp=0xc000620e10 pc=0x7f4d98ce887a
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0007c8ee0?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x4?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620e78 sp=0xc000620e50 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.VarsMiddleware.ServeHTTP(0xc00015eea0, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0, {0x7f4d9e3fe340, 0xc000d00d20})
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/vars.go:61 +0x273 fp=0xc000620f50 sp=0xc000620e78 pc=0x7f4d98cf0b13
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*VarsMiddleware).ServeHTTP(0xc0007c8fb8?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x7f4d9e23dce0?, {0x7f4d9e3fe340?, 0xc000d00d20?})
           <autogenerated>:1 +0x38 fp=0xc000620f90 sp=0xc000620f50 pc=0x7f4d98cfa238
   github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc0003a4dd0?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000620fd0 sp=0xc000620f90 pc=0x7f4d98ce887a
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3fe340?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000d067e0?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620ff8 sp=0xc000620fd0 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc000621070 sp=0xc000620ff8 pc=0x7f4d98ce85e5
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0001f6a08?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x7f4d9e3fe340?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000621098 sp=0xc000621070 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Subroute).ServeHTTP(0xc000495720, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0, {0x7f4d9e3fe340, 0x7f4d9e3f1eb8})
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/subroute.go:74 +0x67 fp=0xc0006210d8 sp=0xc000621098 pc=0x7f4d98cf0747
   github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000495720?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000621118 sp=0xc0006210d8 pc=0x7f4d98ce887a
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3fe340?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000d067e0?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000621140 sp=0xc000621118 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc0006211b8 sp=0xc000621140 pc=0x7f4d98ce85e5
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e410740?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xe?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc0006211e0 sp=0xc0006211b8 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:268 +0x244 fp=0xc000621258 sp=0xc0006211e0 pc=0x7f4d98ce8504
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3410e0?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc0009b9690?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000621280 sp=0xc000621258 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).enforcementHandler(0x10?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x0?, {0x7f4d9e3fe340?, 0xc000495980?})
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:429 +0x24b fp=0xc000621358 sp=0xc000621280 pc=0x7f4d98cead6b
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Provision.(*Server).wrapPrimaryRoute.func1({0x7f4d9e40ce80?, 0xc0007b2078?}, 0x7f4d97f157cf?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:405 +0x35 fp=0xc000621398 sp=0xc000621358 pc=0x7f4d98cc4115
   github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc00022fb20?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000d067e0?)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc0006213c0 sp=0xc000621398 pc=0x7f4d98ccd469
   github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).ServeHTTP(0xc0002662c8, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d06480)
           /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:341 +0xc47 fp=0xc000621cf0 sp=0xc0006213c0 pc=0x7f4d98ce9b67
   net/http.serverHandler.ServeHTTP({0x80?}, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000152a68?)
           /usr/local/go/src/net/http/server.go:3137 +0x8e fp=0xc000621d20 sp=0xc000621cf0 pc=0x7f4d981a518e
   net/http.initALPNRequest.ServeHTTP({{0x7f4d9e410740?, 0xc000b85ad0?}, 0xc000362388?, {0xc0004091d0?}}, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d06480)
           /usr/local/go/src/net/http/server.go:3745 +0x231 fp=0xc000621eb0 sp=0xc000621d20 pc=0x7f4d981a8351
   net/http.(*initALPNRequest).ServeHTTP(0xc0007db5c0?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x0?)
           <autogenerated>:1 +0x4f fp=0xc000621ef8 sp=0xc000621eb0 pc=0x7f4d981c942f
   net/http.Handler.ServeHTTP-fm({0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc00019fba0?)
           <autogenerated>:1 +0x3b fp=0xc000621f28 sp=0xc000621ef8 pc=0x7f4d981ca55b
   golang.org/x/net/http2.(*serverConn).runHandler(0x7f4d97eb29dd?, 0x0?, 0x0?, 0x0?)
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:2368 +0xbb fp=0xc000621fb0 sp=0xc000621f28 pc=0x7f4d985c42bb
   golang.org/x/net/http2.(*serverConn).scheduleHandler.gowrap1()
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:2303 +0x2c fp=0xc000621fe0 sp=0xc000621fb0 pc=0x7f4d985c3eac
   runtime.goexit({})
           /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000621fe8 sp=0xc000621fe0 pc=0x7f4d97ee2d81
   created by golang.org/x/net/http2.(*serverConn).scheduleHandler in goroutine 80
           /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:2303 +0x21d
   
   rax    0x55
   rbx    0x7f4d43bcc710
   rcx    0x0
   rdx    0xd
   rdi    0x7f4d9b247ffc
   rsi    0x7f4d4462be70
   rbp    0x7f4d9b247ffc
   rsp    0x7f4d455ca638
   r8     0x51
   r9     0x8080808080808001
   r10    0x7f4d9b247ffc
   r11    0x246
   r12    0xe
   r13    0x7f4d43bc9120
   r14    0xa
   r15    0x0
   rip    0x7f4d9ac01a1b
   rflags 0x10246
   cs     0x33
   fs     0x0
   gs     0x0
9. emaia commented on Mar 25, 2024

   

   emaia

   on Mar 25, 2024

   More actions

   Another update: I'm using Laravel Debugbar in my dev env and if I disable it, this crash doesn't happen.
10. jeremyj11 commented on May 29, 2024

    

    jeremyj11

    on May 29, 2024

    Last edited by jeremyj11

    Contributor

    More actions

    I was gettting the same crash, just updated to v1.1.5 and now just get "Segmentation fault" after a page reload and caddy exits.

    Ubuntu 20.04
    Not using docker, downloaded frankenphp-linux-x86_64 from github releases
    Freshly created Laravel 11 app, migrating code over from Laravel 8 app
    Works fine with php-fpm 8.3

    2024/05/29 13:41:51.204 DEBUG http.handlers.rewrite rewrote request {"request": {"remote_ip": "127.0.0.1", "remote_port": "41848", "client_ip": "127.0.0.1", "proto": "HTTP/1.1", "method": "GET", "host": "heycast.test", "uri": "/api/schedule/Agent", "headers": {"Accept": ["application/json, text/plain, */*"], "User-Agent": ["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"], "Sec-Gpc": ["1"], "Accept-Language": ["en-AU,en;q=0.9"], "Origin": ["http://localhost:4200"], "Referer": ["http://localhost:4200/"], "Accept-Encoding": ["gzip, deflate"], "Connection": ["keep-alive"], "Account": ["Agent"], "Authorization": []}}, "method": "GET", "uri": "/index.php"} Segmentation fault

    My caddyfile:

    {
      debug
      frankenphp
      order php_server before file_server
    }
    
    heycast.test:80 {
      root * /home/me/htdocs/mydomain/public
      encode zstd br gzip
      php_server
    }
    

    any advice?
11. dunglas commented on May 29, 2024

    

    dunglas

    on May 29, 2024

    Member

    More actions

    @jeremyj11 would you be able to provide us a minimal reproducer triggering the crash, or to gather a stack trace by following these steps? https://frankenphp.dev/docs/contributing/#debugging-segmentation-faults-with-static-builds

    Thanks!
12. theaungmyatmoe commented on May 29, 2024

    

    theaungmyatmoe

    on May 29, 2024

    More actions

    I am facing the same issue in my latest docker build.

    docker run -it --rm breeze-settings sh
    /var/www/html/apps/settings $ php artisan octane:start --server=frankenphp --host=0.0.0.0 --port=80 --watch
    
       InvalidArgumentException
    
      Unable to determine admin port. Please specify the [--admin-port] option.
    
      at vendor/laravel/octane/src/Commands/StartFrankenPhpCommand.php:234
        230▕         $defaultPort = 2019;
        231▕
        232▕         return tap($defaultPort + ($this->getPort() - 8000), function ($adminPort) {
        233▕             if ($adminPort < 0) {
      ➜ 234▕                 throw new InvalidArgumentException(
        235▕                     'Unable to determine admin port. Please specify the [--admin-port] option.',
        236▕                 );
        237▕             }
        238▕         });
    
          +29 vendor frames
    
      30  artisan:13
          Illuminate\Foundation\Application::handleCommand(Object(Symfony\Component\Console\Input\ArgvInput))
    
    /var/www/html/apps/settings $ php artisan octane:start --server=frankenphp --admin-port=2019 --host=0.0.0.0 --port=80 --watch
    
       INFO  Server running…
    
      Local: http://0.0.0.0:80
    
      Press Ctrl+C to stop the server
    
    
       WARN  server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server
    
       INFO  SIGSEGV: segmentation violation
    /var/www/html/apps/settings $
    
    
    
    
    
    
    
    
    
    
13. theaungmyatmoe commented on May 29, 2024

    

    theaungmyatmoe

    on May 29, 2024

    More actions

    @dunglas You can reproduce with this:

    ARG PHP_VERSION=8.3
    ARG FRANKENPHP_VERSION=latest
    
    FROM dunglas/frankenphp:${FRANKENPHP_VERSION}-php${PHP_VERSION}-alpine
    
    

    Here is supervisor config.

    [program:octane]
    process_name=%(program_name)s_%(process_num)02d
    command=php %(ENV_APP_PATH)s/artisan octane:start --server=frankenphp --host=0.0.0.0 --port=80 --watch
    #command=php %(ENV_APP_PATH)s/artisan octane:start --server=frankenphp --host=localhost --port=443 --admin-port=2019 --https --http-redirect
    user=%(ENV_USER)s
    autostart=true
    autorestart=true
    environment=LARAVEL_OCTANE="1"
    stdout_logfile=/dev/stdout
    stdout_logfile_maxbytes=0
    stderr_logfile=/dev/stderr
    stderr_logfile_maxbytes=0
    
14. theaungmyatmoe commented on May 29, 2024

    

    theaungmyatmoe

    on May 29, 2024

    More actions

    Here is trace log that might help you to debug.

    {"level":"info","ts":1717008503.903582,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
    {"level":"warn","ts":1717008503.904085,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
    {"level":"info","ts":1717008503.9044886,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
    {"level":"info","ts":1717008503.9045434,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
    {"level":"info","ts":1717008503.9045532,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
    {"level":"info","ts":1717008503.9046576,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40001d0c00"}
    {"level":"info","ts":1717008503.9051423,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40001d0c00"}
    Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
    {"level":"info","ts":1717008504.383381,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
    {"level":"warn","ts":1717008504.3852797,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
    {"level":"info","ts":1717008504.3882213,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
    {"level":"info","ts":1717008504.3883011,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
    {"level":"info","ts":1717008504.388307,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
    {"level":"info","ts":1717008504.3886786,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40003f4500"}
    {"level":"info","ts":1717008504.3888597,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40003f4500"}
    Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
    {"level":"info","ts":1717008505.101319,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
    {"level":"warn","ts":1717008505.1025002,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
    {"level":"info","ts":1717008505.104571,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
    {"level":"info","ts":1717008505.104849,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
    {"level":"info","ts":1717008505.104858,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
    {"level":"info","ts":1717008505.1066546,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x4000598480"}
    {"level":"info","ts":1717008505.1066604,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x4000598480"}
    Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
    {"level":"info","ts":1717008505.8572512,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
    {"level":"warn","ts":1717008505.8585649,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
    {"level":"info","ts":1717008505.8616953,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
    {"level":"info","ts":1717008505.8617735,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
    {"level":"info","ts":1717008505.8617826,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
    {"level":"info","ts":1717008505.8622727,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40005dec00"}
    {"level":"info","ts":1717008505.8622785,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40005dec00"}
    Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
    {"level":"info","ts":1717008506.997365,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
    {"level":"warn","ts":1717008506.9984906,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
    {"level":"info","ts":1717008507.0004244,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
    {"level":"info","ts":1717008507.000491,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
    {"level":"info","ts":1717008507.0005004,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
    {"level":"info","ts":1717008507.001825,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40005f5e80"}
    {"level":"info","ts":1717008507.0021057,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40005f5e80"}
    Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
    {"level":"info","ts":1717008508.938355,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
    {"level":"warn","ts":1717008508.9387708,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
    {"level":"info","ts":1717008508.9415684,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
    {"level":"info","ts":1717008508.94166,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
    {"level":"info","ts":1717008508.9416745,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
    {"level":"info","ts":1717008508.9421663,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40003a8500"}
    {"level":"info","ts":1717008508.9421775,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40003a8500"}
    Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
    
    
15. emaia commented on May 29, 2024

    

    emaia

    on May 29, 2024

    More actions

    Another update: I'm using Laravel Debugbar in my dev env and if I disable it, this crash doesn't happen.

    This reduced occurrences of this bug but it continues to happen. Changing Laravel APP_DEBUG to false also helps to minimize occurrences.
16. dunglas commented on May 29, 2024

    

    dunglas

    on May 29, 2024

    Member

    More actions

    @aungmyatmoethegreat there is a known segfault with Alpine/musl and Octane. Could you try to switch to the Debian variant to see if this fixes the issue please?
17. jeremyj11 commented on May 30, 2024

    

    jeremyj11

    on May 30, 2024

    Contributor

    More actions

    @jeremyj11 would you be able to provide us a minimal reproducer triggering the crash, or to gather a stack trace by following these steps? https://frankenphp.dev/docs/contributing/#debugging-segmentation-faults-with-static-builds

    Thanks!

    Here you go:

    
    #0  get_nominal_size (end=0x7f936a6e020c "\223\177", p=0x7f936a6df6d0 "\300\241y\\\223\177") at src/malloc/mallocng/meta.h:163
    #1  __libc_free (p=p@entry=0x7f936a6df6d0) at src/malloc/mallocng/free.c:110
    #2  0x00007f93b5ea4654 in __libc_realloc (p=0x7f936a6df6d0, n=256) at src/malloc/mallocng/realloc.c:49
    #3  0x00007f93b4cb5b5f in OPENSSL_LH_insert ()
    #4  0x00007f93b4cb7607 in namemap_add_name ()
    #5  0x00007f93b4cb8498 in get_legacy_cipher_names ()
    #6  0x00007f93b4cb5f02 in OPENSSL_LH_doall_arg_thunk ()
    #7  0x00007f93b4cd887e in OBJ_NAME_do_all ()
    #8  0x00007f93b4cb7daf in ossl_namemap_stored ()
    #9  0x00007f93b4c9a65e in evp_generic_do_all ()
    #10 0x00007f93b4ca0dca in EVP_KEYMGMT_do_all_provided ()
    #11 0x00007f93b4e47cd5 in OSSL_DECODER_CTX_new_for_pkey ()
    #12 0x00007f93b4ce00f0 in pem_read_bio_key ()
    #13 0x00007f93b4ce0554 in PEM_read_bio_PrivateKey ()
    #14 0x00007f93b41ca9fc in php_openssl_pkey_from_zval (val=0x7f935ca180d0, public_key=0, passphrase=0x7f935fc00268 "", passphrase_len=0, arg_num=1)
        at /go/src/app/dist/static-php-cli/source/php-src/ext/openssl/openssl.c:3659
    #15 0x00007f93b41ce38a in zif_openssl_pkey_get_private (execute_data=0x7f935ca18080, return_value=0x7f935ca18060) at /go/src/app/dist/static-php-cli/source/php-src/ext/openssl/openssl.c:4889
    #16 0x00007f93b3ff5468 in ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER () at /go/src/app/dist/static-php-cli/source/php-src/Zend/zend_vm_execute.h:1337
    #17 execute_ex (ex=0x7f935ca16020) at /go/src/app/dist/static-php-cli/source/php-src/Zend/zend_vm_execute.h:57216
    #18 0x00007f93b40127f0 in zend_execute (op_array=0x7f935ca65000, return_value=0x0) at /go/src/app/dist/static-php-cli/source/php-src/Zend/zend_vm_execute.h:61604
    #19 0x00007f93b3e7e83f in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /go/src/app/dist/static-php-cli/source/php-src/Zend/zend.c:1891
    #20 0x00007f93b3dbec37 in php_execute_script (primary_file=0x7f935f8763e0) at /go/src/app/dist/static-php-cli/source/php-src/main/main.c:2531
    #21 0x00007f93b3d2c0c1 in frankenphp_execute_script (file_name=0x0) at frankenphp.c:851
    #22 0x00007f93b3d29717 in _cgo_b7d6fd74a0c8_Cfunc_frankenphp_execute_script (v=0xc0005d2dd8) at /tmp/go-build/cgo-gcc-prolog:55
    #23 0x00007f93b29c7d61 in runtime.asmcgocall () at /usr/local/go/src/runtime/asm_amd64.s:918
    #24 0x000000c00028f180 in ?? ()
    #25 0x00007f935f876550 in ?? ()
    #26 0x00007f935f8765b4 in ?? ()
    #27 0x0000000000000000 in ?? ()
    
    
18. theaungmyatmoe commented on May 30, 2024

    

    theaungmyatmoe

    on May 30, 2024

    Last edited by theaungmyatmoe

    More actions

    Hi @dunglas , I can now reproduce the issue with alpine.
    After adding $PHPIZE_DEPS and PHP grpc, protobuf extensions, the server cannot started and throw the previous error in both images.

    Here is step to reproduce.

    ARG PHP_VERSION=8.3
    ARG FRANKENPHP_VERSION=latest
    
    FROM dunglas/frankenphp:${FRANKENPHP_VERSION}-php${PHP_VERSION}-alpine
    
    
    RUN apk add --no-cache \
        $PHPIZE_DEPS \ # <- this
        supervisor \
        nodejs \
        npm  \
        libzip-dev \
        libxml2-dev \
        librdkafka-dev \
        g++ \
        make \
        autoconf \
        && apk del autoconf g++ make  \
        && rm -rf /tmp/*  \
        && rm -rf /var/cache/apk/*
    
    
    RUN install-php-extensions \
        redis \
        rdkafka \
        exif \
        pdo_mysql \
        zip \
        sockets \
        pcntl \
        opcache \
        mongodb \
        gd \
        opcache \
        intl \
        bcmath\ 
        grpc \ # <- this
        protobuf <- this

    Supervisor config

    command=php %(ENV_APP_PATH)s/artisan octane:start --server=frankenphp --host=0.0.0.0 --port=80 --admin-port=2019 --watch
    
19. theaungmyatmoe commented on May 30, 2024

    

    theaungmyatmoe

    on May 30, 2024

    More actions

    Franken is fine without grpc and prtobuf extensions and started to work. Then I cannot use grpc. 🥲
    It might be an issue with $PHP_DEPS? You can reference below. Thanks.

    docker-library/php/issues/412
20. dunglas commented on May 30, 2024

    

    dunglas

    on May 30, 2024

    Member

    More actions

    @jeremyj11 , thanks! This is likely this issue: php/php-src#13648

    As a workaround, switching to a FrankenPHP build that uses glibc (like the one we provide in the Debian Docker images) instead of musl should fix the problem.

    👍React with 👍1Reacted by jeremyj11
21. dunglas commented on May 30, 2024

    

    dunglas

    on May 30, 2024

    Member

    More actions

    @jeremyj11 is the code calling openssl_pkey_get_private() public, or could you share it?
22. jeremyj11 commented on May 30, 2024

    

    jeremyj11

    on May 30, 2024

    Last edited by jeremyj11

    Contributor

    More actions

    @jeremyj11 is the code calling openssl_pkey_get_private() public, or could you share it?

    It's public - a fresh Laravel 11 install with Passport should contain it, it's here:
    https://github.com/thephpleague/oauth2-server/blob/master/src/CryptKey.php

    I've confirmed the above is what causes the crash, but from searching my vendor directory the following packages also contain it:
    Symfony\Component\Mime\Crypto\DkimSigner;
    Lcobucci\JWT\Signer\OpenSSL;
    Firebase\JWT\JWT;

    I'd be happy to create a minimal reproduction git repo for you if you like?
23. dunglas commented on May 30, 2024

    

    dunglas

    on May 30, 2024

    Member

    More actions

    @jeremyj11 I think we'll switch to the PHP bug tracker, as @bukka where already investigating a similar issue and has a good knowledge of OpenSSL-related things, but I'm sure a minimal reproducer will help!
24. theaungmyatmoe commented on May 30, 2024

    

    theaungmyatmoe

    on May 30, 2024 via email

    More actions

    I can confirm the same issue with openssl lib in alpine. Aung Myat Moe Technical Lead and Software Architect https://aungmyatmoe.me

    …

    On Thu, 30 May 2024 at 7:30 PM Kévin Dunglas ***@***.***> wrote: @jeremyj11 <https://github.com/jeremyj11> I think we'll switch to the PHP bug tracker, as @bukka <https://github.com/bukka> where already investigating a similar issue and has a good knowledge of OpenSSL-related things, but I'm sure a minimal reproducer will help! — Reply to this email directly, view it on GitHub <#676 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APTVKCP5AA2QGNS6GKNK4SLZE4PIDAVCNFSM6AAAAABE5IRLWOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZZGUYDMNJTHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

    👍React with 👍1Reacted by Kévin Dunglas
25. jeremyj11 commented on May 31, 2024

    

    jeremyj11

    on May 31, 2024

    Contributor

    More actions

    @jeremyj11 , thanks! This is likely this issue: php/php-src#13648

    As a workaround, switching to a FrankenPHP build that uses glibc (like the one we provide in the Debian Docker images) instead of musl should fix the problem.

    I compiled frankenphp from sources and it's working fine now 🎉

    ❤️React with ❤️1Reacted by Kévin Dunglas
26. dunglas

    closed this as not plannedon Jun 10, 2024
27. g-ra commented on Aug 6, 2024

    

    g-ra

    on Aug 6, 2024

    More actions

    @jeremyj11 , thanks! This is likely this issue: php/php-src#13648
    As a workaround, switching to a FrankenPHP build that uses glibc (like the one we provide in the Debian Docker images) instead of musl should fix the problem.

    I compiled frankenphp from sources and it's working fine now 🎉

    can you sgare examples please
    i have same issue #944
28. jeremyj11 commented on Aug 6, 2024

    

    jeremyj11

    on Aug 6, 2024

    Contributor

    More actions

    @jeremyj11 , thanks! This is likely this issue: php/php-src#13648
    As a workaround, switching to a FrankenPHP build that uses glibc (like the one we provide in the Debian Docker images) instead of musl should fix the problem.

    I compiled frankenphp from sources and it's working fine now 🎉

    can you sgare examples please i have same issue #944

    I followed the instruction here https://frankenphp.dev/docs/compile/ for Linux