29 Replies
There are 2 potential reasons for high meme usage:
- allocator is just not releasing memory after usage, which is fine if you have enough mem available
- there is a mem leak
@xamast_51679 could you check on your side what is the mem usage when you are not running any requests
Then could you check the memory profile as you start doing requests
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
Not releasing memory is not an issue, if you want the process to use less mem, then limit it, the allocator will just be more savyy
@xamast_51679 do you have a graph of memory before and after the last request ?
I will profile on my side but it would help to understand on your side as well
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
There are 2 spikes of mem, do they correspond to your requests ?
also near the 0B it was still running right ?
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
So right now, if you start a clean app, do no request, then meme usage is minimal and only increase when requests are made right ?
Can you do one such heavy request on a clean app and give back the mem profile after it ?
From there I should be able to reproduce and investigate, it might just be the allocator working normally
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
Thanks!
Sorry for all the requests but it's really much easier for me with reproduction, thank you for your patience
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
@xamast_51679 can you check if you're displaying virtual or resident memory
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
Is there a way you can reproduce this with fake bytes generated locally so I can run the same script and investigate locally ?
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
Also since this is the app and not the worker, could you let me know what interaction you have with it ?
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
Gotcha, this is really weird then
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
Gotcha
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
with that script of json file, you do not have the same issue?
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
👍
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
looking now
looking into it
yeah I can somewhat reproduce
Unknown User•14mo ago
Message Not Public
Sign In & Join Server To View
On latest, the mem usage should be greatly improved for apps. We do zero copy result for processing the json
The issue is simply that rust serde is very expensive in memory and we were doing deserialization when we didn't need to
Unknown User•2w ago
Message Not Public
Sign In & Join Server To View
it's not enough information for us to help but we can do a review of your setup as part of an EE subscription