Thread 'What is "Waiting for Shared Memory" and why is it hurting my production?'

Author	Message
Zeiss Ikon Send message Joined: 2 Jul 17 Posts: 49	Message 116483 - Posted: 27 Jul 2025, 16:21:53 UTC I appear to have been upgraded to BOINC client 8.2.4 (Debian 12.11, I run sudo apt upgrade without looking too closely at the contents). I just noticed the notification about needing Docker, so I install Podman, but I also notice I have a bunch of GPU tasks (Einstein) "waiting for shared memory" with only one actually running, and a bunch of CPU tasks (MilkyWay) with the same "shared memory" message, but none running. What do I need to fix, and how would I do that? ID: 116483 · Reply Quote

robsmith Volunteer tester Help desk expert Send message Joined: 25 May 09 Posts: 1347	Message 116486 - Posted: 27 Jul 2025, 20:51:43 UTC - in response to Message 116483. Shared memory in memory (RAM) that shared betwwwn the cpu and gpu. Obviously both cannot use the same areas of memory so rhw operating systwm has to shuffle data around, pause jobs, wait for jobs to finish, all of which take time. Sometimes rhe disk can be uswd, and this makes thinga even slower. One way to avoid this problem is to reduce the number of boinc tasks being run as this will reduce the memory demands. There are other ways, that other people may well suggest ID: 116486 · Reply Quote

Keith Myers Volunteer tester Help desk expert Send message Joined: 17 Nov 16 Posts: 910	Message 116488 - Posted: 27 Jul 2025, 23:41:36 UTC I have concern with newcomers to BOINC being confused the messaging about Docker on the new 8.2.4 client. FYI, Docker is not needed for 99% of ALL Boinc projects. And Podman is not needed for 99.9999% of all Boinc projects with only Boinc Central project requiring this unique software configuration. Boinc Central is an alpha testing project and has issues with their apps and producing any significant amount of work. Not interested in testing their alpha software. Maybe in a few years they will have worked out all the bugs. I'd advise removing Docker and Podman and just attach to all the legacy Boinc projects that run perfectly well with the client and don't produce error messages about insufficient memory on the host. ID: 116488 · Reply Quote

Dave Help desk expert Send message Joined: 28 Jun 10 Posts: 2951	Message 116490 - Posted: 28 Jul 2025, 6:06:11 UTC - in response to Message 116488. I'd advise removing Docker and Podman and just attach to all the legacy Boinc projects that run perfectly well with the client and don't produce error messages about insufficient memory on the host. Or leave Podman and Docker there ready for when/if needed but still stick to only attaching projects directly and just remove BOINC Central. It shouldn't get in the way of the other projects. It certainly doesn't with any projects I have tried. ID: 116490 · Reply Quote

Vitalii Koshura Volunteer developer Help desk expert Send message Joined: 29 Mar 17 Posts: 93	Message 116498 - Posted: 28 Jul 2025, 19:54:28 UTC - in response to Message 116488. In reply to Keith Myers's message of 27 Jul 2025: Boinc Central is an alpha testing project and has issues with their apps and producing any significant amount of work. Not interested in testing their alpha software. Maybe in a few years they will have worked out all the bugs. I can't agree here with you. If you pay attention and read some news posted on the BOINC Central forum, you will notice that we have a running real research for several months already. BOINC maintainer. For any insight, check my BOINC Development Blog. ID: 116498 · Reply Quote

Zeiss Ikon Send message Joined: 2 Jul 17 Posts: 49	Message 116500 - Posted: 28 Jul 2025, 22:45:21 UTC - in response to Message 116486. In reply to robsmith's message of 27 Jul 2025: One way to avoid this problem is to reduce the number of boinc tasks being run as this will reduce the memory demands. There are other ways, that other people may well suggest I haven't seen this before the upgrade to 8.2.4, and my system has 32 GB RAM -- I can load Kerbal Space Program with RP-1 and a couple dozen parts mods and only use about 60% of my RAM. ID: 116500 · Reply Quote

Zeiss Ikon Send message Joined: 2 Jul 17 Posts: 49	Message 116501 - Posted: 28 Jul 2025, 22:48:59 UTC - in response to Message 116490. Last modified: 28 Jul 2025, 22:52:24 UTC In reply to Dave's message of 28 Jul 2025: Or leave Podman and Docker there ready for when/if needed but still stick to only attaching projects directly and just remove BOINC Central. It shouldn't get in the way of the other projects. It certainly doesn't with any projects I have tried. I haven't changed projects in years -- Einstein on GPU and MilkyWay on CPU, but I've never seen a message like this before. I have the compute preferences set to allow at most 50% of my 32 GB RAM -- I'd think that's enough to run one task each on CPU (8-core AMD Fx8350) and GPU (nVidia RTX 2070 with 4 GB of its own). IOW, I'd expect issues if I were running on a potato, but this computer is not a potato. I believe this started on 25 July, as I see a sharp downturn in my stats on Einstein from that date. ID: 116501 · Reply Quote

Dave Help desk expert Send message Joined: 28 Jun 10 Posts: 2951	Message 116504 - Posted: 29 Jul 2025, 6:50:49 UTC I have the compute preferences set to allow at most 50% of my 32 GB RAM There are things you can do to to test what is happening. One increase available memory to 75% and see if the problem goes. Two run top in a terminal and see how much memory different processes are using. Is it possible that one of your two projects has released a more memory hungry application? I think this unlikely but if true I would expect to see lots of posts on the project boards about it. ID: 116504 · Reply Quote

Zeiss Ikon Send message Joined: 2 Jul 17 Posts: 49	Message 116508 - Posted: 30 Jul 2025, 0:05:50 UTC - in response to Message 116504. I see several "boincmgr" processes, the largest is consuming about 1.7% of RAM -- but at present I have no tasks running, all show "waiting for shared memory". And bumping "use at most" to 90% of memory has changed nothing. I don't know why either Einstein or MilkyWay would knowingly release tasks that require more than 16 GB RAM just to run -- they've always had their tasks built to run in fairly modest machines; today, I'd expect a 4 GB system to be able to run MilkyWay, maybe a little more for Einstein (or maybe not, I only run Einstein on GPU). I do see my user totals still rising slowly, so it's apparently still getting some work done while I'm not looking, but I'm used to checking "Tasks" and seeing one Einstein (GPU) and up to several MilkyWay running simultaneously (or one claiming all 8 CPU cores), or a message about "CPU busy" that goes away if I leave it for a minute... ID: 116508 · Reply Quote

Dave Help desk expert Send message Joined: 28 Jun 10 Posts: 2951	Message 116509 - Posted: 30 Jul 2025, 5:15:22 UTC - in response to Message 116508. How much memory is being used according to the free command? ID: 116509 · Reply Quote

Zeiss Ikon Send message Joined: 2 Jul 17 Posts: 49	Message 116534 - Posted: 2 Aug 2025, 11:16:03 UTC - in response to Message 116509. Last modified: 2 Aug 2025, 11:16:43 UTC total used free shared buff/cache available Mem: 32830044 3894336 21500816 56280 7956532 28935708 Swap: 67108860 0 67108860 ID: 116534 · Reply Quote

Dave Help desk expert Send message Joined: 28 Jun 10 Posts: 2951	Message 116537 - Posted: 2 Aug 2025, 14:56:04 UTC Now I am running out of ideas. Unless your projects have both released some applications requiring lots of memory which seems unlikely. Is there anything on their forums about it? On the subject of tasks requiring a lot of RAM, CPDN will in the next month or so, possibly sooner be sending out tasks requiring 26GB RAM/task. Initially at least they are limiting these to one per machine so i will have to use a VM to run two at once on this machine. ID: 116537 · Reply Quote

Zeiss Ikon Send message Joined: 2 Jul 17 Posts: 49	Message 116542 - Posted: 2 Aug 2025, 22:01:28 UTC - in response to Message 116537. Okay, I'll take a look on the Einstein and MilkyWay forums and see if there's something going on. With both projects giving the same error, though, I'm inclined to believe it's a BOINC problem rather than a project issue. And apparently one closely tied to my OS, since my hardware seems adequate. I don't think I mentioned this above, but I'm running Debian Bookworm 12.11 on an AMD Fx8350 and an antique Gigabyte motherboard, with 32 GB RAM and several terabytes of storage; BOINC has its own partition with 50 GB of space and is required to leave 1 GB free; also allowed to use up to 75% of my 64 GB swap space (which showed zero used in the free report above). ID: 116542 · Reply Quote

Jord Volunteer tester Help desk expert Send message Joined: 29 Aug 05 Posts: 15661	Message 116543 - Posted: 2 Aug 2025, 23:51:53 UTC Last modified: 2 Aug 2025, 23:59:39 UTC This is a rewrite of an old FAQ of mine, it's originally for the MacOS, but may work on Linux as well. ANd then again, having checked a little, it may be too old to run on present day Linux. But at least it'll give you an idea that you should try to increase the shared memory pool. BOINC science applications use shared memory to communicate with the client, and a certain amount is reserved for each current task, whether running or waiting. The default configuration of a multi-CPU (regardless of how much RAM is installed) is sometimes inadequate to support several tasks at once. If you get this error message, it may help to set your BOINC general preference "Leave applications in memory while suspended?" to no; however, the problem will probably just become less frequent rather than being solved, and moreover some projects' applications, those that save checkpoints infrequently (or erratically), may not take well to this setting. A better solution is available if you don't mind reconfiguring your system's kernel a little. It involves creating a text file /etc/sysctl.conf that contains the following commands: kern.sysv.shmmax=16777216 kern.sysv.shmmin=1 kern.sysv.shmmni=128 kern.sysv.shmseg=32 kern.sysv.shmall=4096 and rebooting the computer. This should quadruple the default allocation of shared memory. See https://man7.org/linux/man-pages/man5/sysctl.conf.5.html for more information on sysctl.conf and you can play around with the values, of course. ID: 116543 · Reply Quote

Copyright © 2025 University of California.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.