Spawning multiple processes

[Richard Russell]

In a recent post I explained that in order for my BASICs to take advantage of the multiple CPUs/cores that all modern computers have, it's necessary to spawn multiple processes, not multiple threads. Whilst multiple threads share the same memory, multiple processes don't, and this can make coordinating different processes quite tricky (especially in a cross-platform fashion)

I have therefore written a new library spawnlib.bbc, which is compatible with both BB4W and BBCSDL, and with all the supported desktop platforms (Windows, x86 MacOS, Raspberry Pi and Linux). This library significantly simplifies the task of coordinating multiple processes by sharing a nominated block of memory - in the form of a structure - between them.

This example is of dividing the task of drawing the Mandelbrot set between eight sub-processes, each of which renders one strip of the final image:

Code: Select all

   10 REM Mandelbrot Parallel Renderer
   20 REM Demonstrates multi-process rendering using shared memory structures
   30 REM Originally suggested by DeepSeek AI, got working by Richard Russell
   40
   50 INSTALL @lib$ + "spawnlib"
   60
   70 REM Graphics setup
   80 MODE 20  : REM 800x600
   90 OFF
  100
  110 REM Set window dimensions
  120 Width% = 800 : Height% = 560
  130
  140 REM Number of child processes (strips)
  150 numProcs% = 8
  160
  170 REM Divide screen into horizontal strips
  180 stripHeight% = Height% DIV numProcs%
  190
  200 REM Maximum number of iterations
  210 maxIter% = 20
  220
  230 REM Create shared structure array for each child process
  240 DIM Shared{(numProcs%-1) bfType&(1), bfSize%, bfReserved%, bfOffBits%, \
  250 \       biSize%, biWidth%, biHeight%, biPlanes&(1), biBitCount&(1), \
  260 \       biCompression%, biSizeImage%, biXPelsPerMeter%, biYPelsPerMeter%, \
  270 \       biClrUsed%, biClrImportant%, palette%(255), p&(Height%-1,Width%-1), \
  280 \       startY%, fullY%, complete%, xmin, xmax, ymin, ymax, maxIter%}
  290
  300 REM Populate the structures
  310 FOR p% = 0 TO numProcs% - 1
  320   REM Initialise bitmap
  330   Shared{(p%)}.bfType&(0) = ASC"B"
  340   Shared{(p%)}.bfType&(1) = ASC"M"
  350   Shared{(p%)}.bfSize% = ^Shared{(p%)}.startY% - Shared{(p%)}
  360   Shared{(p%)}.bfOffBits% = ^Shared{(p%)}.p&(0,0) - Shared{(p%)}
  370   Shared{(p%)}.biSize% = 40
  380   Shared{(p%)}.biWidth% = Width%
  390   Shared{(p%)}.biHeight% = stripHeight%
  400   Shared{(p%)}.biPlanes&(0) = 1
  410   Shared{(p%)}.biBitCount&(0) = 8
  420   Shared{(p%)}.fullY% = Height%
  430
  440   REM Set palette
  450   FOR index% = 1 TO maxIter%-1
  460     r% = &FF
  470     g% = 100 + 155 * index% / maxIter%
  480     b% = 0
  490     Shared{(p%)}.palette%(index%) = &FF000000 OR (r% << 16) OR (g% << 8) OR b%
  500   NEXT
  510
  520   REM Fractal parameters - adjust for interesting views
  530   Shared{(p%)}.xmin = -2.1
  540   Shared{(p%)}.xmax = 0.8
  550   Shared{(p%)}.ymin = -1.2
  560   Shared{(p%)}.ymax = 1.2
  570   Shared{(p%)}.maxIter% = maxIter%
  580
  590   REM Calculate strip boundaries
  600   Shared{(p%)}.startY% = p% * stripHeight%
  610 NEXT
  620
  630 REM Update display in a timer interrupt
  640 ON TIME PROCdisplay : RETURN
  650
  660 REM Spawn processes
  670 PRINT " Spawning "; numProcs%; " worker processes... ";
  680 PRINT "Each process renders a different horizontal strip of the screen."
  690 TIME = 0
  700 FOR p% = 0 TO numProcs%-1
  710   PROC_spawn(PROCmandelworker(), Shared{(p%)}, "-hidden")
  720   PRINT ," Spawn ";p%;
  730 NEXT p%
  740
  750 REM Wait for all children to complete
  760 REPEAT
  770   allDone% = TRUE
  780   FOR p% = 0 TO numProcs%-1
  790     IF NOT Shared{(p%)}.complete% THEN allDone% = FALSE
  800   NEXT p%
  810
  820   WAIT 5  : REM Don't hammer the CPU
  830 UNTIL allDone%
  840
  850 PRINT STRING$(POS, CHR$127); " Rendering complete in "; TIME/100; " seconds"
  860
  870 REPEAT UNTIL INKEY(0) <> -1
  880 END
  890
  900 DEF PROCdisplay
  910 LOCAL p%
  920 *HEX 64
  930 FOR p% = 0 TO numProcs%-1
  940   OSCLI "MDISPLAY " + STR$~Shared{(p%)} + " 0," + STR$(p% * stripHeight% * 2)
  950 NEXT
  960 *REFRESH
  970 ENDPROC
  980
  990 REM Mandelbrot Strip Renderer for Parallel Processing
 1000 REM Parameters are passed in via shared structures
 1010
 1020 DEF PROCmandelworker(shared{})
 1030
 1040 xmin = shared.xmin : xmax = shared.xmax
 1050 ymin = shared.ymin : ymax = shared.ymax
 1060 startY%  = shared.startY%
 1070 fullY%   = shared.fullY%
 1080 maxIter% = shared.maxIter%
 1090 width%   = shared.biWidth%
 1100 height%  = shared.biHeight%
 1110
 1120 REM Renders a horizontal strip starting from startY%
 1130
 1140 LOCAL x%, y%, iter%, cr, ci, zr, zi, zr2, zi2, smooth, col%
 1150
 1160 FOR y% = 0 TO height% - 1
 1170   REM Map y pixel to imaginary coordinate
 1180   ci = ymax - ((y% + startY%) / fullY%) * (ymax - ymin)
 1190
 1200   FOR x% = 0 TO width%-1
 1210     REM Map x pixel to real coordinate
 1220     cr = xmin + (x% / width%) * (xmax - xmin)
 1230
 1240     zr = 0.0
 1250     zi = 0.0
 1260     zr2 = 0.0
 1270     zi2 = 0.0
 1280     iter% = 0
 1290
 1300     REM Iterate until escape or max iterations reached
 1310     WHILE (zr2 + zi2) < 4.0 AND iter% < maxIter%
 1320       zi = 2.0 * zr * zi + ci
 1330       zr = zr2 - zi2 + cr
 1340       zr2 = zr * zr
 1350       zi2 = zi * zi
 1360       iter% += 1
 1370     ENDWHILE
 1380
 1390     REM Colour mapping - smooth gradient for escaped points
 1400     IF iter% = maxIter% THEN
 1410       col% = 0  : REM Black for points in the set
 1420     ELSE
 1430       REM Smooth colouring for nicer gradients
 1440       smooth = iter% + 1 - LN(LN(zr2 + zi2) / 2) / LN(2)
 1450       col% = smooth
 1460     ENDIF
 1470
 1480     REM Plot the pixel
 1490     shared.p&(y%,x%) = col%
 1500   NEXT x%
 1510
 1520   REM Allow other processes to run - good practice for co-operative multitasking
 1530   WAIT 0
 1540 NEXT y%
 1550
 1560 shared.complete% = TRUE
 1570 ENDPROC

https://youtu.be/03mEBCjx2VU

[hellomike]

Unlike with BBCSDL, in BB4W 6.16a it also works but in the process it quickly opens and closes 8 blank windows.....

Even with these flash windows, strangely it is about 50% faster than BBCSDL.

[Richard Russell]

Unlike with BBCSDL, in BB4W 6.16a it also works but in the process it quickly opens and closes 8 blank windows.....

I'm sure I don't need to explain that this is because the "-hidden" switch, which the Mandelbrot program passes in the PROC_spawn() call, has no effect in BB4W, it's a BBCSDL-specific switch:

Code: Select all

      PROC_spawn(PROCmandelworker(), Shared{(p%)}, "-hidden")

There is no command-line switch that you can pass to BB4W to hide the window, unfortunately. But you can reduce the visual impact of the child windows by hiding them as soon as possible, rather than leaving them displayed until the rendering is complete. Try that.

Even with these flash windows, strangely it is about 50% faster than BBCSDL.

It's not particularly strange. As you can see from the code the measured time includes both the spawning of the sub-processes and the rendering of the graphics. The time it takes to spawn a sub-process will be related, to a first approximation, to the size of the executable program. In BB4W that executable (bbcwrun6.exe) is 82 Kbytes whereas the BBCSDL executable (bbcsdl.exe) is 390 Kbytes, not far off five times larger!

[jinx100]

Running your sample program -

Windows Resource Monitor shows all cores near 100% while the output window is drawing the strips as expected. After the drawing is completed, the spawned processes show terminated and eventually disappear but the cores still show fairly high activity. Only closing the output window drops the core activity back to near zero. Is this normal?

Windows 11 23H2 on an older Ryzen 3 with four cores.

Edit: Simple. The "REPEAT UNTIL INKEY(0) <> -1" line is running. Just REM it out.

Also added a "PROCdisplay" line to finish the drawing and a semicolon at the end of the last PRINT to suppress the linefeed.

Code: Select all

  840 PROCdisplay
  850 PRINT STRING$(POS, CHR$127); " Rendering complete in "; TIME/100; " seconds";
  860
  870 REM : REPEAT UNTIL INKEY(0) <> -1
  880 END