Proposal for a 'listlib' library

Discussions related to the code libraries supplied with BB4W & BBCSDL
Hated Moron

Re: Proposal for a 'listlib' library

Post by Hated Moron »

DDRM wrote: Tue 19 Dec 2023, 10:02 I suspect it's related to the decision made for "range", and is all consistent with using 0-based numbering. If you write something like

Code: Select all

for x in range(10):
          <Block>
It will execute the block 10 times - but x will run from 0 to 9.
Hmm, "related to", maybe, but I don't really see how one follows from the other. Python arrays/lists are zero-index-based just like BBC BASIC arrays are, but never in BBC BASIC does one specify an index that is one greater than the end of a range, as Python slicing does.

In my experience there are two common ways of representing a range: first and last or first and count. So for example if one has a list of numbers [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] the two ways of representing the subset [4, 5, 6] would be either (in a BBC BASIC-like syntax) 4 TO 6 or 4 BY 3. Python's 4:7 is an outlier.
Range has more sophisticated forms to run between two integers, with an integer step
I referred to that earlier in the thread. I could support it with 'copied' (read-only) slices but not with 'true' (read-write) slices.

As a putative BBC BASIC listlib ought to use a BBC BASIC-like syntax as far as possible, I propose that the range be specified as first and count (which is different from, but not inconsistent with, the Python usage) so my earlier example would become:

Code: Select all

DIM parent(9)
parent() = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

PROC_listslice(parent(), child(), 3, 3)
child() = 11, 12, 13
Hated Moron

Re: Proposal for a 'listlib' library

Post by Hated Moron »

I've realised that another limitation of a 'true' slice is that the dot-product operator doesn't work; you would have to use the 'copied' kind of slice for that. Taking the dot-product of two 1-dimensional arrays, returning a scalar, is a rather niche requirement anyway, and it can alternatively be achieved by multiplying the arrays and using the SUM() operator, which will work with slices:

Code: Select all

scalar() = one() . two()
PRINT scalar(0)
is equivalent to:

Code: Select all

product() = one() * two()
PRINT SUM(product())
Hated Moron

Re: Proposal for a 'listlib' library

Post by Hated Moron »

I've abandoned the listlib library. It turns out that the attraction of Python Lists is not just their functionality (which I can substantially provide via a library) but also their elegant syntax (which I can't).

Providing the functionality but not the elegance doesn't cut the mustard, and won't attract Python users back to BBC BASIC. :(

If I was in the business of once again extending BBC BASIC (which for many reasons, not least my age and health, I'm not) the better solution would be to add native List functionality to the language. This wouldn't be desperately difficult, it would need two principal enabling changes:
  1. A syntax for specifying a List Slice, I would suggest list(first TO last) or list(first BY count) or even both.
  2. Unlocking the latent ability of a variant variable to be a string instead of a numeric, so a list can contain mixed objects.
In BBCTTY and the coded-in-C editions of BBCSDL, variant (suffixless) variables could hold strings now if there wasn't an explicit 'Type mismatch' error message to stop you doing it! This is admittedly not the case in BB4W or the assembler editions of BBCSDL.

So a prerequisite for incorporating native list functionality would be to discontinue the 32-bit x86 editions of BBC BASIC. The Windows edition(s) would be 64-bits only.

Anyway, this is for somebody else to consider once I've gone, or am too unwell to have an active interest in the language.
User avatar
hellomike
Posts: 194
Joined: Sat 09 Jun 2018, 09:47
Location: Amsterdam

Re: Proposal for a 'listlib' library

Post by hellomike »

Thanks for looking into this Richard.

Don't worry as I, for one, will firmly stick with doing all my programming using BB4W and BBCSDL.

Regards,

Mike
Richard Russell
Posts: 458
Joined: Tue 18 Jun 2024, 09:32

Re: Proposal for a 'listlib' library

Post by Richard Russell »

Hated Moron wrote: Mon 05 Feb 2024, 10:17 the better solution would be to add native List functionality to the language. This wouldn't be desperately difficult, it would need two principal enabling changes:
  1. A syntax for specifying a List Slice, I would suggest list(first TO last) or list(first BY count) or even both.
  2. Unlocking the latent ability of a variant variable to be a string instead of a numeric, so a list can contain mixed objects.
Things have moved on since that was written, and of course I have now incorporated native array slicing into BB4W, BBCSDL and BBCTTY. This once again makes a listlib library attractive, since native slicing restores some of the elegance of Python lists.

So I've revisited the library, including removing the slicing routines that are no longer needed. The list of routines it now has is as follows:

Code: Select all

      REM PROC_listcreate(list()): Create a new (initially empty) list
      REM PROC_listclear(list()): Remove all the items from a list
      REM PROC_listappend(list(), item): Add an item to the end of a list
      REM PROC_listinsert(list(), item, idx%): Insert an item into a list
      REM PROC_listdelete(list(), idx%): Delete an item from a list
      REM PROC_listremove(list(), item): Delete the 1st occurrence of item
      REM PROC_listcopy(dest(), source()): Create a copy of a list
      REM PROC_listconcat(one(), two()): Append list two() to list one()
      REM PROC_listreverse(list()): Reverse the order of the list items
      REM PROC_listprint(list()): Print the contents of a list

      REM FN_listlen(list()): Return the total number of items in a list
      REM FN_listprint(list()): Return the contents of a list as a string
      REM FN_listcount(list(), item): Count the items with a given value
      REM FN_listindex(list(), item): Find the 1st item with a given value
      REM FN_listitem(item): Return a single list item as a string
      REM FN_listtype(item): Return the type of an item as a string
My question (principally for Python programmers - DDRM?) is: have I omitted anything important?
Richard Russell
Posts: 458
Joined: Tue 18 Jun 2024, 09:32

Re: Proposal for a 'listlib' library

Post by Richard Russell »

Hated Moron wrote: Mon 05 Feb 2024, 10:17 Unlocking the latent ability of a variant variable to be a string instead of a numeric, so a list can contain mixed objects.
As is usual for binary floating-point numbers, the x86 Extended Precision format reserves certain 'exponent' values for special purposes. BB4W, BBCSDL and BBCTTY take advantage of this to implement 'variant' variables which can contain either floating-point numbers or 64-bit integers.

In the listlib library I have extended this so that variant arrays can be used as lists, which are able to contain floating-point values, (64-bit) integers, (pointers to) strings and (pointers to) sub-lists, as follows:

Code: Select all

Bits 64-79  x86 Extended Precision usage  BBC BASIC usage
0x0000      Positive subnormal (or zero)  64-bit integer
0x7FFF      Positive infinity (or NaN)    String (listlib)
0x8000      Negative subnormal            Sub-list (listlib)
0xFFFF      Negative infinity (or NaN)    String (internal)
The internal use of 0xFFFF to represent a string applies only to the the coded-in-C interpreters, not the x86 assembly-language interpreters.
Richard Russell
Posts: 458
Joined: Tue 18 Jun 2024, 09:32

Re: Proposal for a 'listlib' library

Post by Richard Russell »

I'm reasonably happy with the revised listlib library. Below is the output from a demo program, based on a suggestion by DeepSeek (originally in Python of course); you can run it in your own browser by clicking here.

I've also listed the source code of the demo program below, I hope you will agree that the syntax is fairly elegant and not that different from the equivalent Python code (or if you don't know Python, which I don't, that it's clear what is happening and why you might want to use the library).

listdemo.png

Code: Select all

      REM Example of using the 'listlib' library, inspired by DeepSeek
      REM By Richard Russell, https://www.bbcbasic.co.uk/, 22-Jul-2025

      VDU 23,22,800;500;8,20,16,128
      INSTALL @lib$ + "listlib"
      LINE 800,0,800,998
      COLOUR 2,0,100,0

      REM Initialise a list with a mixture of different types
      DIM myStuff(2)
      myStuff() = 42, FN_sv("Hello"), 3.14 : REM int, str and float

      VDU 28,1,24,49,0

      PRINT "=== Original List ==="
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      REM 1. Append
      PRINT '"1. Append sublist [1, 2, 3]:"
      DIM sublist(2) : sublist() = 1, 2, 3
      PROC_listappend(myStuff(), FN_lv(sublist()))
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      REM 2. Concatenate
      PRINT '"2. Concatenate list [2, 3, 4]:"
      sublist() = 2, 3, 4
      PROC_listconcat(myStuff(), sublist())
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      REM 3. Insert
      PRINT '"3. Insert string 'world' at index 4:"
      PROC_listinsert(myStuff(), FN_sv("world"), 4)
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      REM 4. Clear
      PRINT '"4. Clear the list temporarily (then restore):"
      PROC_listcopy(backup(), myStuff())
      PROC_listclear(myStuff())
      COLOUR 4 : PRINT FN_listprint(myStuff()); : COLOUR 2
      PRINT "  (the list length is now "; FN_listlen(myStuff()); ")"
      PROC_listcopy(myStuff(), backup())
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      REM 5. Index
      PRINT '"5. Find the position of an item:"
      sublist() = 1, 2, 3
      COLOUR 2
      PRINT "The sublist [1, 2, 3] is at index "; FN_listindex(myStuff(), FN_lv(sublist()))
      PRINT "The string  'world'   is at index "; FN_listindex(myStuff(), FN_sv("world"))
      PRINT "The numeric 3.14      is at index "; FN_listindex(myStuff(), 3.14)
      COLOUR 0

      REM 6. Count
      PRINT '"6. Count occurrences of an item:"
      COLOUR 2
      PRINT "The string  'Hello'   occurs "; FN_listcount(myStuff(), FN_sv("Hello")); " time(s)"
      PRINT "The sublist [1, 2, 3] occurs "; FN_listcount(myStuff(), FN_lv(sublist())); " time(s)"
      COLOUR 0

      VDU 28,51,24,99,0

      REM 7. Count with recursion
      PRINT "7. Count items, recursing into sublists:"
      COLOUR 2
      items% = FN_listcount_recurse(myStuff(), 2)
      PRINT "The numeric 2 occurs "; items%; " time(s)"
      COLOUR 0

      REM 8. Reverse
      PRINT '"8. Reverse the list:"
      PROC_listreverse(myStuff())
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      REM 9. Reverse sublist
      PRINT '"9. Reverse the sublist:"
      list = FN_vl(myStuff(4)) : PROC_listreverse(list)
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      REM 10. Type
      PRINT '"10. Find the type of an item:"
      COLOUR 2
      FOR index% = 4 TO 7
        PRINT "The type of item "; index% " is '" FN_listtype(myStuff(index%)) "'"
      NEXT
      COLOUR 0

      REM 11. Remove
      PRINT '"11. Remove the first occurrence of 3.14:"
      PROC_listremove(myStuff(), 3.14)
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      REM 12. Pop
      PRINT '"12. Remove and return the item at index 4:"
      popped = FN_listpop(myStuff(), 4)
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0
      COLOUR 2 : PRINT "The popped item was "; FN_listitem(popped) : COLOUR 0

      REM 13. Delete
      PRINT '"13. Delete the item at index 1:"
      PROC_listdelete(myStuff(), 1)
      COLOUR 4 : PROC_listprint(myStuff()) : COLOUR 0

      OFF
      REPEAT WAIT 4 : UNTIL FALSE
      END

      DEF FN_listcount_recurse(RETURN l%%, v) LOCAL I%,N%,t%%,l()
      IF l%%!1 THEN PTR(l()) = l%% : N% = FN_listcount(l(), v) ELSE = 0
      FOR I% = 0 TO DIM(l(),1)
        IF FN_listtype(l(I%)) = "list" THEN
          t%% = FN_vl(l(I%)) : N% += FN_listcount_recurse(t%%,v)
        ENDIF
      NEXT
      = N%
You do not have the required permissions to view the files attached to this post.
DDRM
Posts: 20
Joined: Mon 17 Jun 2024, 08:02

Re: Proposal for a 'listlib' library

Post by DDRM »

Hi Richard,

That looks very nice!

D
Richard Russell
Posts: 458
Joined: Tue 18 Jun 2024, 09:32

Re: Proposal for a 'listlib' library

Post by Richard Russell »

DDRM wrote: Thu 24 Jul 2025, 08:39 That looks very nice!
Thanks. I was quite surprised to discover that internally Python implements lists in exactly the same way as I have in my listlib library, i.e. as dynamic 1D arrays rather than - for example - linked lists.

The only significant difference I could find is that Python 'over-allocates' memory for the array, so that increasing the length of the list does not always result in a new allocation, whereas for simplicity it does in my library.

But in general I would expect both Python and my library to share some common characteristics, such as accessing a list item by index being very fast but inserting a new item into a list being relatively slow (because it involves moving all the items above it to make space).

If you have any Python code, using lists, that you feel would be particularly 'testing' of my library I would like to see it.
Richard Russell
Posts: 458
Joined: Tue 18 Jun 2024, 09:32

Re: Proposal for a 'listlib' library

Post by Richard Russell »

I should probably point out that although listlib only provides explicit support for list items to be numbers, strings or sublists, you can include other objects in lists simply by storing a pointer to them. For example you could store a pointer to an array, or to a structure, or to a lambda etc.

Unlike strings and sublists there will be no metadata stored in the list to indicate that the items are pointers, your program will just have to 'know' that, but nevertheless this is a perfectly good way of extending the functionality of lists to other objects.