!fnexplode.bsi [114] - function to 'explode' a string into tokens !------------------------------------------------------------------------- !Edit History ! [114] 29-May-26 / jdm / Add comments, param defaults; minor cleanup ! [113] 28-Jun-24 / jdm / Add FXF_PPN flag to ignore commas inside [xxx,xxx] ! [112] 06-Sep-23 / jdm / add keywords for funcidx, minor modernization ! [111] 02-Nov-18 / jdm / Adjust debug.prints to use new (level,tags) ! [110] 06-Jul-18 / jdm / fix bug introduced in [108] dealing with contiguous ! delimiters (was lumping 2nd delimiter with next field) ! [109] 01-Jul-18 / jdm / Fix bug in which it set field to the trailing deliminter ! when it should have been null. Also replace .clear ! with redimx in Fn'Explode'Ary() to preserve 6.4 ! compatibility (see Requirements below) ! [108] 19-May-18 / jdm / Add Fn'Explode'Ary() for array mode; support multiple ! delimiters; fix bug with returning FXF_CONT unnecessarily; ! FXF_QUOTE now ignores delimiters within quoted substrings ! [107] 08-Jul-17 / jdm / Combine Fn'Explode and Fn'ExplodeEx. (The Fn'Explode() ! wrapper wasn't able to handle the unnamed output ! params, and it's silly to duplicate it) ! [106] 13-May-17 / jdm / Fix infinite loop if only 1 item to explode ! [105] 21-Apr-17 / jdm / Close loophole in which it failed to return 0 after ! all substrings processed ! [104] 03-Aug-16 / jdm / Add Fn'ExplodeEx() to allow explicit delimiter; ! add FXF_TRIM, FXF_QUOTE ! [103] 23-Sep-15 / jdm / add :inputonly ! [102] 23-Oct-14 / jdm / Remove limit on input string size ! [101] 23-Oct-14 / jdm / Add ++ifdef ! [100] 06-Sep-12 / jdm / Increase field limit from 25 to 125; variable len !----------------------------------------------------------------------------- !Keywords: parsing tokenization ! !Functions: ! Fn'ExplodeEx(string$,delimiter$,flags,f1${,f2$,...fn$}) - explicit delimiter ! Fn'Explode'Ary(string$,delimiter$,flags,f$()) - [108] array mode ! !Requirements: ! 6.3.1530+ (for INSTRF_ANYxx flags) ! Fn'Explode'Ary() requirements: ! Compiler 865 (or 837.1) to call Fn'Explode'Ary() with a local DIMX ! array from a function (w/o premature release runtime error) ! 6.4.1557.7 / 6.5.1639.3 to avoid problem with an empty X0 array ! element returned from Fn'Explode'Ary() not satisfying test ! IF FLD$(X)="". Alternate workaround is to use IF ASC(FLD$(X))=0 ! (fnexplode.bp test program [102] now tests for this issue) !----------------------------------------------------------------------------- ++ifndef FN_EXPLODE_STRLEN ++include'once ashinc:ashell.def define FN_EXPLODE_STRLEN = 0 ! max len of input string [101] (now variable) define FXF_CONT = &h01 ! continuation needed or in progress define FXF_TRIM = &h02 ! [104] trim spaces from tokens (default for whitespace delimiter) define FXF_QUOTE = &h04 ! [104] remove quotes; ignore delimiters within quoted substrings [108] define FXF_PPN = &h08 ! [113] don't treat comma in [xxx,xxx] as a delimiter !------------------------------------------------------------------------------ ![107] Function Fn'Explode(string$ as s0:inputonly, flags as b2) as i2 ! Old version which only supported space delimiter; replaced by Fn'ExplodeEx() !------------------------------------------------------------------------------ !--------------------------------------------------------------------- !Function: ! Splits a string into tokens separated by specified delimiter(s) ! Replacement for original Fn'Explode that assumed whitespace delimiter !Params: ! string$ [string, in] - string to explode ! delimiter$ [s1, in] - delimiter char(s) (null string = spaces or tabs) see notes ! flags [num, in/out] - flags: FXF_xxx ! f1$...fn$ [string, out] - field 1 ... field n (125 max) ! !Returns: ! # of fields exploded from string (up to # of f$ vars passed) !Globals: !Notes: ! [107] Routine recognizes whether delimiter specified and ! acts accordingly, e.g. ! Fn'ExplodeEx(string$, delimiter$, flags, ) ! or ! Fn'ExplodeEx(string, flags, ) ! ! [108] delimiter may now contain multiple chars (any of which will work) ! ! If delimiter is space (or ""), any number of contiguous spaces are ! treated as a single delimiter. ! ! Field parameters beyond the data actually exploded will be nulled out. !-------------------------------------------------------------------------- Function Fn'ExplodeEx(string$ as s0:inputonly, delimiter$ as s10:inputonly, flags=0 as b2) as i2 Static map1 s_string$,s,FN_EXPLODE_STRLEN,"" map1 locals map2 x,i,4 map2 y,i,4 map2 i,i,4 map2 p,i,4 map2 fcount,b,1 map2 token$,s,0 map2 flagsarg,b,1,3 map2 instrflags,b,4,INSTRF_ANY ! [108] allow for multiple delimiters ! ![107] figure out how we were called (in order to handle confusion left over ![107] from Fn'Explode/Fn'ExplodeEx separation if (.argtyp(2) and ARGTYP_MASK) = ARGTYP_S then debug.print (9,"fnexplode") ABC_CURRENT_ROUTINE$, delimiter$, flags, .argcnt else ! delimiter not passed - defaulting to whitespace flags = val(delimiter$) delimiter$ = "" flagsarg = 2 ! arg # of flags endif fcount = (.ARGCNT - flagsarg) min 125 ! # of fields specified (128 is internal limit) if flags and FXF_CONT then string$ = s_string$ !debug.print (9,"fnexplode") "Reprocessing from ["+string$+"]" endif ! [108] if delimiter$ = "" then ! remove leading spaces/tabs (8) + trailing spaces/tabs (32) + condense spaces/tabs (16) string$ = edit$(string$,EDITF_SPTB1 or EDITF_SPTBR or EDITF_SPTB1) delimiter$ = " " ! after edit/condense, delimiter is now space endif if flags and FXF_QUOTE then ! [108] instrflags = INSTRF_ANYQT ! [108] endif ! [108] if (string$ # "") then y = 1 ! [108] need to start at 1 for INSTR_ANYQT [110] (was 0) do x = instr(y,string$,delimiter$,instrflags) ! find next delimiter [108][110] (was y+1) y = y max 1 ! [108] fixup needed for INSTR_ANYQT i += 1 if x = y then ! [109] special case for delimiter in first pos xputarg flagsarg+i, "" ! [109] (needs to return "", not the delim) else if flags and FXF_PPN then ! [113]- check if delimiter is comma in [xxx,xxx] if (x > 0) and instr(1,delimiter$,",") then p = .instrr(x,string$,"[") if (p >= (x - 4)) then p = instr(x+1,string$,"]") if (p > 0) and (p <= (x + 4)) then p = p - y ! [113] yes - skip over it to ] and repeat i -= 1 repeat endif endif endif endif ! -[113] if flags and (FXF_TRIM or FXF_QUOTE) then ! [104] token cleanup needed? token$ = string$[y,x-1] xcall TRIM, token$, 1 if flags and FXF_QUOTE then if token$[1,1] = """" then if token$[-1,-1] = """" then token$ = token$[2,-2] endif endif endif xputarg flagsarg+i, token$ else xputarg flagsarg+i,string$[y,x-1] endif endif y = x + 1 loop until x < 1 or i >= fcount Fn'ExplodeEx = i ! [105] else Fn'ExplodeEx = 0 ! [105] endif if x = 0 then ! we hit end of string ! [108] now we remove the FXF_CONT any time we have fewer fields exploded than return params if i < fcount then ! [108] i = 0 then ! [105] don't remove the FXF_CONT flag until we flags = flags and not FXF_CONT ! have a pass with no output endif s_string$ = "" ! no string left to process do while i < fcount i += 1 xputarg flagsarg+i,"" ! null out unused fields loop else flags = flags or FXF_CONT ! set continuation flag s_string$ = string$[y,-1] ! and store remaining part of string endif xputarg flagsarg,flags EndFunction !--------------------------------------------------------------------- !Function: ! Variation of Fn'ExplodeEx() that outputs to an auto_extend array of x elements. !Params: ! string$ [string, in] - string to explode ! delimiter$ [s1, in] - delimiter char(s) (null string = spaces or tabs) see notes ! flags [num, in/out] - flags: FXF_xxx ! fld$() [array of x, byref] - output fields (must be x, not s) [108] ! !Returns: ! # of fields exploded from string (see notes) !Globals: !Notes: ! Unfortunately there is no way currently to use the same ! function definition for the array and fields modes, nor can ! we handle case of missing delimiter arg ! ! Since prior to 6.5.1624 (when .CLEAR(ary()) was allowed on traditional ! arrays), it wasn't possible to determine if the array passed was ! auto_extendable. Instead, we'll trap subscript out of range to ! detect case of caller passing a fixed array that is too small. ! ![109] ! Remove the .CLEAR(ary()) to improve backward compatibility. (We could ! test the runtime version and use it if available, but it's not really ! necessary since the function return value gives the # of fields exploded. ! ! Warning: see Requirements at top of file for notes related to issues ! specific to the fld$() array of x0 elements !--------------------------------------------------------------------- Function Fn'Explode'Ary(string$ as s0:inputonly, delimiter$="" as s10:inputonly, & flags=0 as b2, fld$() as x0) as i2 Static map1 s_string$,s,FN_EXPLODE_STRLEN,"" map1 locals map2 x,i,4 map2 y,i,4 map2 i,i,4 map2 p,i,4 map2 fcount,b,2 map2 token$,s,0 map2 instrflags,b,4,INSTRF_ANY ! [108] allow for multiple delimiters on error goto trap ! preclear the array depending on whether passed as dimx or fixed. ! [109] remove to improve backwards compatibility (see notes above) !>! if .argtyp(4) and ARGTYP_DIMX then !>! .clear fld$() !>! endif if flags and FXF_CONT then string$ = s_string$ !debug.print (9,"fnexplode") "Reprocessing from ["+string$+"]" endif if delimiter$ = "" then ! remove leading spaces/tabs (8) + trailing spaces/tabs (32) + condense spaces/tabs (16) string$ = edit$(string$,EDITF_SPTB1 or EDITF_SPTBR or EDITF_SPTB1) delimiter$ = " " ! after edit/condense, delimiter is now space endif if flags and FXF_QUOTE then ! [108] instrflags = INSTRF_ANYQT ! [108] endif ! [108] if (string$ # "") then y = 1 ! [108] need to start at for INSTR_ANYQT [110] (was 0) do x = instr(y+p,string$,delimiter$,instrflags) ! find next delimiter [108][110] was y+1 [113] +p p = 0 ! [113] y = y max 1 ! [108] fixup needed for INSTR_ANYQT i += 1 if x = y then ! [109] special case for delimiter in first pos fld$(i) = "" ! [109] (needs to return "", not the delim) else if flags and FXF_PPN then ! [113]- check if delimiter is comma in [xxx,xxx] if (x > 0) and instr(1,delimiter$,",") then p = .instrr(x,string$,"[") if (p >= (x - 4)) then p = instr(x+1,string$,"]") if (p > 0) and (p <= (x + 4)) then p = p - y ! [113] yes - skip over it to ] and repeat i -= 1 repeat endif endif endif endif ! -[113] if flags and (FXF_TRIM or FXF_QUOTE) then ! [104] token cleanup needed? token$ = string$[y,x-1] xcall TRIM, token$, 1 if flags and FXF_QUOTE then if token$[1,1] = """" then if token$[-1,-1] = """" then token$ = token$[2,-2] endif endif endif fld$(i) = token$ else fld$(i) = string$[y,x-1] endif endif y = x + 1 loop until x < 1 ! or i >= maxextent Fn'Explode'Ary = i ! [105] else Fn'Explode'Ary = 0 ! [105] endif cont: if x = 0 then ! we hit end of string flags = flags and not FXF_CONT ! have a pass with no output s_string$ = "" ! no string left to process else flags = flags or FXF_CONT ! set continuation flag s_string$ = string$[y,-1] ! and store remaining part of string endif xputarg @flags exitfunction trap: ! err 8 would occur only if caller passed a non-auto-extent array that didn't ! have enough elements in it; in that case, just treat same as ! Fn'ExplodeEx() with insufficient fields if err(0) = 8 then ! subscript out of range !debug.print (9,"fnexplode") "Hit end of fld$() array! x="+x+",y="+y+", i="+i Fn'Explode'Ary = .extent(fld$()) flags = flags or FXF_CONT x = 1 ! handles case where overflow token was last one (x=0) resume cont else resume endfunction with_error endif EndFunction ++endif