Previous Thread
Next Thread
Print Thread
B64ENC Limit #37289 19 Apr 24 04:13 PM
Joined: Aug 2016
Posts: 329
J
John Andreasen Offline OP
Member
OP Offline
Member
J
Joined: Aug 2016
Posts: 329
Hi,

I seem to be running into an issue with using B64ENC to decode a very long base64 encoded string. The string I am specifying is 309392 bytes long, but the status value returned from B64ENC is only 35436. I did check the limits and capacities page and did not see a mention of a limit for this routine. Is it something that could be easily expanded?

Thanks,
John Andreasen

Re: B64ENC Limit [Re: John Andreasen] #37291 19 Apr 24 05:07 PM
Joined: Jun 2001
Posts: 11,645
J
Jack McGregor Online Content
Member
Online Content
Member
J
Joined: Jun 2001
Posts: 11,645
Hi John -

Offhand I don't see any obvious limitation in the length of the data strings supported by B64ENC. So perhaps there is some other issue -- status variable overflow? dynamic variable complication?

To help pin that down, I resurrected an old test program, adding a "Big Test", and uploaded it here: b64enc.bp

(See the notes in the source about configuring the length of the data to encode - currently it's at 3,000,000 bytes (much larger than your example).

Here's what my output looks like:
Code
.run b64enc
Base 64 encoding...
1)ENC file, 2)ENC string, 3)DEC string, 4)Sample test, 5)Big test, 0)End: 5
Generating random data to encode ( 3000000 bytes )...
 50 , 49 , 49 , 49 , 49 , 50 , 49 , 50 , 49 ,...
Encoding...
Status =  4000000 (bytes in encoded string)
Decoding...
Status =  3000000 (bytes in decoded string)
Decoded data matches original
Program stop at location counter &h6E0 of B64ENC.RUN
Enter CR to continue:
Re-encoding...
Status =  4000000 (bytes in encoded string)
Success
Encoded and re-encoded data matches

Re: B64ENC Limit [Re: John Andreasen] #37292 19 Apr 24 05:17 PM
Joined: Aug 2016
Posts: 329
J
John Andreasen Offline OP
Member
OP Offline
Member
J
Joined: Aug 2016
Posts: 329
Hi Jack, thanks for the quick response! I will study the difference between your example and my program and see if I can find the difference.

Re: B64ENC Limit [Re: John Andreasen] #37303 19 Apr 24 07:35 PM
Joined: Aug 2016
Posts: 329
J
John Andreasen Offline OP
Member
OP Offline
Member
J
Joined: Aug 2016
Posts: 329
Hi Jack,

I wanted to let you know that I located the problem. Our function wrapper that does base64 decoding was using SPACE() to expand a dynamic X variable to receive the decoded data in. SPACE() must have some type of internal limit. Changing to FILL$() fixed the issue. Sorry to have troubled you with this!

Thanks,
John Andreasen

Re: B64ENC Limit [Re: John Andreasen] #37305 19 Apr 24 08:12 PM
Joined: Jun 2001
Posts: 11,645
J
Jack McGregor Online Content
Member
Online Content
Member
J
Joined: Jun 2001
Posts: 11,645
John - thanks for the detective work! And for bringing up the issue in the first place. It's not yet clear to me why SPACE() should have any length limitation different than FILL$() -- both are constrained by the memory available on the string stack and the size of the destination, but that should be the same in both cases. And deserves further investigation.

On the bright side, the documentation did lead you to the solution. But this is a good example of what I was mentioning earlier in regard to the Stephen's question about the "stable 7.0" -- the more you turn over stones in the code, the more things you find that probably should be cleaned up. In this case, it would make more sense (and be more consistent with DRY principles) if SPACE() internally used the FILL$() logic, since after all, SPACE(n) is equivalent to FILL$(chr(32),N). I'll put that on the to-do list...

Thanks again,
Jack

Re: B64ENC Limit [Re: John Andreasen] #37306 19 Apr 24 10:34 PM
Joined: Aug 2016
Posts: 329
J
John Andreasen Offline OP
Member
OP Offline
Member
J
Joined: Aug 2016
Posts: 329
Hi Jack,

Well, I am glad some good came out of this! Thanks again for the help.

John

Re: B64ENC Limit [Re: John Andreasen] #37307 20 Apr 24 12:33 AM
Joined: Jun 2001
Posts: 11,645
J
Jack McGregor Online Content
Member
Online Content
Member
J
Joined: Jun 2001
Posts: 11,645
To put a cap on this, I did discover that length argument passed to SPACE(N) was getting truncated at 64K, which explains the problem. But I ended up mapping it to the implementation of FILL$(chr(32),N), whose maximum theoretical length is 2GB. (The practical limit remains bound by the available size of the string stack.)

As a bonus it's also about 10X faster when filling with single-character pattern (not that the difference will be meaningful to anyone!). So yes, some good did come out of it.

The update will 1757.3, although I'm going to wait to try to get a couple more fixes taken care of first.

Re: B64ENC Limit [Re: John Andreasen] #37317 23 Apr 24 05:44 PM
Joined: Aug 2016
Posts: 329
J
John Andreasen Offline OP
Member
OP Offline
Member
J
Joined: Aug 2016
Posts: 329
OK, that is great. Thank you Jack


Moderated by  Jack McGregor, Ty Griffin 

Powered by UBB.threads™ PHP Forum Software 7.7.3