Of larger numbers

[longtimebasic]

Good day,

I'm working with ISBN numbers for books, in particular the 978-prefixed series. For example, 9780064431781. A vendor we're using takes a subset of the whole ISBN number and uses it as an identifying string for a given product. Having to only enter 8 digits as opposed to the full set of 13 does speed things up a bit. I'm concerned about possible duplicate numbers. So, I was attempting to calculate all the legal ISBN numbers for the whole series. I'd broken up the program so it would run on my i7 as four separate processes, might as well maximize the efficiencies of having four cores!   

So, the program was going along calculating the ISBNS:

series 1: 9780000000000 to 9782499999999
series 2: 9782500000000 to 9784999999999
series 3: 9785000000000 to 9787499999999
series 4: 9787500000000 to 9789999999999

Now, determining if an ISBN is valid or not requires computing a checksum value which is derived from applying a mask against the first 12 digits then deriving the remainder via a modulo 10 division. That result is subtracted from 10 and further given a modulo 10 division. if the now single number matches the 13th digit of the ISBN its calculated the correct checksum. This part of the program was working fine and had no problems.

Finally, once it found a valid ISBN it would write both the vendor subset,full ISBN as a single string to a sequential file (opened for output). A single entry, using the example above would look like: 06443178,9780064431781.

Once the file size got over 4GB, however, Just BASIC wasn't happy anymore and brought them all to a screeching halt! it gave me an error of needing a reference to a #long variable type. My issue and the reason for this post is that whilst I've seen references to Just BASIC integers being "huge," obviously 4 GB isn't huge enough. I can't quite find where I can assign a variable to be designated as "long" so that the process won't stop. 4GB... why am I suddenly thinking this is a problem due to JB being a 32 bit implementation? If this is indeed the case, are there any plans for a 64 bit version to get around this limit anytime in the near future?

Thanks for any assistance, even if it is only to confirm my suspicion of running into the wall. 



The error log gives this:

Error log timestamp Monday 10/22/18 09:57:12



Runtime error: Invalid argument #2. Expected #long



Error(Exception)>>defaultAction

Error(Exception)>>activateHandler: <anUndefinedObject>

Error(Exception)>>handle

Error(Exception)>>signal

Error class(Exception class)>>signal: <'Invalid argument #2....'>

BasicRunProgram(Object)>>error: <'Invalid argument #2....'>

BasicRunProgram(BasicProgram)>>terminateRun: <anAPICallInvalidArgumentType>

[] in BasicProgram>>errorHandlerBlock

ExceptionHandler>>evaluateResponseBlock: <aBlockClosure> for: <anAPICallInvalidArgumentType>

[] in ExceptionHandler>>handle:

ProtectedFrameMarker(BlockClosure)>>setUnwind: <aBlockClosure>

BlockClosure>>invisibleEnsure: <aBlockClosure>

ExceptionHandler>>handle: <anAPICallInvalidArgumentType>

ExceptionHandler>>findHandler: <anAPICallInvalidArgumentType>

APICallInvalidArgumentType(Exception)>>activateHandler: <anExceptionHandler>

APICallInvalidArgumentType(Exception)>>handle

APICallInvalidArgumentType(Exception)>>signal

KernelDLL(DynamicLinkLibrary)>>invalidArgument

KernelDLL>>setFilePointer: <aFileHandle> offset: <4294967296> offsetHigh: <aByteArray> move: <0>

FileHandle>>readInto: <'

96235701,869623570...'> atPage: <1048576> pageSize: <4096>

FileHandle>>readInto: <'

96235701,869623570...'> atPosition: <4294967296>

File>>readBuffer: <'

96235701,869623570...'> atPosition: <4294967297>

FileStream>>position: <4294967296>

FileStream>>putBytesFrom: <'96235887,86962358878'> from: <1> to: <20>

FileStream(WriteStream)>>putBytesFrom: <'96235887,86962358878'>

FileStream>>nextPutAll: <'96235887,86962358878'>

BasicFile>>writeItem: <'96235887,86962358878'>

BasicFile>>writeItemCr: <'96235887,86962358878'>

[] in PrintCommand>>deviceCr:handle:

BlockClosure>>value: <aBasicRunProgram> value: <'#ilist'> value: <aBasicStringContext>

BasicTripleParameterContextHolder>>value

[] in BasicRunProgram>>begin

ExceptionHandler>>evaluateProtectedBlock: <aBlockClosure>

[] in ExceptionHandler>>activateDuring:

ProtectedFrameMarker(BlockClosure)>>setUnwind: <aBlockClosure>

BlockClosure>>invisibleEnsure: <aBlockClosure>

ExceptionHandler>>activateDuring: <aBlockClosure>

ExceptionHandler class>>handle: <anError class> with: <aBlockClosure> during: <aBlockClosure>

BlockClosure>>on: <anError class> do: <aBlockClosure>

BasicRunProgram>>begin

BasicRunProgram(BasicProgram)>>run

BasicOnDemandCompiler class>>readTknFile: <'C:\BProgs\cfindisbns...'> callingProgram: <anUndefinedObject> commandLine: <''>

Basic class>>start

Message>>perform

NotificationManager>>empty

NotificationManager>>runPendingEvents

NotificationManager>>runEventLoop

Message>>perform

Message>>evaluate

Process>>safelyEvaluate: <aMessage>

[Rod]

While Just BASIC can deal with large integers it cannot deal with large numbers of them. The 4gb limit has cropped up in the past. Commands like lof() and seek etc are bound by the largest number, or Ndebele value that Just BASIC can hold which is 4gb.

Perhaps a different strategy? Rather than hold ALL numbers on file why not calculate on the fly.

[tsh73]

It looks like 4Gb file restriction is imposed on JB/LB by it 32-bitness
a$="06443178,9780064431781"
print a$
print len(a$)
'+crlf => 24
print 24*2500000000/1e9
'60.0
Mb=2^20
Gb=2^30
print Mb, Gb
print 24*2500000000/Gb
'55.8(...)

So it does exceed 4Gb, pretty much.

You can split it to smaller chunks, sub-4Gb in size
Like 1e9. Then each your part will require 60 parts of 1e9 numbers.

[tsh73]

Ok I was wrong calculating number of bytes - not all numbers are valid!

So I suggest checking file size as you write - and then it exceeds some given size, close it and open next one
Like this
fileNum = 1
maxFileSize = 1000  'replace for 4e9 for "near 4GB"

a$="06443178,9780064431781" '22 bytes, + 2 for crlf ==24
'let's write 100 of them
'then file grows bigger then maxFileSize, close it and open next one

fileName$="data"+right$(str$(1000000+fileNum), 4)+".txt"
print fileName$

open fileName$ for output as #1

for i = 1 to 100
   print #1,a$
   if lof(#1)>maxFileSize then
       close #1
       fileNum =fileNum +1
       fileName$="data"+right$(str$(1000000+fileNum), 4)+".txt"
       print fileName$
       open fileName$ for output as #1
   end if
next
close #1
print "-- over --"

outputs
data0001.txt
data0002.txt
data0003.txt
-- over --

(two files of 1008 bytes, last one 384)

[Rod]

I am a little puzzled. You are trying to verify an ISBN number is "legal" and you are dealing with a specific range 978xx.

So why store ALL numbers when you can calculate whether the number is legal or not. To be legal it must lie within the number range and must end in the calculated check digit. That is why they exist, to verify human input keying. If the check digit calculates to the one keyed you have a one in ten chance the operator keyed the whole number correctly. You can still get transformations and such which will give the correct check digit but is not the intended number.

Is it a duplicate? well that would require that you had a database of all assigned ISBN numbers.

if every number starts with 978 do you need to store that?

if the vendor is always 06443178 do you need to store that?

But I guess I need to better understand why you feel the need to store ALL valid numbers rather than calculate on the fly.

once it found a valid ISBN

Every number is a valid ISBN when the correct check digit is added, perhaps you need to imagine the number as 12 digits not 13.

[longtimebasic]

Thanks for the prompt replies. Appreciated. I moved the lot over to QB64 and without too much conversion it all works well.