Strange behavior while creating and saving a large BITMAP PowerBASIC Peer Support Community (original) (raw)

• 
  Member
  • Join Date: Jun 2005
  • Posts: 412

I'm making a large BITMAP (10000 pixels by 9000 pixels) and coloring each pixel according to some function. The complete code is NOT shown because it uses a color function that has many, many dependencies. However, I can give some basic pseudo-code:
Code:
FUNCTION PBMAIN () AS LONG
LOCAL y AS LONG
LOCAL x AS LONG
LOCAL xPixel AS LONG
LOCAL yPixel AS LONG
LOCAL hBMP_big AS DWORD
LOCAL HexColor AS DWORD
x = 32000 : y = 0
GRAPHIC BITMAP NEW 10000, 9000 TO hBMP_big

FOR yPixel=0 TO 8999
FOR xPixel=0 TO 9999
HexColor = SomeUnshownFunction that uses xPixel and yPixel, returns an RGB color value in DWORD
' attach to the big target bitmap each time because the HexColor
' function above attaches to other bitmaps
GRAPHIC ATTACH hBMP_big, 0, REDRAW
GRAPHIC SET PIXEL (xPixel, yPixel), HexColor
IF RND(1, 1400) = 4 THEN GRAPHIC REDRAW
IF ERR > 0 THEN PRINT ERR; ERROR$(ERR)
NEXT xPixel
NEXT yPixel
GRAPHIC ATTACH hBMP_big, 0
GRAPHIC REDRAW
GRAPHIC SAVE "+." + DEC$(x, 5) + "." + DEC$(y, 5) + ".bmp"
GRAPHIC BITMAP END
END FUNCTION
Here's what's happening: I first started with smaller bitmaps (e.g.: 1000 by 1000 pixels) and everything was fine. Everything ran as expected. But as I increased the size of the bitmap, two things happened. First, I would get no error, but the bitmap would not fully render. The saved .BMP file would be the right size, but it looks like PowerBasic stopped plotting pixels about 1/3 the way thru (and stopped mid-loop). I tried inserting a periodic REDRAW, but it made no difference. Secondly, bizarrely, the saved .BMP file has the filename:
Yes, really, with that crazy Chinese Unicode character. But when I scale it back down to a smaller bitmap, all these problems go away.
I'm guessing that I'm overflowing something. But what? And why no error message? How is the output filename getting corrupted?
Am I just pushing the limits of PB's BITMAP functions too hard?
Christopher P. Becker
signal engineer in the defense industry
Abu Dhabi, United Arab Emirates *
Member
* Join Date: Jan 2000
* Posts: 8891

What happens if you increase digits from 5 to 6 or 7? (DEC$(x, 5))
## Comment *
Member
* Join Date: Aug 1998
* Posts: 4703

A 10000x9000 bitmap in 24-bit color will require approximately 257.5 MB of memory.
Ensure your system and application can allocate this amount of contiguous memory. If you're working in a 32-bit environment, memory fragmentation could become a limitation despite the theoretical capability. A 64-bit system is recommended for handling large bitmaps like this.
Patrice Terrier
www.zapsolution.com
www.objreader.com
Addons: GDImage.DLL 32/64-bit (Graphic library), WinLIFT.DLL 32/64-bit (Skin Engine).
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

I suspect you are running out of memory
Note REDRAW does nothiing with a Bitmap, it is only relevant to a Graphic Window (or a PBWin Graphic Control)
And it was very slow even without complicated HexColor calculations (about 1.5 minutes)
(90 million iterations.)
Take a look at GRAPHIC GET BITS, GRAPHIC SET BITS and pointers.
Last edited by Stuart McLachlan; 17 Jan 2025, 05:01 PM.
=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Jun 2005
* Posts: 412

Originally posted by Borje Hagsten View Post
What happens if you increase digits from 5 to 6 or 7? (DEC$(x, 5))
This filename...
Christopher P. Becker
signal engineer in the defense industry
Abu Dhabi, United Arab Emirates
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

About 1 second including 90 000 000 function calls versus 1.5 minutes with no function calls. And no issue with filename:
'
Code:
#COMPILE EXE
#DIM ALL
' #OPTION LARGEMEM32 may be needed if the rest of the application uses a lot of memory.
FUNCTION PBMAIN () AS LONG
LOCAL y AS LONG
LOCAL x AS LONG
LOCAL xPixel AS LONG
LOCAL yPixel AS LONG
LOCAL hBMP_big AS DWORD
LOCAL HexColor AS DWORD
LOCAL t AS DOUBLE
LOCAL PixelPtr AS LONG PTR
LOCAL i AS LONG
LOCAL sBmp AS STRING
x = 32000 : y = 0
t = TIMER
GRAPHIC BITMAP NEW 10000,9000 TO hBMP_big
GRAPHIC ATTACH hBMP_big, 0
GRAPHIC GET BITS TO sBmp
GRAPHIC BITMAP END ' free up hBmp_big memory for whatever Hexcolor() does
PixelPtr = STRPTR(sBmp) + 8
FOR ypixel = 0 TO 8999
FOR xpixel = 0 TO 9999
HexColor = TestFunction(xPixel,ypixel)
@PixelPtr = BGR(Hexcolor)
INCR PixelPtr
NEXT
NEXT
GRAPHIC BITMAP NEW 10000,9000 TO hBMP_big 'get the bits back into a bitmap.
GRAPHIC ATTACH hBMP_big, 0
GRAPHIC SET BITS sBmp
RESET sBmp
GRAPHIC SAVE "+." & DEC$(x, 5) & "." & DEC$(y, 5) & ".bmp"
GRAPHIC BITMAP END
? FORMAT$(TIMER -t,"0.00"); " seconds"
WAITKEY$
END FUNCTION
FUNCTION Testfunction(x AS LONG,y AS LONG) AS LONG
FUNCTION = (x * y) MOD &H00FFFFFF
END FUNCTION
'
Last edited by Stuart McLachlan; 18 Jan 2025, 12:30 AM.
=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

Originally posted by Patrice Terrier View Post
A 10000x9000 bitmap in 24-bit color will require approximately 257.5 MB of memory.
Ensure your system and application can allocate this amount of contiguous memory. If you're working in a 32-bit environment, memory fragmentation could become a limitation despite the theoretical capability. A 64-bit system is recommended for handling large bitmaps like this.
Actually 360 MB as a Packed DIB in memory. (10000 * 9000 * 4). It goes down to 270 MB (257.5 MiB) when saved as a BMP.
That's still not a very large memory block. PB can handle several of them at once.
Last edited by Stuart McLachlan; 17 Jan 2025, 07:35 PM.
=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Oct 1987
* Posts: 10807

My guess is that the problem is located in Christopher's code that isn't shown. Memory corruption such as that produced by array out-of-bounds errors, etc. etc. Need to see compiler metastatements related to error handling.
"Not my circus, not my monkeys."
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

Originally posted by Eric Pearson View Post
My guess is that the problem is located in Christopher's code that isn't shown. Memory corruption such as that produced by array out-of-bounds errors, etc. etc. Need to see compiler metastatements related to error handling.
Yep, the original code works (allbeit very slowly) with Hexcolour = %RGB_RED

=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Oct 1987
* Posts: 10807

Is that info in this thread? I must be missing something. I thought the OP was the only person seeing an error.
"Not my circus, not my monkeys."
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

Originally posted by Eric Pearson View Post
Is that info in this thread? I must be missing something. I thought the OP was the only person seeing an error.
Testing,,I got the same strange file name at one stage, but I can'r reproduce it (which is why I've edited my last post, making your comment moot )
In my recycle bin:

=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

Just dug up the relevant backup. The messed up filename was the result of an error on my behalf
Trying to speed up by commenting out unneeded operations, I got it when I commented out the ATTACH in the loop without placing it before the loop :
Result was GRAPHIC SET PIXEL with no attached bitmap!
Possibly a similar case in Christopher's function.
Code:
GRAPHIC BITMAP NEW 10000, 9000 TO hBMP_big
'should have moved the ATTACH to here!!!
FOR yPixel=0 TO 8999
FOR xPixel=0 TO 9999
HexColor = %RGB_RED
'SomeUnshownFunction that uses xPixel and yPixel, returns an RGB color value in DWORD
' attach to the big target bitmap each time because the HexColor
' function above attaches to other bitmaps
'GRAPHIC ATTACH hBMP_big, 0, REDRAW 'commented out but did not move it higher!!!
GRAPHIC SET PIXEL (xPixel, yPixel), HexColor
'IF RND(1, 1400) = 4 THEN GRAPHIC REDRAW
'IF ERR > 0 THEN PRINT ERR; ERROR$(ERR)
NEXT xPixel
NEXT yPixel
=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Mar 2004
* Posts: 6249

a tid bit
character should be 0
0 is &h0030
Asian character showing is &h8030, so 1 bit corrupted
I have no theory why this particular error. I just wonder how more of the file name wasn't corrupted.
something I did not post yesterday because smaller bitmaps do not cause the problem
+ is not a legal file name character (from what I found on Google)
period is legal, but multiple periods could confuse where extension starts. (search left from end - OK, search right from last \ - not so much
(added - just searched again, + not listed as illegal either!)
suggest, repeat just suggesting
lose the +, use hyphen instead of period (one period for extension only)
((+ subject to added comment))
(edit - late to the party again. but above still applies )
Last edited by Dale Yarker; 18 Jan 2025, 11:21 PM.
Dale
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

Originally posted by Dale Yarker View Post
+ is not a legal file name character (from what I found on Google)
According to MS, it is OK in Windows file systems: https://learn.microsoft.com/en-us/wi.../naming-a-file
(if you are targeting another file system that doesn't allow it, it would be problematic.)
* Use any character in the current code page for a name, including Unicode characters and characters in the extended character set (128–255), except for the following:
* The following reserved characters:
* < (less than)
* > (greater than)
* : (colon)
* " (double quote)
* / (forward slash)
* \ (backslash)
* | (vertical bar or pipe)
* ? (question mark)
* * (asterisk)
* Integer value zero, sometimes referred to as the ASCII NUL character.
* Characters whose integer representations are in the range from 1 through 31, except for alternate data streams where these characters are allowed. For more information about file streams, see File Streams.
* Any other character that the target file system does not allow.
But I agree wholeheartedly that, as a rule, you should avoid anything other than alphanumerics, hyphens and underscores in file names and only use a single period to designate an extension. ( Spaces in file names are the bane of my life).
Hmmm, Everything tells me I have 3 513 objects starting with a period . Most of them are connected with Firefox and most of the rest are either web related (under WAMPServer ) or applications that have both Windows and Linux versions)
=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Mar 2004
* Posts: 6249

Most of them are connected with Firefox ...
relative paths maybe "period backslash" ?
((forum removes the backslash, so spelled it out))
Dale
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

Originally posted by Dale Yarker View Post
relative paths maybe "period backslash" ?
((forum removes the backslash, so spelled it out))
Nope, FF creates a profile folder for every site visited and each one contains , inter alia, a .metadata-v2 file and a ls folder that contains a small sqlite file
=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Oct 1987
* Posts: 10807

I stand by my previous guess.
"Not my circus, not my monkeys."
## Comment *
Member
* Join Date: Aug 1998
* Posts: 43772

Ensure your system and application can allocate this amount of contiguous memory. If you're working in a 32-bit environment, memory fragmentation could become a limitation despite the theoretical capability. A 64-bit system is recommended for handling large bitmaps like this.
Everything here reads (screams?) "memory issues." I think Erik/Stuart ( corruption not related to code shown) Patrice (allocation issues) both have avenues worth exploring.
On no error code from PB.. a whole bunch - actually I think all - DDT statements manipulating windows don't return error codes. (e.g the underlying API call SetWindowText() returns a sucess/fail cde but no such vale is returned by CONTROL SET TEXT) . But you can test for memory errors using structured exception handling - there are examples here in the source code forum.
That said, PB has, in my experience, always been very good about reporting ("Error 7") memory allocation issues.
Or... you could re-engineer your software to use four (some other number) of separate bitmaps and display them adjacent to each other. We old had to do this kind of stuff back in the MS-DOS days.. when you got maybe 400 Kb (yes, that's "K" as in "kraken") of user memory space but some had to be used by your program so you had maybe 200K-300K usable for your data. We learned how to be creative in our uses of what we got. (You will notice i did NOT say " .. you could easily re-engineer...").
Michael Mattias
Tal Systems (retired)
Port Washington WI USA
[email protected]
http://www.talsystems.com
## Comment *
Member
* Join Date: Mar 2000
* Posts: 11080

Just had a thought. If using the much more efficient pointer to a string and GRAPHIC SET BITS method in Post #6, there's no need to create the bitmap initially, just create the string directly
'
Code:
' GRAPHIC BITMAP NEW 10000,9000 TO hBMP_big
' GRAPHIC ATTACH hBMP_big, 0
' GRAPHIC GET BITS TO sBmp
' GRAPHIC BITMAP END ' free up hBmp_big memory for whatever Hexcolor() does
sBmp= MKL$(10000) & MKL$(9000) & NUL$(10000*9000 * 4)
'
=========================
https://camtech.com.pg
=========================
## Comment *
Member
* Join Date: Oct 1987
* Posts: 10807

Nah. You don't even need graphics for this.
Use MS Paint to create and save a "template" bitmap of the correct size, all black.
Dim a 2D DWORD array, fill it with numbers as needed.
Open the bitmap file, SEEK to the end of the header (LOF minus array data length), PUT the array into the file. Done. Bitmap headers contain only _meta_data; size, color depth, etc. The pixels don't matter; you can simply replace them without changing the header.
All numeric, tiny program, no strings or graphics. Fast.

"Not my circus, not my monkeys."

Comment