VB6 Get/Put no Xojo equivalents?!?!?!

[Matthew_Combatti]

Here is the project file and basicpage.vfile to be opened/saved by the following code (also found in the project file)

Download: http://www.simulanics.com/projectfile.zip

Ultimately the issue is Xojo equivalents to VB6’s Get/Put binary file functions. I’ve translated a good 99% of the code from an old project, and am having a lapse as how to get this last little piece…placing an array structure into a binary stream, and retrieving existing array structures from a binary stream (ie the included basicpage.vfile from an original template). The original code has been shortened (confirmed to still function in VB6) for ease sake. Any help to get this working would greatly be appreciated.

OpenProject(sFilename as String) Method

[code] //Attempted VB6-Xojo translation
Dim fName as FolderItem = GetFolderItem(sFilename)
Dim vFileContents as BinaryStream

InitVirtualFiles()

vFileContents = BinaryStream.Open(fName,True)

Dim NumberOfItems as Integer
//VB6 Get Code to get contents (storing NumberOfItem as the number of items in the structure)
ReDim VirtualFiles(NumberOfItems)
For i as Integer = 1 to NumberOfItems
//VB6 Get Code to get next bit of data

Dim vFileExtension as Integer = VirtualFiles(VirtualFiles.Ubound - NumberOfItems + i).Extension

If vFileExtension = EX_MODULE then
  Window1.ListBox1.AddRow "Modules"
elseif vFileExtension = EX_ENTRY Then
  Window1.Listbox1.AddRow "Window"
elseif vFileExtension = EX_DIALOG then
  Window1.Listbox1.AddRow "Dialog"
elseif vFileExtension = EX_MENU then
  Window1.Listbox1.AddRow "Menu"
end if

Next

vFileContents.Close()[/code]

//VB6 Code for HTML designer 'Open cd.FileName For Binary As #FileNum 'Get #FileNum, , NumberOfItems 'ReDim VirtualFiles(UBound(VirtualFiles) + NumberOfItems) As TYPE_VIRTUAL_FILE 'For i = 1 To NumberOfItems 'Get #FileNum, , VirtualFiles(UBound(VirtualFiles) - NumberOfItems + i) 'If VirtualFiles(UBound(VirtualFiles) - NumberOfItems + i).Extension = EX_MODULE Then 'tvProject.Nodes.Add "Modules" 'ElseIf VirtualFiles(UBound(VirtualFiles) - NumberOfItems + i).Extension = EX_ENTRY Then 'tvProject.Nodes.Add "Entry" 'ElseIf VirtualFiles(UBound(VirtualFiles) - NumberOfItems + i).Extension = EX_DIALOG Then 'tvProject.Nodes.Add "Windows" 'ElseIf VirtualFiles(UBound(VirtualFiles) - NumberOfItems + i).Extension = EX_MENU Then 'tvProject.Nodes.Add "Menus" 'End If 'Next i 'Close #1

SaveProject Method

[code]//Attempted VB6-Xojo translation
Dim fName as FolderItem = GetSaveFolderItem("",“untitled.vfile”)

Dim bFile as BinaryStream

bfile = BinaryStream.Create(fName,true)

Dim i as Integer
For i = 1 To VirtualFiles.Ubound
//??? VB6 Put Array item VirtualFiles(i) as TYPE_VIRTUAL_FILE in file
Next i

bFile.Close[/code]

//VB6 Code for HTML designer 'Open cd.FileName For Binary As #FileNum 'Put #FileNum, , UBound(VirtualFiles) 'For i = 1 To UBound(VirtualFiles): Put #FileNum, , VirtualFiles(i): Next i 'Close #FileNum

[Matthew_Combatti]

Anyone? Any ideas? The feat is an easy task for many other languages, but I cannot wrap my head around accomplishing this easy file writing task using Xojo. Bad comes to worse I can write a plugin using C++, but I’d rather use pure Xojo code if that is indeed possible. It seems that Xojo cannot write a binary set of memory locations which hold a type-specific array directly to file?!?!

[Jeff_Tullin]

It seems that Xojo cannot write a binary set of memory locations which hold a type-specific array directly to file?!?!

Why would you think that?
Create a memory block or structure of the right size, fill it and save the .StringValue to a binary file…

http://documentation.xojo.com/index.php/Structure

The StringValue getter and setter methods let you treat the structure as a String. This is useful for copying structures into and out of MemoryBlocks, for reading and writing structures to files, and for transmitting structures through sockets.

You can create a structure in either modules or classes. A structure can be created only in the IDE. Refer to User Guide Book 3: Framework, Chapter 10: Advanced Features, Section 5: Structures. Once created, you can use a structure in code.

[Matthew_Combatti]

Not as easy as it looks. The Xojo methods must work with all few hundred thousand existing files (capable of loading them. One is included in the download) The data structure cannot fit into a memory block because it is an array of a user defined data type (not string) and the 2 string types in the structure do not have a defined length (variable length). So by Xojo’s requirement for string datatypes within a structure to have a specified size, a Xojo Structure is not possible, so I created a class to hold the “structure” since string properties do not require a defined length. Please see the attached code with demo file. Get and Put automatically ‘know’ variable lengths of fields in a binary steam. A Xojo binarystream must be “told” exactly how long a binary field is… but this variable in the equation is always different. With 99% of the code converted and included in the project file… if the task is as easy as it sounds please provide the 2-3 lines of code to make it function. There is a commented line where each line of Xojo equivalent code should exist in the open/save methods. I’ve done all the hard work. The code is part of a special emulator with html designer. In VB6, .Net, LiveCode, obj-C, C++, etc the task of writing direct memory locations with the data assigned to them as user-defined datatypes to file, requires a single or couple lines of code… and the same to load them back into the same memory locations at a later time. No data lengths are known. An array would be simple to write to file, but this is an array of user-defined datatypes containing boolean, integers, and unknown string lengths. Please see the download. I know it to be possible in Xojo… but can it be done more easily than reinventing the binarystream class or creating a plugin… really this code just needs to be able to load the preexisting files correctly…one is included…

[Matthew_Combatti]

The conundrum is that every language* except Xojo permits structures with variable lengths *at runtime. So I need to work around this fact…

[Harrie_Westphal]

There HAS to be something in that stream of data the indicates the length of each field or specs of some sort that give the layout of everything. Things DO NOT work via pure magic, there has to be a definition, specification, whatever in order to read and write that data.

[Norman_P]

CR LF separators from what I can see

[Harrie_Westphal]

Norm, after your comment I decided to download it and use a hex editor to look at the file and you are right; however, there are also some FF FF bytes that seem to denote the ending of some section of data as well as the very end of the file. These would probably have to be handled in some fashion. But, this is only one very small file so who knows what other criteria may exist in the entire scheme of things. But it appears to be a fairly simple format, at least with this file.

[Matthew_Combatti]

@177525:@Harrie Westphal

There HAS to be something in that stream of data the indicates the length of each field or specs of some sort that give the layout of everything. Things DO NOT work via pure magic, there has to be a definition, specification, whatever in order to read and write that data.

I have suspected this, but after loading even the simple file included, and comparing it against dozens of others, even the ‘unseen’ characters show no pattern or similarities. All I know is that the standard practice for variable lengths is 10bytes plus the byte length of the actual string (this proves true in Xojo IF the lengths for the structures are known for the demo files included)…finding the exact start and stop point of the first binary structure is proving more difficult with Xojo than I anticipated, as no 100% visible identifier is evident defining the content length. What kind of witchcraft is this!!! All these languages have a single Get/Put function that magically handles all the data and structure lengths for you, moving to binary structures as if they were records in a recordset. Even PHP can read from/save to the files with these user-defined structures correctly.

Here’s the visible/invisible characters (found in the test file):

[STX][NUL][NUL][NUL][VT][NUL]Entry Point[NUL][NUL][NUL]Ê[NUL]application HTML5 webapp;[CR][LF]
[CR][LF]
include “phpframework.inc”;[CR][LF]
[CR][LF]
entry[CR][LF]
[CR][LF]
html.open;[CR][LF]
title = “Basic WebPage”;[CR][LF]
body.open;[CR][LF]
CreateBasicLoginDialog(“myLogin”,_myLogin);[CR][LF]
body.close;[CR][LF]
html.close;[CR][LF]
[CR][LF]
end.ÿÿ[ENQ][NUL]Form1[STX][NUL][NUL][NUL]¥[NUL]0|[CR][LF]
504| 372|Form[CR][LF]
PROPERTY|Form1:CloseBox:True[CR][LF]
PROPERTY|Form1:Frame:Document[CR][LF]
PROPERTY|Form1:MaxBox:True[CR][LF]
PROPERTY|Form1:MinBox:True[CR][LF]
PROPERTY|Form1:Resize:True[CR][LF]
ÿÿ

As norm noted double sets of CRLF’s seem to delimit the binary structure lengths…but problematic when 2 sets of CRLF naturally occur in the “Content” portion of the array.

After some investigation…
There is an industry accepted standard for reading/writing sequential data structures from a file in binary mode that is just not common knowledge to developers; rather to development engineering companies. SQL has adapted its own version of this protocol.

According to one site, VB accepts both random and sequential binary modes (so do the aforementioned previous languages)

Creating Binary Files using Visual Basic

According to the refs, if sizes MUST be defined in type structures, then only random access protocols are in place…true binary mode works in both random and sequential mode (giving self locating positions). Does the Xojo BinaryStream have a sequential mode? (no)…even still, a definitive structure terminator would make it possible to create a custom class to implement the ‘missing sequential’ binary handling.

May be worth creating a class in place of making a plugin…

So we can assume that:

and ÿÿ[ENQ][NUL] terminates a variable length type (but ‘ÿÿ’ always is the last character of the binary file)

[STX][NUL][NUL][NUL] - always starts the binary file sequential data, and marks the last record of the data structure

Now to find docs outlining the whole sequential protocol. Surely if it truly is an industry standard, there should be some sort of outline for interoperability sake?! NUL seems to be the delimiter, in larger demo files with multiple arrays stored, [NUL][DC1], [NUL][DC2], [NUL][DC3], [NUL][DC4] delimit each array. According to the little text I could find so far, there are ‘invisible’ headers defining sizes somewhere in the file…perhaps I’ll need to open it in a hex editor or NotePad++.

#WhenSimpleThingsBecomeDifficult

[Phillip_Zedalis]

That’s why we get paid the big bucks. Terrible formats are the norm. That’s the advantage of SQLite as you can at least query the information out of there in the future. Many projects its overkill and I still use it.

[Norman_P]

It sure looks like a text file delimited by CRLF to me
Every line has one

[Norman_P]

[quote=177542:@Matthew Combatti]I have suspected this, but after loading even the simple file included, and comparing it against dozens of others, even the ‘unseen’ characters show no pattern or similarities.
[/quote]
CR LF delimited text file

@177542:@Matthew Combatti

According to the refs, if sizes MUST be defined in type structures, then only random access protocols are in place…true binary mode works in both random and sequential mode (giving self locating positions). Does the Xojo BinaryStream have a sequential mode? (no)

YES
You read 1 byte, then read another, and another and another.
That’s really all there is to sequential files
Try this https://msdn.microsoft.com/en-us/library/aa903295(v=vs.71).aspx

[quote=177542:@Matthew Combatti]…even still, a definitive structure terminator would make it possible to create a custom class to implement the ‘missing sequential’ binary handling.
[/quote]
If you’re reading and writing fixed length structures then thats easy
Read & write that many bytes using the structure’s string (its just bytes)
If you’re reading and writing variable length structures in any language then you need to write some code that can read the fields of he structure & figure out how big they are and do the reading accordingly.

@177542:@Matthew Combatti

Now to find docs outlining the whole sequential protocol. Surely if it truly is an industry standard, there should be some sort of outline for interoperability sake?!

There isn’t an “industry standard”
A sequential file is literally “one byte follows another and you can read it that way”
A TextInput stream is a sequential file - you can read but you cant randomly jump around

[Norman_P]

0200 - not sure yet
00000B00 - intel byte reversed form for 11 (Entry point)
< following 11 bytes for the text “Entry Point”>
00000000 - not sure
00CA - size of following text (in this case 202 bytes which lands at the end of the “end.”)
FFFF - terminator
0500 - size of name - Form1
<5 bytes for name “Form1”>
0200 - not sure
A500 (this is 165 bytes)
<165 bytes that are cr lf separated> - first chunk I’m unsure of (this is 0| 504| 372|Form)
trailing cr lf
FFFF - terminator

[Matthew_Combatti]

TYPE_VIRTUAL_FILE
Extension as Integer
Content as String
Name as String
Used as Boolean
END TYPE

Before the data is written, it is assigned as below.

Array 1:
Extension = 0 (Entry Point)

Content =
application HTML5 webapp;

include “phpframework.inc”;

entry

html.open;
title = “Basic WebPage”;
body.open;
CreateBasicLoginDialog(“myLogin”,_myLogin);
body.close;
html.close;

end.

Name=Form1

Array 2:
0|504| 372|Form
PROPERTY|Form1:CloseBox:True
PROPERTY|Form1:Frame:Document
PROPERTY|Form1:MaxBox:True
PROPERTY|Form1:MinBox:True
PROPERTY|Form1:Resize:True

I think Norm is onto something… the 0200’s mark the start points of each array (“record”) it appears?

[Matthew_Combatti]

Used=False in the template, so I assume it doesn’t get written? Unless its byte is marked by one of the mystery hexes? perhaps the 8 0’s? 8-bits is 1 byte…the same length to store a boolean…and is either 00000001 or 00000000?

I appreciate your help norm! this makes one appreciate the simplicities set in place to hide the complexities underlying development. Your quick analysis of the file breakdown should make this a bit easier to decipher now that there’s a rhyme and reason behind the file structure. I’m off to compare multiple templates and see if I can’t get Xojo to speak Get/Put jibberish!

00111010 00101101 00101001

[Norman_P]

A hex editor + 5 minutes to peek is all I spent
Took me longer to write it down than figure it out