Date Storage Issue

Getting Started
1

I am using a TextInputStream and TextOutputSteam to store date info in seconds to a HD file. I’m not always returning the same date info stored. Should I be using some other text encoding?

Save routine:

Rem Save to HD

Var arrayElement as integer
Var Load_Save_Class_Instance As LoadSaveClass
Load_Save_Class_Instance = new LoadSaveClass

Var Txt_Output_Stream as TextOutputStream

Rem FileAndFolderVerify method used to see if folder and file exists
Var myFile As FolderItem = SpecialFolder.Desktop.Child (HCPrefsCurentFolderLocation).Child("Away Return Dates")
Load_Save_Class_Instance.FileAndFolderVerify("Away Return Dates")

Txt_Output_Stream = TextOutputStream.create(MyFile) Rem Clears fill for rewrite

// Save Away Return Data  prefs to HD
For arrayElement = 0 To 1
    Txt_Output_Stream.WriteLine Str(Pointer_To_Away_Return_Array(ArrayElement).totalSeconds)
next
Txt_Output_Stream.Close[/code]

Load routine:

[code]//Load array from HD
Var i as integer
Var myFile6 As FolderItem = SpecialFolder.Desktop.Child(HCPrefsCurentFolderLocation).Child("Away Return Dates")
Load_Save_Class_Instance.FileAndFolderVerify("Away Return Dates") Rem Verify all files and folder are in place
Txt_Input_Stream_Instance = TextInputStream.Open(MyFile6)

for i = 0 to 1
  //Leave date i =0 Return date = 1

  Away_Return_Array(i)= new date

  Away_Return_Array(i).TotalSeconds = val(Txt_Input_Stream_Instance.ReadLine)
  
Rem if no date exists
  if Away_Return_Array(i).TotalSeconds = 0 then
    Away_Return_Array(i).TotalSeconds = d.TotalSeconds
  end
next

Txt_Input_Stream_Instance.close
2

How do you know?

3

Included in the above code were two more MessageBoxs to verify in and out values and the values placed in the eventual list boxes were the same as the MSGbox values.

Messagebox str(Away_Return_Array(1).LongTime)

messagebox str(Pointer_To_Away_Return_Array(1).LongTime)

4

It appears the numbers for Away_Return_Array(0) = 3.676738e+9 in a text file and Away_Return_Array(1) 3.708320e+9 in the same text file stored on the HD are not sufficient to yield the correct date when reading back out. Am I off track on this line of thinking and if not how might I address it?

5

Try using FORMAT instead of STR when you write the number to the file… making sure your formatSpec is more than long enough… since STR will convert to scientific notation if the number is long enough… FORMAT will not do that conversion.

6

Txt_Output_Stream.WriteLine Format(Pointer_To_Away_Return_Array(ArrayElement).totalSeconds)

Not enough arguments: missing Double value for parameter “value”
Txt_Output_Stream.WriteLine Format(Pointer_To_Away_Return_Array(ArrayElement).totalSeconds)

Parameter “Line” expects type String, but this is type Int32.
Txt_Output_Stream.WriteLine Format(Pointer_To_Away_Return_Array(ArrayElement).totalSeconds)

7

Format takes another parameter:

Txt_Output_Stream.WriteLine Format(Pointer_To_Away_Return_Array(ArrayElement).totalSeconds, “##########”)

Check the docs for specifics.

8

Yes,Yes,Yes. Problem solved guys and i see format also changes the date value to a string which is good to know.

9

[quote=495298:@Tim Hare]Format takes another parameter:

Txt_Output_Stream.WriteLine Format(Pointer_To_Away_Return_Array(ArrayElement).totalSeconds, “##########”)

Check the docs for specifics.[/quote]

Tim, I’m not sure I would have figured out how to use this looking at the docs the first time but I do know.

Thanks

10

Personally I’d use an SQLite database and write it as a Double into the db. That way you get back in what you wrote out with no conversion issues. SecondsFrom1970 is what you want.

11

I do store a number of arrays as text files. Would you consider SQLite an easier way to store and retrieve data (years worth of climate information) and pref files overall?

12

SQLite would be a much better way to store the info, but balance that against another learning curve. Maybe get your app working using the text files and then think about converting to sqlite. One major change at a time. (And once you get the app stabilized as is, the leap to sqlite won’t be that bad. It could be a major battle if you try to change mid-rewrite.)

13

Good point Tim. Keeping frustration manageable at this time is good or I’ll never get through this. The curve is still very steep.

14

I use Format with “################0” otherwise the String returns blank for zero. I fell foul with Str() when it returned large integers as “1.23e7”.

15

i using JSON
(2019r3.1)

Public Function Save() as JSONItem
  // Part of the IO interface.
  
  Var j As New JSONItem
  j.Compact = False
  
  If Me.Birthday = Nil Then 
    j.Value("Birthday") = "Nil"
  Else
    j.Value("Birthday") = Me.Birthday.SQLDateTime
  End If
  j.Value("FamilyName") = Me.FamilyName
  j.Value("FirstName") = Me.FirstName
  
  Return j
  
End Function

[code]Public Sub Load(j as JSONItem)
// Part of the IO interface.

If j.Value(“Birthday”) = “Nil” Then
Me.Birthday = Nil
Else
Me.Birthday = DateTime.FromString(j.Value(“Birthday”))
End If

Me.FamilyName = j.Value(“FamilyName”)
Me.FirstName = j.Value(“FirstName”)

End Sub
[/code]

[code]Public Sub LoadFrom(f As FolderItem)
Clear

Var data As String = “”

Var st As TextInputStream = TextInputStream.Open(f)

data = st.ReadAll

st.Close

Var j As New JSONItem(data)

Load(j)

End Sub
[/code]

[code]Public Function SaveAs(f As FolderItem) as Boolean
System.DebugLog “Save As”
System.DebugLog f.NativePath

Var data As String = Save.ToString

Var st As TextOutputStream = TextOutputStream.Create(f)

st.Write data

st.Close

Return True

End Function
[/code]

example of write an object array

[code]Public Function Save() as JSONItem
// Part of the IO interface.

Var j As New JSONItem
j.Compact = False

'-----------------------------------------------------------------------

j.Value(“EntireStaff.Count”) = EntireStaff.Count
Var temp2 As New JSONItem
For i As Integer = EntireStaff.FirstRowIndex To EntireStaff.LastRowIndex
temp2.Add EntireStaff(i).Save
Next
j.Value(“EntireStaff”) = temp2

'-----------------------------------------------------------------------

Return j
End Function
[/code]

example load an object array

[code]Public Sub Load(j as JSONItem)
// Part of the IO interface.

'-----------------------

Var jEntireStaff As JSONItem = j.Value(“EntireStaff”)

For i As Integer = 0 To jEntireStaff.Count - 1
Var st As New Module1.Staff
st.Load jEntireStaff.Value(i)
EntireStaff.AddRow st
Next

'-----------------------

End Sub
[/code]

16

Good thought David. In my case if the date is blank I default to the current date. But I will use this thought for other situations.

17

[quote=495326:@Markus Rauch]i using JSON
(2019r3.1)

Public Function Save() as JSONItem
// Part of the IO interface.

Var j As New JSONItem
j.Compact = False

If Me.Birthday = Nil Then
j.Value(“Birthday”) = “Nil”
Else
j.Value(“Birthday”) = Me.Birthday.SQLDateTime
End If
j.Value(“FamilyName”) = Me.FamilyName
j.Value(“FirstName”) = Me.FirstName

Return j

End Function

Public Sub Load(j as JSONItem)
// Part of the IO interface.

If j.Value(“Birthday”) = “Nil” Then
Me.Birthday = Nil
Else
Me.Birthday = DateTime.FromString(j.Value(“Birthday”))
End If

Me.FamilyName = j.Value(“FamilyName”)
Me.FirstName = j.Value(“FirstName”)

End Sub

Public Sub LoadFrom(f As FolderItem)
Clear

Var data As String = “”

Var st As TextInputStream = TextInputStream.Open(f)

data = st.ReadAll

st.Close

Var j As New JSONItem(data)

Load(j)

End Sub

Public Function SaveAs(f As FolderItem) as Boolean
System.DebugLog “Save As”
System.DebugLog f.NativePath

Var data As String = Save.ToString

Var st As TextOutputStream = TextOutputStream.Create(f)

st.Write data

st.Close

Return True

End Function

example of write an object array

Public Function Save() as JSONItem
// Part of the IO interface.

Var j As New JSONItem
j.Compact = False

'-----------------------------------------------------------------------

j.Value(“EntireStaff.Count”) = EntireStaff.Count
Var temp2 As New JSONItem
For i As Integer = EntireStaff.FirstRowIndex To EntireStaff.LastRowIndex
temp2.Add EntireStaff(i).Save
Next
j.Value(“EntireStaff”) = temp2

'-----------------------------------------------------------------------

Return j
End Function

example load an object array

Public Sub Load(j as JSONItem)
// Part of the IO interface.

'-----------------------

Var jEntireStaff As JSONItem = j.Value(“EntireStaff”)

For i As Integer = 0 To jEntireStaff.Count - 1
Var st As New Module1.Staff
st.Load jEntireStaff.Value(i)
EntireStaff.AddRow st
Next

'-----------------------

End Sub[/quote]
Thanks for the detailed explanation, Markus. I have saved your example. This project is large and I find with my limited programming skills the learning curve is overwhelming at times and I get bogged down. By the time I finish rewriting this project, I will have gained enough knowledge to revisit this and maybe do some restructuring. I have stored over 20 years of data on energy consumption and how it’s affected by the homes passive solar gain element but have not analyzed the data in a meaningful way. It is intriguing to see how the passive element integrates well with the hydronic heating and mass storage system for passive gain.

18

if you can also use binary instead of text (BinaryStream)
something with name & value is maybe better than this convertings from value to string and back to value.

19

[quote=495389:@Markus Rauch]if you can also use binary instead of text (BinaryStream)
something with name & value is maybe better than this convertings from value to string and back to value.[/quote]

My original control system finished in 2001 used Binary pStrings and they worked great. I had this lengthy discussion on the forum before restarting the rewrite and a few people encouraged me not to use them this time. I admit it is easier to troubleshoot a text file but I did like not having to switch between input and output streams or converting to strings.

20

Note that with pString, you are still converting to/from a text format. I think Markus is suggesting writing UInt32 values, which would require zero conversion, but would be not human readable in the saved file. If you’re going to save to a file on disk as opposed to an sqlite database, I would recommend sticking with text format. There is little problem converting to/from integer representations. Encodings don’t come into play and as long as you’re using Format instead of Str, you should be good. If you’re going to take the next step in data storage, I’d recommend switching directly to SQLite once you have your current implementation working.