Losing text on TextInputStream

General
1

I have an HL7 message, which is a simple text file. It consists of several ‘segments’ delimted with Chr(13). I’m trying to read in the file and store it in an SQLite DB which has been created and encrypted with Xojo. This is a Windows service application. The variables have all been declared properly.

Var InBoundFileStream As TextInputStream
Var InBoundFileFolderItem As FolderItem // (assigned to passed parameter = the FolderItem of the file to be read)
Var InBoundFileText As String

InBoundFileStream = TextInputStream.Open(InBoundFileFolderItem)
InBoundFileStream.Encoding = Encodings.UTF8
InBoundFileStream.BytePosition = 0 // trying this makes no difference
InBoundFileText = InBoundFileStream.ReadAll

The file message text: (PHI removed)
MSH|^~&|MIL7|sender|reciever|site|20210311117||ORM^O01|Q276934T31967545|P|2.3||||||8859/1
PID|1||00048517^^^ST01A^MR~0941^^^ST01^PI||ZZTEST^STCHARGESTWO||19806|F||3|3533 DRIVEWAY^^A CITY^STATE^79343^US^^^1||^PRN^PH^AN EMAIL~512287401^PRN^PH||E|X||200233^^^Account Number^FIN NBR||||1|||0|||||N
PV1|1|E|RRR IP^^^A^^^A|2|||7777 -PHYSICIAN, ED^7777 -PHYSICIAN, ED|||ERY||||2||||3|69331^^^TEMP FIN^VISITID|SP|||||||||||||||||||A||A|||202025aadf00
ORC|SC|874527^HM_ORDERID|||CM||||2021031166|^LNAME^FNAME|||||2021035316|||Written^Written|^LNAME^FNAME
OBR|1|871227^M_ORDER||LEOP^LE Arterpler|||20210311313|201031316|||||||Rad Type&Rad Type|||||00322U34564^HNA_AN~501^HNA_ACID~1640^HN_PAID||2021065316||RA|||1^^0^2021750^^R||||^PAD|||||||||||||||CD:27^Radiology^^RA^Radiology

  1. Examining the FolderItem in debug confirms the entire message has been read.
  2. A code check for Left(3) = “MSH” passes.
  3. Print(InBoundFileText) to the console truncates the MSH segment and prints from the PID segment.
  4. Using standard DatabaseRow and adding this as a record to the DB also truncates the MSH segment.
  5. Examing InBoundFileText right after the ReadAll via Print(InBoundFileText) = no MSH but the code check for MSH passes.

Latest Xojo, Windows 11, working/running in Administrator account.

Why am I losing the MSH segment text? Did not have this problem on MacOS.
Would greatly appreciate any advice, insight, comments, etc. Many thanks in advance.

2

Does it work any better if you use this, it shouldn’t but…

InBoundFileStream = TextInputStream.Open(InBoundFileFolderItem)
InBoundFileText = InBoundFileStream.ReadAll( Encodings.UTF8 )
3

Nope, does not make a difference, thanks!

4

What is the character at position 4? maybe it’s a chr(0) or some other character that is interfering with your debug output

looking at the format for hl7 files the segment delimiter is 0x0D Marks the end of each segment.

5

Is it perhaps the length that is causing an issue. Try:

Print Left( InBoundFileText, InBoundFileText.InStr( Chr( 13 ) ) )
6

Just as a test, what happens if, after reading the contents of the file, you replace the line endings with something that doesn’t otherwise appear, like ~?

InBoundFileText = InBoundFileText.ReplaceLineEndings( "~" )

Will that write to, and read back from, the database in its entirety? If so, it simply doesn’t like CR on Windows.

7

The ‘pipe’ character is at 4. “MSH|^~&|…”. Have never had a problem with these characters. Chr(13) has always worked for me; some literature says 0x0D, others Chr(13). All segments are delimited with the same and the remaining message segments come through using them. I was wondering about that too, perhaps Xojo in Windows reads differently but there’s no consistency with the problem. Thanks!

8

Interestingly, this only return the MSH segment, but this confirms it’s there. It just seems to disappear for Print and the DB storage.

9

I suspect it is to do with chr(13) being the line delimiter. On Windows chr(13) moves the cursor to the start of line but doesn’t move it down a line. It is then overwritten by the next line. Do the following:

print InBoundFileText.ReplaceLineEndings( EndOfLine.Windows )

I suspect it will all display correctly?

Mac historically used chr(13) as line endings.
Linux and macOS X or above uses chr(10)
Windows actually uses chr(13) followed by chr(10).

  • chr(13) go to start of line.
  • chr(10) go down a line.
10

Bingo!, almost. Doing the replace line endings it now returns the entire message. In practice can’t use the tilde as it’s a designated HL7 character but that’s beside the point. This at least points me in the right direction. Kudos!

1 Like
11

And the winner IS… EndOfLine.Windows
That works perfectly!
Thanks everyone. Might get this done on time now.

12

For the record 0x0D is chr(13), it is Hexadecimal notation.

It is a display issue on Windows only. HL7 says use chr(13) then stick to that.

13

Agree, thanks Ian.

14

You’re not losing any data. It’s just a display issue on Windows.

15

So chr(10) alone permits to print text one line below at the same x position than the previous line? A weird concept.

16

That was the original concept back in the teletype days

2 Likes
17

I expected this. But dos it make sense these days?

18

Yes That’s why ReplaceLineEndings exists. If you’re doing a single platform, then it doesn’t matter, but if you’re doing cross platform, then it’s essential.

1 Like
19

This is a really important message. I also had to learn the hard way, that as soon as you have to process Strings from various platforms, ReplaceLineEndings and encoding knowledge are a must :slight_smile:

2 Likes
20

Not only !

When I work with text I created with Apple’s TextEdit application, I have to use ReplaceLineEndings too (on macOS !!!)